MINNEAPOLIS, MN (SEP 10, 2024) –
The Academic Technology (AT) group at the University of Minnesota (UMN) is a team of technology professionals charged with maintaining and managing the university’s learning management system (LMS), and partnering with faculty to use technology effectively to engage learners.
Since joining Unizin in 2014, UMN has successfully used the UDP to report on LTI usage to inform decision making in the colleges. The AT team’s goal moving forward is to expand the use of the UDP from operational applications to academic insights. To achieve these ends, the AT team formed a partnership with the University of Minnesota Carlson School of Management (CSOM). CSOM houses the Carlson Analytics Lab (CAL), a premier consulting service where graduate students from the MS in Business Analytics program leverage their expertise to tackle real-world data challenges.
Each semester, leading businesses and non-profit enterprises “pitch” data-driven projects to CAL students. Organizations like the Minnesota Wild Hockey team, the Mall of America and the Land O’ Lakes corporation have all sponsored projects in the past. Students select projects that appeal to them; student teams dedicate approximately 1,000 hours to address project goals and expand analytics capabilities.
In the fall of 2023, the AT team pitched its UDP analytics project with a unique appeal: a real-world scenario familiar to students everywhere. The AT team invited students to use the UDP to evaluate the impact of course design elements on student success. These student researchers had spent months engaging with digital coursework within Canvas and digital learning tools deployed in different ways by different instructors. Analyzing UDP data to improve course design and enhance the student learning experience was a professional challenge with personal resonance. The student researchers embarked on the first large-scale exploration of the University of Minnesota’s UDP data.
The process of developing and implementing an institution-wide UDP analysis offered valuable lessons to the AT team and the Unizin development team. These lessons will inform future UMN research projects and can be applied across the broader Unizin membership.
1. Creating a Safe Workspace for Graduate Student Researchers
In order to create a safe workspace for the students to conduct their research, the Unizin team created a copy of UMN’s UDP production environment. This mirror dimension differed from the UMN’s production environment in that it:
- was void of any identifiable student characteristics;
- contained spring and fall 2023 courses; and
- excluded classes with 30 or fewer students and all MSBA courses (those in the students’ graduate program)
Before graduate student researchers were granted access to this mirrored UDP environment, the university legal team reviewed the project to ensure data privacy and security.
In addition, the AT team established the following expectations for the use of Generative AI:
- The client permits the following:
- If the students felt that there was value in using generative AI tools and capabilities, we asked that they submit a proposal information about what tool the team planned to use and why they felt the need to use it.
- We ask that your team only leverage self-hosted AI tools.
- We ask that your team cite the use of AI tools and capabilities so it aligns with scholarly practices.
- The client prohibits the following:
- The use of cloud-based AI services. Even though the data is de-identified, it should not be supplied to any third-party cloud-based service because it violates Intellectual Property.
- Generative AI tools should only be used as a scaffolding tool and not a substitute for your own work.
2. Building Foundations
As the first focused effort to analyze UDP data for learning insights, the research team tested hypotheses and learned on the job. The simple questions posed within the initial project proposal proved to be complex for statistical analysis, given the sheer number of variables available in the UDP. However, conducting hands-on analysis was key to helping the AT team identify and resolve gaps in the data. Exploration of data is the foundation for conducting richer analyses. “The students’ work compelled us into deeper knowledge of our data, as a team we had to ‘reckon with’ how to best leverage the UDP as a data source” Jeff Weber, UMN Service Owner – AT Tools.
3. Setting Future Direction for the AT Team’s Research Approach
Given the broad research question posed and the number of variables available in the data, the AT team can learn from the results but can’t apply the findings. However, the findings do illustrate the promise of the UDP and the opportunity for future, more focused research. As a result of the experience, the team can go forward with more confidence and expertise.
Building on the insights learned from this project, the AT team has begun exploring a new data analytics and visualization service offering which includes the use of the newly available synthetic data offered by Unizin.
“Providing teaching and learning data from sources such as Canvas directly to students in the context of coursework presents an incredible opportunity to support various authentic learning opportunities,” said Bart Pursel, CEO of Unizin. “The amazing work from the students at Minnesota is the tip of the iceberg in how UDP data can support analytics, data science, and statistics courses. With Unizin’s synthetic data now in beta, all Unizin institutions can now access a completely fabricated instance of UDP data, that still maintains many of the statistical characteristics found within an institution’s live data.”
Unizin members interested in using synthetic data to support student coursework should email support@unizin.org to schedule a time to discuss their ideas.