Unizin Reports
Winter 2024
This report is to be shared by design to communicate the full scope of Unizin impact, work products, and status at your institution.
Reflections on 2023…Bart Pursel
With 2024 off to a great start, now is a good time to revisit some of the major accomplishments from last year. The year’s biggest headline is that Unizin added two new members: the University of California, Irvine and Chapman University! We are thrilled to welcome our first schools from California and also excited that both are already in production with their respective Unizin Data Platforms (UDP). Speaking of the UDP, much of 2023 was focused on our migration from the original Canvas data stream to Canvas Data 2.0 (CD2). Migrations began in February, with the goal of having all our members migrated by the end of April. More information about CD2 can be found in the story below, as well as the initial release of our synthetic data set, another major accomplishment last year.
In the quest to create the “largest learning laboratory” in the form of the UDP, Unizin added two new data integrations in 2023. Packback, a robust, AI-powered discussion application with growing adoption across Unizin members now has event data flowing into the UDP. RedShelf, the eReader that powers Unizin’s content program, also sends event data to the UDP. These two applications represented 28 million events in 2023 sent to the UDP that, before 2023, were data blind spots. Work continues with the University of Minnesota to send events from Leganto, a Library resource list application, to the UDP, which we hope to have in place for all members later in 2024.
Data marts continue to be a strategic direction for Unizin, where the Unizin community come together and identify high-value data and information that often represents student activity. Unizin staff works to pre-process these data into resulting focused marts. The goal of data marts is twofold: decrease the technical resources required at member institutions to wrangle and process data from the UDP while simultaneously increasing the value of the UDP by providing high-value information that is easily accessible. Two data marts of specific note that arrived in 2023 include:
- Student Activity Score – inspired by Indiana University’s Canvas Activity Score, this is an activity analytic designed to take data elements about assignment and session length and calculate a representation of activity for every student, which is then used in proactive advising practices. Matt Rust and Ben Motz have a pre-print of a research article exploring the score’s efficacy, with positive findings such as a decrease in the number of students that receive a D, F, or withdraw, and an increase in student persistence to the following semester.
- Student Success Weekly Distribution – this mart takes individual student data found in the level one student success mart, and aggregates data to the class level. In addition to providing weekly rollups of different types of activity (assignment activity, discussion activity, and sessions activity), this mart also calculates ‘average’ activity in every course, which some members have found value in comparing student weekly averages to a generic course average, which can help identify students that might be struggling early in a semester.
Towards the end of 2023, Unizin conducted an inventory of applications and dashboards sourcing data from the UDP. We were able to identify 15 applications or dashboards originating from seven different Unizin members. While more applications likely exist, this is a good sign and speaks to the maturation of the UDP. What started as a platform to support various research and development efforts is now beginning to support production-level applications and workflows.
On the content side of the house, the Unizin Engage program, which started in 2017 and provides eTexts and Courseware at reduced prices, has reached over $100M in student savings! Seven of Unizin’s 14 members leverage the content program, with growing interest from current and potential members as affordability continues to be a critical component of student success efforts.
The final item for reflection (and celebration) is the Unizin team. From late 2021 to mid-2022, Unizin experienced significant turnover, with nearly half of Unizin staff leaving for new opportunities. In 2023, no Unizin staff member left the organization for new opportunities, and everyone is incredibly excited about how well the team is working together to support our members’ student success efforts in 2024 and beyond!
Bart Pursel
Interim Chief Executive Officer
CD2 Migration has Arrived
When Canvas Data 2.0 (CD2) arrived for testing in early 2023, Unizin began its planning – anticipating challenges unique to member instances, gathering the impact on member production services, determining what was unique about the CD2 schema and what aligned easily with the Unizin Common Data Model (UCDM), and timelines and allocation of resources based on ongoing member requirements and internal engineering priorities.
Unizin began mapping the UCDM to the Canvas Data 2 schema in March 2023. Since that date, almost all of Unizin’s personnel have dealt with the daily tasks of building and testing the CD2 pipeline, standing up test CD2 environments, and, most time-consuming given some of the irregularities early in the data, but also in light of CD2 being in many ways apples to CD1’s oranges – they are decidedly different representations. Validating against CD1 production environments presented many challenges, particularly around LMS identifiers. This required coordination and collaboration with Instructure, sometimes requiring Unizin to preprocess data or Instructure to change aspects of the data from the source. This iterative process with Instructure led to positive results, though we also found the need to extend the original migration timeline of late 2023 to early 2024.
This collaboration also demonstrated Instructure’s commitment and willingness to partner daily with Unizin and work through issues that would lead to a common good: for Instructure to successfully transition from CD1 to CD2, and for Unizin to stand up CD2 infrastructure, validate and report the results to members, and transition to CD2 during a designated nightly pull of Canvas data.
In late 2023, Unizin began working with Penn State and the University of Michigan to validate existing CD1 data in production with modeled (UCDM) CD2 data in a staging environment.
“We began working with both Penn State and the University of Michigan given their investment in and reliance on institutional production services of Elevate/Course Insights and My Learning Analytics (MyLA), respectively,” reflected Bart Pursel, interim CEO of Unizin. “Given the analytics teams they had and the applications in play, we felt confident that if we could check most if not all boxes with these two members at the start of migration, we would be in a good position to move efficiently forward with our other members.”
On February 13th, Penn State became the first Unizin member to move their production UDP environment to CD2. The University of Michigan migrated on February 26th, and Chapman, given they are beginning their UDP with CD2, went into CD2 production on February 27th.
With successful ingests of CD2 data into these members’ UDP, Unizin Engineering, Services, and Data Services and Solutions are working with the next group of members to migrate, with the goal of migrating all members by the end of April 2024, if not sooner.
“The value of the UCDM has never been more apparent than during this transition from CD1 to CD2”, said Pursel. “The UCDM is similar to a data insurance policy, shielding our members from the significant re-work that needs to happen when rebuilding queries, code, and pipelines to account for the enormous differences from CD1 to CD2.”
Special thanks goes out to all of the Unizin team members (Engineering/DevOps, Services, Data Services and Solutions, Product), who played an invaluable role in making this substantial project come to fruition – from architecting the pipeline, validating and reporting, working with members to resolve data issues and discrepancies, to building the infrastructure, they are all an amazing and dedicated group of professionals! Thank you to the incredible personnel across our member institutions who worked closely with us (and were patient in the process!) to move this important step in Unizin’s growth forward!
Synthetic Data Gains Traction
Unizin continues to develop and build on synthetic data as a multipurpose tool for analytics and investigation of learner and learning process data found in the UDP. Beginning with the University of Colorado Hackathon on October 26th and 27th, 2023, Unizin assembled its first trial data set for exploration. In late November, Unizin again provided synthetic data sets for the University of Michigan for their “MHacks” three-day event for students.
As awareness and comfort levels around synthetic data grow, more members are becoming interested in the potential of synthetic data. From the University of Florida’s interest in utilizing synthetic data for analytics assessments during interviews, Indiana University’s interest in utilizing synthetic data for model testing and exploration, to a brainstorming session at the University of Minnesota, where the Learning Analytics and Learning Platform Groups came together to explore synthetic data with a focus on the following areas:
- Increase access to data, from few to many
- Growth opportunities – data reporting, visualization
- Prototype with synthetic data first, real data next
- Governance can work on synthetic data projects between meetings
- Increase Involvement of colleges and campuses to understand trends and support their organization by getting staff involved
This special session provided University of Minnesota stakeholders and Unizin Data Services and Solutions breakout opportunities to answer questions about the distributions informing synthetic data, join University stakeholders in brainstorming its potential, and “imagine the possible” relative to safely exploring analytical methods and models.
During the February Unizin Town Hall, Data Services and Solutions discussed updates and the next steps for Unizin’s synthetic data offering. UDP data marts can now be created using synthetic data, providing even greater flexibility around testing, such as proof of concept analytics. Additionally, Unizin has moved to utilize the native GCP infrastructure to optimize the production of synthetic data. And finally, DSS has improved logic to create greater consistency across terms for course subject and number.
The next steps will involve defining yet-to-be-represented fields and entities to represent all of the UDP.
While synthetic data is fabricated based on representative or logical distributions across the consortium, data steward approval is still required to access synthetic data. This allows Unizin to create and provide members with specific access to a separate project space.
In terms of supporting documentation, all standard UDP documentation applies to synthetic data, given that the data follows the same schema as actual UDP data.
Please submit a ticket to support@unizin.org if you are interested in exploring or would like to discuss potential projects involving synthetic data.
Chapman UDP Implementation
Congratulations to Chapman University! As of February 27th, Chapman University has implemented their UDP (adopting Canvas Data 2 out of the gate).
While integrating and mapping Student Information System (SIS) data to the Unizin Common Data Model (UCDM) can be a lift for most institutions, Chapman did an amazing job aligning the right people and resources to move their UDP implementation forward. Extra special thanks to the Chapman team for being such amazing partners during the implementation process. An extra thanks to the Unizin Data Services and Solutions (DSS) and Unizin Services teams for their weekly meeting and problem-solving.
Security Program Planning for 2024
Many changes are afoot around Unizin’s security program and will continue through 2024. First, we began the year by promoting Cody Simpson from Senior Site Reliability Engineer to Director of Security and Cloud Operations. This new title underscores Cody’s major role in ensuring our critical infrastructure’s security, compliance, and operational integrity.
Second, we are searching for a new penetration test partner to replace REN-ISAC. Our plans are to finalize this search and resume our annual cadence of testing in Spring of 2024.
As part of our engagement with a potential Unizin member, we submitted security information to a state’s Recognize, Assess, Minimize, Prepare (RAMP) program for approval to host student data in the cloud. The process of aligning our security program to the RAMP protocol was time well spent, and we are excited to see what types of feedback come from this process that can continue to strengthen Unizin’s security program.
Parallel to our efforts in securing a new penetration test partner, we are looking to establish a plan, external guidance, and planning toward beginning formal cybersecurity compliance through a Systems and Organization Controls (SOC2) audit for our IT infrastructure in 2025. Our aim is to couple annual penetration tests with biennial SOC2 audits, allowing us to move far more expeditiously through new and existing member security and risk assessments.