Unizin Reports
March/April/May 2022
This report is to be shared by design to communicate the full scope of Unizin impact, work products, and status at your institution.
583 registrants representing 62 institutions from 27 states and 7 countries participated in our two-day event. While overall attendance was slightly lower than last year, the conversations generated through our keynotes, panel discussions and demonstrations were no less engaging and thought-provoking. It is abundantly clear that the value of Unizin learner data is being mined for real impact across our consortium and beyond. I want to thank everyone who shared their talents at the Unizin 2022 Summit. As always, we couldn’t have done it without the power of us!
For those of you who were unable to attend the Summit, all of the session are available on the Unizin YouTube Summit 2022 Channel. I highly encourage everyone to watch the replays of these engaging and informative sessions. With 29 sessions in total, here are two possible ways to compile your playlist.
Based on real-time participation, our Top 10 Sessions for the 2022 Unizin Virtual Summit were:
| 1. A (Show)Case for Research on Improving Student Success |
| 2. The Truth about Self-Care |
| 3. Enabling Student Analytics |
| 4. ASU: Using data insights to maximize inclusiveness and learning experiences for everyone |
| 5. Sponsor Speed Dating |
| 6. Leveraging Learning Data |
| 7. Data science and human centered design: a new toolbox for university-scale T&L problems |
| 8. Feedback Intervention: Personalized Feedback from Learning Data |
| 9. Building cross-institutional value: the Unizin Data Book |
| 10. LMS 2023: Imagining the Learning Management System of the Future with Instructure (not recorded at vendor request.) |
Although I found every session interesting and informative, this is my list of personal favorites:
- Feedback Intervention: Personalized Feedback from Learning Data (Adama Brummett, Chemistry Lecturer; Anna-Marie Smith, Sr. Analytics Specialist; Sara Nasrollahian, Asst. Director Center for Teaching and Jae-Eun Russell, Director, R&A at the University of IA) This presentation provides a multitude of perspectives on the implementation of personalized feedback, based on the Unizin Data Platform, on large-scale Chemistry classes at the University of IA.
- Using Analytics to Optimize Student Success at the University of FL (Aaron Thomas, Sr. Data Scientist at UFIT and Tobin Shorey, Director Curriculum and Policy UFL) This session looks at how UFL is solving the critical need to identify students for coaching and tutoring ethically based on both predictive analysis and past performance statistical analysis. Furthermore UFL is using Unizin data to identify trends in student access including graduation, barriers to success in high DEW courses, major yield and other areas
- Leveraging Learner Data (Christina Bifulco @ Rutgers, Alexandra Bitton-Bailey at UFL, Gwen Gorzelsky @ CSU, Jae-eun Russell @ IA and Damji Stratton @ MO System) A sneak peak into the new Unizin data marts from the strategic taskforce focused on learning analytics for student success. This is a facilitated discussion on how best to utilize these new data marts and the opportunities that they present with faculty and instructional staff. The focus is on how best to use these new data marts ethically, equitably and effectively.
- Cross-Institutional Transfer Learning for Educational Models (Josh Gardner @ UWA; Rene Kizilcec @ Cornell; Renzhe Yu @ UCI; and Christopher Brooks at UM) This session seeks to create a dialogue and present recent research on cross-institutional learning – the use of predictive models trained on one instution, deployed at another.
- Enabling Student Analytics (Holly Drake & Jordan Klapper @ Ohio State’s Privacy Office) This session focuses on the ethical use of data, student analytic principles and what it means to apply those principles at OSU. In this latter part, this session runs through several common scenarios and demonstrates how to apply the student analytic principles in those situations.
- Responsible Learning Analytics (Kim Arnold @ UW – Madison; Marcia Ham @ Ohio State and Robin Pappas @ Oregon State). This interactive presentation is a facilitated discussion that asks, “What are the implications for Higher Education in this data rich environment ethnically?”
- ASU: Using Data Insights to Maximize Inclusiveness and Learning Experiences (Matt Rhoton, CTO and Jonathan Carroll, Director of Learning Insights @ASU EdPlus; Jesus Gomez, Sr. Strategic Business Executive and Jess Masciarelli, Account Manager at Google Cloud for Higher Education) If you’ve wondered what’s going on with the ASU EdPlus pilot of the Unizin Data Platform, this is your chance to hear it first hand, “Unizin is providing us a near turn-key solution to standardize our multi-LMS environment within our own Google Cloud organization that enables us to focus our efforts on further data enrichment and activation.”
Cathy O'Bryan
Chief Executive Officer
Emerging Use Cases for the Unizin Data Platform: Athlete Assessment and Accreditation
Learning outcomes assessment and accreditation are unique challenges, and vitally important across our institutions. Learning outcomes assessment requires a large commitment across faculty and staff from many different areas of a university. First, outcome parameters need to be defined across these programs, then specific learning objectives from within courses need to be mapped to the program outcomes. Once course learning objectives are defined, the course then needs to be designed in such a way that everything maps back to the objectives, particularly the content and associated assessments from that course that are designed to measure each learning objective.
This is challenging work, and work that is required as different programs or entire universities prepare for visits from accreditors for reaccreditation purposes. In my experience this process can sometimes feel like a rollercoaster. As the reaccreditation process approaches, a surge of activity begins, bringing diverse stakeholders together that need to collect and analyze diverse outcomes data across several cohorts of students. Then reports need to be generated that align with what the accreditors require for review. Assuming all goes well and the program or university is reaccredited, everyone takes a moment to celebrate and then returns to the work they sometimes had to abandon temporarily to meet the needs of the reaccreditation process.
I’m convinced the UDP can help in reaccreditation efforts. Even more importantly, the UDP can help the folks on our campuses who think about learning outcomes assessment every day. Together we need to work hard to systemically move the needle towards a culture of perpetual learning outcomes assessment independent of when an accrediting body is scheduled to visit campus.
-Bart Pursel
Unizin Board Member Changes
Strategic Taskforce Update: Engage eReader Replacement Project (RedShelf) Update
Strategic Taskforce Update: Learning Analytics Update
The discussion centered on a recap of the Taskforce goals, and some history on how we distilled the uses cases from each taskforce in to a plan or “matrix” based on use case themes and levels of analysis. Working with a small working group (Gwen Gorzelsky at Colorado State, Lauren Marsh at the University of Minnesota, and Cid Freitag at the University of Wisconsin), the DSS team has focused on realizing 9 core data marts for level 1 analytics (at the student level) for all three use cases (impact of course activities, measures of activity/engagement, and student profiles). Course and curriculum aggregations (level 2 and 3) will naturally follow from the foundational data exposed in the level one data marts.
Functional, “proof of concept” data marts already reside within the Colorado State University production UDP environment, and include:
- Course_profile
- Enrollments_grades_courses
- File_accesses
- Level1_aggregated
- Level1_assignments_submissions
- Student_term_profile
- Time_spent_view_days
- Tool_launches
- Weekly_discussions
Next steps will include folding some specific Taskforce-driven use cases into new “self-guided” bootcamp training that DSS is developing and will deliver to Colorado State University at the end of June. DSS will then reconvene both Learning Analytic Taskforces, provide an overview of the data marts produced, and provide access to the data marts within member BigQuery production environments for feedback in mid-July.
Unizin Involvement in Canvas Data 2.0 Alpha Program
Possibly the most exciting aspect of Canvas Data 2.0 is the improvement in data latency. Currently, the batch data can be up to 48 hours old. This presents a challenge when, for example, advisers and teachers are using calculated Canvas data representations to inform if, when, and how they reach out to a student. In some instances, a concerned teacher may reach out to a student about a low overall grade when the student received an A on an exam the day prior. Not having that A included in the data can sometimes misrepresent things to whomever is using the data to inform decisions. Canvas Data 2.0 is designed to be refreshed every 4 hours or less, which allows for a much more accurate, near real-time representation of activity when we provide different data views to end users.
Instructure provided an informative comparison table, highlighting some of the differences between CD1.0 and 2.0.
Course Insights: Illustrating the Value of Unified Learning Data in the UDP
The theme of unification continues to come up in discussions here at Unizin and with members throughout the consortium, as it’s a core strategy for the success of the UDP. We envision creating the largest, unified, learning data platform in higher education, what we’re really talking about is ensuring our data is unified; specifically, the data related to students and courses. With an ever-growing digital learning ecosystem, and more faculty members continuing to use digital tools adopted during the pandemic, it’s vital to unify all this data. For example, if a student takes a course that leverages the LMS and two additional learning tools, we often only had data from the LMS to help us understand the digital learning experience. Sometimes we had data from the other two tools, though without a way to link the student’s identify across the tools via unification, we can’t understand and assess the student’s complete digital learning experience.
We use a process called enrichment in the UDP, where we have technology that examines events as they come into the event pipeline, looking for a unique identifier from the UDP’s context store. For example, if an event from Top Hat is queued to move into the UDP, our enrichment process looks at the event, and within that event sees a Canvas LMS student identifier. Perfect! We can now enrich that event, providing an identifier that allows us to recognize that student across two digital learning environments: Canvas and Top Hat.
At Penn State, there is an instructor-facing learning analytics application known as Course Insights (similar to Elevate, an advisor-facing learning analytics application). It all began by using Canvas event data to calculate a rolling 7-day Canvas activity score for every student in every course. This score is then used in various comparisons within courses to provide new data to instructors that inform if, when, and how they reach out to a student. This spring semester the team was able to incorporate Top Hat data into Course Insights, meaning now the activity score is no longer only predicated on Canvas data, but is coupled with Top Hat data. Please note the addition of Top Hat data in the Course Insight screen shot below:
This approach provides a better, more cohesive view of the student’s digital learning experience, particularly in courses that leverage Top Hat. As we enrich more data coming into the UDP, our hope is that our members can continue to pursue these types of projects, generating a better view of students’ digital learning experiences that can help inform important decisions around teaching, learning, and course design. Next up: Unizin is working hard on enriching Kaltura data within the UDP.
Highlights from Data Services and Solutions Member Engagements
- IU: There is ongoing efforts to validate and improve the IUPUI (Indianapolis) Student Activity score. Together Unizin and IU are close to a UDP-based solution but just need to iron out the right mix of assignments for cross-listed courses.
- UFL: Jose Silva-Lugo’s student performance Machine Learning project fueled by UDP is nearing completion. We performed a final validation of enrollments and scoped GPA calculations for his feature extraction, and Jose is starting the process of submitting paper proposals.
- CSU & Unizin Consortium: -Work has begun on a live, self-service training portal for the UDP to replace the in-person Bootcamps of last year. We are nearing completion of version 1 of the site for late June, and the CSU team will be the first to test run it. Get ready for its proliferation across the Unizin consortium!
- UMN & Unizin Consortium: Based on the initial feedback on the Unizin Data Book presentation from Summit, Unizin has begun talking to interested member institutions. At UMN, Alison Link and some UMN team members are interested in joining our focus groups as we begin forming the Data Book. Alison has provided the initial feedback on analytics/use cases that would be interesting and approved by UMN. More to follow soon.
- PSU & Unizin Consortium – As you would expect, data modeling around enrollments and tool integrations are a high priority for our engagements. Together with PSU, we are working on adding new fields and exposing more source fields around course enrollments, and TopHat and Kaltura have active engagements around the events and data we receive.
- UMich and MyLA member Users: MyLA is nearing the completion of its migration to the UDP from the Unizin provided data warehouse! A new feature in the next UDP MyLA release includes a new schema called “report” with a table named “publish_info” that contains the last piece of info the MyLA team needs to finish. We are targeting a fall release this version.
- CSU & Unizin Consortium: Currently, the prototypes of the Learning Analytic taskforce datamarts exist in CSU’s GCP environment. We’ve been working with the Taskforce team (Gwen, Lauren, and Cid) to translate last year’s taskforce deliverables into tangible tables that can be tested. Round 1 of these tables will be tested by the CSU team late June, early July.
Unizin Data Book - Summit Brainstorming & Reflections
Key takeaways from our presentation and the ensuing discussions focused on the following:
- Applicable to multiple audiences – Data aggregated at the level of the consortium allows us to represent the richness and utility of the UCDM (the Unizin Common Data Model which is the means by which UDP data is normalized, modeled, and interrelated) while expressing meaningful data on learners and learning environments over time, representing 3.1M students, ~5.9M courses; and ~ 26B events. It would allow members to see the application and the impact of their data in clear and straightforward framework. And that leads to a starting point for academic and institutional researchers who may be new to the UDP and who would benefit from representations of how the UCDM is put into practice.
- Richly visual – We want the Unizin data book to be as much a catalyst for researchers as it is a compelling example of what is possible for prospective members.
- Consortium level insights leading to deeper investigations – surfacing what technologies (mobile or desktop for example) students use to access their courses in the LMS and the mode’s potential influence on their work opens the door for deeper investigation at the institutional level.
- Thought leadership – Provide a glimpse of what is possible with modeled learner and learning environment data presented through the data book and the UDP. Potential key indicators of the state of higher education – seeing the data book as a unique, UDP-driven reflection of the state of higher education among a group of R1 institutions.
- Thought leadership behind the modeling of learning analytics – This will include how our methods, data modeling and features are selected and used in the data book.
- Multiple formats– Static and interactive formats – as a pdf in addition to a web interface that would allow a modicum of filtering and manipulation.
Central to the development of the Unizin Data Book, and discussed in some length, are the various layers of governance that would play a key role in such a collaborative effort are:
Unizin’s Data Sharing Agreement (DSA) and the Cross-Institutional Research workflow – The DSA acknowledges an institutions willingness to share their institutional data; and the cross-institutional research process operationalizes the agreement in giving institutions the opportunity to vet and review the data and methods involved.
Formation of a Committee comprised of member academic and institutional researchers – We want member institution researchers to drive our investigation into what are the most compelling features to provide; and we want institutional input and guidance on how we employ the very important elements of race/ethnicity/gender in particular.
Transparency and ethical considerations – We want to facilitate the use of the UDP through the data book – to make understanding what can be achieved through the UDP as clear as possible, with a clear focus on improving learner outcomes, to lead to actionable insights toward student success. We want to be careful with proportions and percentages, especially for enrollments; we want to avoid any institutional benchmarking; anonymize all data and tools through categorical aggregation – and not identify vendors (Video, grading tools, collaborative, feedback, etc.).
Unizin’s Data Services and Solutions team will be reaching out to member institutions in July to build a working group of researchers to guide our efforts in creating the Unizin data book.
Data Services and Solutions Quarterly Town Hall
One of DSS’s aims, relative to allotting time and resources to address all member needs, is to fold member standing meetings into these town hall sessions quarterly, and keep member meetings focused on particular projects with specific goals and timelines. In this way DSS can convey generalized information efficiently to all members, as well as schedule focused analytics project meetings when needed. Kyle and Sara are amazing in their productivity and expertise; but as members increasingly engage in the UDP (and as we anticipate new members with similar interests), we believe it is crucial that DSS work as efficiently and as equitably as possible in the support that they provide.
Our plan is to start the quarterly cadence of meetings in late July or early August, depending on summer schedules. These sessions will be 90 minutes in duration, and open to all Unizin members.
Visiting Colleagues at the University of Florida
Bart has spent some time with colleagues at Penn State on various learning spaces research in the past, and he was amazed at what he observed at the UFIT’s learning spaces control center, where they have a group of student employees monitoring via video and sensors a large number of classrooms on campus in order to proactively identify when something isn’t working correctly or if a faculty member requires assistance. Mark McCallister, Director of Academic Technology, indicated that sometimes a student employee is in the classroom, helping a faculty member fix a problem before the faculty member even has a chance to call the help desk. What an operation!
Brad Zurcher (Unizin Director of Partnerships) and Noah Spencer (Unizin Assistant Product Manager) are hitting the road this summer in the Midwest, and have visits scheduled at the University of Iowa, the University of Nebraska-Lincoln, and Colorado State University. We look forward to visiting all of the Unizin campuses soon!
Unizin Welcomes New Staff
Tom Legnard joined Unizin in March in the role of a data engineer. Tom’s background includes ETL/integrations, cloud infrastructure, data analytics, and application engineering. Tom is coming to us from a role at M&T Bank, and before that, he was working for GE Energy. Tom comes from a family of educators and found Unizin’s mission resonated with him. Tom is excited to meet members of the consortium and contribute to efforts that will have a positive and meaningful impact on our member campuses.
Milan Byrd joined Unizin in April in the role of product manager. Milan comes from Church & Dwight consulting, where she managed a team focused on SaaS implementations and delivering business intelligence and analytics solutions. Milan also focused on product management in this role, working with stakeholder groups to translate business and technical requirements into innovative solutions. She also received her Bachelor’s degree from one of our member institutions: Penn State! You can learn more about Milan below in our Staff Spotlight series.
Joining Unizin on June 20th will be three new engineers. Zach Wallace is coming to us from the Geospatial intelligence world, where he worked for four years as an analyst. Cody Stephenson joins Unizin after five years of working for a large government contractor. Cody’s roles included analyst, DevOps, and software engineering. Danny Sibley is joining Unizin from a background in physics and electrical engineering and is transitioning his career to focus more on DevOps and Python engineering, joining Unizin as a Site Reliability Engineer. Amanda D’Uva, currently a software engineer with Cigna, will be joining Unizin on August 1st as a Python engineer and brings with her various front-end experience as well. We are excited to welcome a great group of engineers this summer!
Unizin has a remaining project manager position open to complete our Unizin team.
Staff in the Spotlight: Milan Byrd
I have always been passionate about learning and continually push myself to acquire and share knowledge. My friends also affectionately call me a ‘Cyborg’ because of my love for and skill with technology. Unizin has represented the most fantastic harmony of things that are important to me. While alignment is excellent, the team has been the best part. We all believe in the mission and have the most fun together executing it.
My main focus at Unizin is enabling our members to thrive and facilitating that by helping to maintain our internal efficiency. This includes things like hosting brainstorming/user research sessions with our institutions, having meaningful prioritization conversations with our services and engineering teams, or dreaming up new and exciting features with the entire team.
When the workday is done, I leave the (virtual) office and come home to my amazing fiancé and exuberant four-year-old son. When not hanging out with them, music is my best friend. I love listening to it and creating it with multiple instruments, including my voice! I have been a vocalist in various capacities from a very young age, including my first love – musical theater. I even have had the great fortune of being involved in some off-Broadway productions! Thanks for ‘TUNE’-ing in to learn a bit about me and my experience at Unizin!