Unizin Reports

January 2021

by Erik Goens | Feb 9, 2021

This report is the first issue of our new Board Report. This report is to be shared by design to communicate the full scope of Unizin impact, work products, and status at your institution. There are likely to be design changes as we iterate into this new format.

Unizin has a new exciting partner, Desire2Learn (D2L). Unizin’s goal of leveraging data interoperability standards to integrate learning tools, normalizing and consolidating data from various learning tools, the Canvas LMS and SIS, has been realized in the Unizin Data Platform. However, the adoption of standards across Ed Tech is uneven. D2L is a leader in this arena. This maturation of standards enables Unizin and D2L to partner and innovate. This partnership will provide Unizin with the opportunity to attract members that utilize other LMS’s, thus diversifying and enriching our data lake for large scale research and collaboration. D2L as an organization is focused-on and incentivizing research through the Unizin agreement that informs and delivers on their commitment to an equitable, adaptive, and easy-to-use learning management system. We are mutually excited about the exchange of ideas, constructs, and insights with our membership-based consortium. Together we can further student success, enforce and realize the promise of ed-tech standards to promote data-sharing, accessibility, and student success. Together we can expand the impact of Unizin member-based research and abilities.

The Unizin UDP is at an inflection point. I’m privileged to have ‘inherited’ it at this stage. Unizin’s initial, hard-won data platform architecture is perfect for ad-hoc queries and research on unstructured curated data. Over the last six to eight months, the architecture has evolved to support data services and data marts’ development and delivery. These are purposeful, intentional, and designed to solve concrete problems and answer specific questions. These include: What LTI tools are used by a particular curriculum or program, which courses use them? Which LTI tools does the institution use most frequently? Who? Where? When? Can we identify students that are not interacting with their courses? These data marts have been developed in partnership with our members. We see these data services integrating into the everyday work of business analysts in provost offices, institutional research, advisory, student success staff, faculty, and even students to further the ethical usage of learning analytics in the academic mission’s service. We can help leadership make informed, actionable decisions.

The Unizin 2021 Virtual Summit is scheduled for April 21st and April 22nd from noon until 4 pm Central Time. Registration will be open by March 1st. So please save the dates!

Please share your input regarding this report with me, @cathy.obryan@unizin.org Feedback is a gift.

Cathy O'Bryan

Chief Executive Officer, Unizin

Research & Innovation

Each month this section will report on selected research projects that Unizin supports

My Learning Analytics (MyLA): MyLA is a set of visualizations that promotes metacognition, facilitates self-regulated learning, provides students more profound insights into their course performance, and represents Unizin’s first software (SaaS) hosting.

Unizin can steward member institutions implementing the MyLA app;
Unizin is arranging a Board demonstration with Stephanie Teasley and Jennifer Love from the University of Michigan (likely scheduled in late March) and developing a MyLA onboarding “short-course” for Unizin member institutions.

Connecting Libraries and Learning Analytics for Student Success 2 (CLLASS2): The granted CLLASS2 project, funded through the Institute of Museum and Library Services (IMLS), seeks to connect UDP data to captured interactions with library resources to understand the impact between the library and student learning and use this information to guide students in their course work better.

A renewed (CLLASS2) grant proposal has been successfully submitted to the National Leadership Grants for Libraries Program (NLG) of the IMLS by Syracuse University. A full proposal is due March 26th, 2021.
James Russell and Etienne Pelaprat (Unizin) join Megan Oakleaf (Syracuse University), Anthony White (University of Michigan), Shane Nackerud (University of Minnesota), and Ken Varnum (University of Michigan), among many others as members of the CLLASS2 Advisory Board.

Learning Analytics Reflection and Awareness (LARA): LARA analyzes and quantifies grade penalties in STEM courses – a significant, multi-institution accounting of differences in grade outcomes based on gender and across five universities. Unizin delivers LARA in two parts: 1) the source code which lives in GitLab; and 2) data marts that live on top of their production UDP. The current project involves working with Stan Kruse to move the LARA platform from its instance in IU’s UDP to Colorado State University.

Online STEM Learning: Types of Interaction (NSF-funded): While STEM education research has focused on identifying pedagogical practices focused on understanding various student experiences, researchers are now well-positioned, given the current pandemic, to study a novel framework specific to undergraduate STEM student learning and retention. This framework utilizes Social Network Analysis and a Communities of Inquiry framework.

Unizin is working with Tracie Reding (University of Nebraska) to plan the next step in this NSF grant proposal, utilizing cross-institutional UDP data. Unizin is working with Jyotsna Ramanan on building data views and marts toward social network analysis.

This effort will pilot Unizin’s new Facilitating Cross-Institutional Research process and documentation as a means to test the cross-institutional data sharing process, as well as steward institutions in secure and appropriate data security and privacy concerns around shared UDP data.

Research and Innovation Continued

Elements of Success (EoS): Elements of Success (EoS) is a learning analytics platform that helps students achieve their desired course outcomes by providing real-time feedback on their performance. Developed by the University of Iowa Office of Teaching and Learning Technology (OTLT) in close collaboration with University of Iowa faculty members (Jane Russell and Ji Guo, leads), EoS provides students new insights into their course progress.

- Integrating the Digital Learning Scorecard – DLS at the University of Iowa is a predictive model for student success and has reached over 90% accuracy using the first six weeks of term data. Engaging in Weekly meetings, Unizin has partnered with Ross Miller (University of Iowa) on a project (Digital Learning Scorecard) to complement the Elements of Success tool, exploring the possibilities of grade predictions using ML techniques on in-term UDP data.
- Retention and persistence of freshman and sophomore students are critical financially to institutions. DLS focuses on the course grade/risk of DFW predictors in a way that doesn’t rely on historical data. Other course grade predictors weigh cumulative weight cumulative GPA heavily. Sophomores and freshmen, representing high-attrition groups, have little to no cumulative GPA and must be seen relative to specific measures. These measures include the amount of time spent in canvas each week, the number of days each week they log into Canvas, assignments submitted, and LTI tools launched.

Innovative Datamarts and Dashboards: In late Fall, Oregon State University and Unizin began collaborating on UDP-based marts and dashboards that focused on solving concrete Oregon State University use cases with real-time Canvas data. Our first use-case was to provide operational insights into “course readiness.” Unizin delivered a mart and dashboard a week before the Winter quarter began, giving Oregon State University near real-time analytics that helped ensure all Canvas-based courses were published on day one.

From there, Unizin and Oregon State University moved on to other marts and dashboards iteratively and collaboratively. Together, we developed near-real-time marts and dashboards to understand LTI tool use and identify students who are not interacting in their courses. Oregon State University involved a multi-disciplinary team that included: IT, ECampus, and institutional researchers. The coordinated effort dramatically improved the speed, efficiency, and quality of the work.

For Unizin, this collaboration enabled the refinement of the datamarts’ features that will be released across the consortium as standard UDP features. Moving forward, Unizin will seek to reproduce the great experience it has had with Oregon State so far by involving more institutions in the creation of data marts.

Research Support

Bootcamp 2.0: Unizin looks to expand Bootcamp offerings to be both institution and role-based specific. In this way, Unizin can work with and address some of the unique concerns that service owners, for example, might have in accessing and using their UDP instance productively. Future bootcamps will address the needs of faculty researchers through an R-based curriculum.

Analysts, developers, and service owners from Colorado State and IU came together on January 21-22 for the UDP Bootcamp. During the Bootcamp, the twelve individuals from CSU and IU wrote real UDP queries on UDP data covering topics including course enrollments, quiz results split by demographics, LTI launches by person and course, and dwell time for students in Canvas. For more details, see: https://resources.unizin.org/display/UDPB

RStudio and Shiny Server: Working closely with Casey Negen, Chad Brassil, and Jyotsna Ramanan (University of Nebraska), Unizin provides pilot project access to both an RStudio server and Shiny server to facilitate University of Nebraska’s statistical modeling needs around RStudio visualizations and centralized reporting (Shiny server).

Ongoing innovation and data implementation support

Using hand trained ML models on UDP Data, Aaron Thomas (University of Florida) has developed a predictive student risk/grade analysis process. A selection of 37 key courses was identified, and each course has a trained model (advisors as the targeted audience).

Working with Lynn Greenbough, David Goodrum, and David Rios (Oregon State University) to deliver and Canvas-driven and UDP dependent dashboard, allowing various stakeholders to assess and determine course readiness through course publication status, last activity, tool launches, and content interactions.

Working with Ben Motz (Indiana University) on data/data modeling for an additional grant that will implement ManyClasses (methods of providing appropriate, assignment-determined feedback – https://www.manyclasses.org/ ) in over 200 courses among Unizin members.

Employing the Engage reader, UDP data, Jane Russell and Ji Guo (University of Iowa) have developed the eTextbook Study to analyze students’ behaviors using reading tools and linking those behaviors to cognition and overall course performance.

Routine data modeling and CSV delivery of tailored datasets.

Data Concierge – refining the UDP service model: The new Unizin Data Concierge’s mission is to facilitate a path from learning analytics questions to targeted, intentional dataset delivery. When individuals approach Unizin with questions around UDP queries, reporting, or the suitability of UDP-driven research, our charge is to steward them toward the best possible utilization of UDP data. Our service model incorporates:

Exploration – rich dialogue around the capabilities of the UDP.
Data Environment and Modeling – creating GCP infrastructure, writing queries, and defining data marts on top of UDP.
Data Updates – refreshing data and connections; adding features to existing data marts.
Fine-tuning end products – fixing miscalculations or filters on delivered data.

Unizin Data Platform

Unizin Data Platform

Institution	Status	Changes only
University of Florida	Production	Implementation was completed in early January.
University of Michigan	Production	Starting a new implementation of additional SIS data in SIT.
University of Minnesota	Pre-production	SIS data validation is underway and nearing completion. A few issues remain.
University of Missouri System	Implementation	Initial SIS data integration is being tested; ready for automation once data run is confirmed with no issues. Then work may begin on Mizzou’s unified Canvas feed automation.
University of Nebraska – Omaha	Pre-production	Minor upgrades to the SIS feed; data validation still underway.

Unizin Product Releases

Major UDP release in mid-January. The UDP now imports all Canvas data that does not originate in the SIS such that it is part of a holistic data picture. Previously, only Canvas data could be traced back to an SIS term, course, section, enrollment, or person imported (to maintain standard data modeling). The UDP will now contain all learning data from education programs, courses, and sections not managed in the SIS. These typically include online ed, extension, and outreach, and other education programs. All Unizin members benefit from this important upgrade.

Major UDP documentation release provides extended support to all UDP users, including researchers, developers, engineers, BI, data analysts, and other stakeholders who need to use UDP data or push data into the UDP. The UDP end-user documentation provides a simple, easy-to-understand, and convenient reference for institutional staff who have a stake in UDP data. Additionally, we have included technical documentation for third-party vendors to reference when they implement UDP data integrations.

Minor Marketplace (aka, Order Tool) release provides a handful of bug fixes and minor enhancements to the end-user experience, particularly for program administrators. Impact: after significant new feature releases in the Fall (public catalog, end-to-end entitlements management) that make Order Tool a comprehensive solution for digital courseware and eText management, our efforts in January focused on bug fixes and minor enhancements.

Security and Privacy Committee (SPC)

SPC Goals for 2021

Finalize the Learning Analytics Guiding Principles – the result of a collaborative set of practical guidelines that consortium members may apply and build on in their institution’s ethical use of learning analytics. The Committee’s full review, followed by Unizin legal counsel’s consultation, and finalized by April 2021.
Review charge and charter – largely if not entirely met. Proposed items for a new charter submitted to the Committee by March 2021; finalized by April 2021.

The new charter should consider new data services, such as data marts, APIs, etc.

Understanding UDP Security – Presentation on the UDP architecture and the security processes and controls in place – presented by Etienne Pelaprat, February 8th, 2021
Securing Cross-Institutional Research – Review the Research Advisory Committee’s work and its process and documentation behind cross-institutional research (Facilitating Cross-institutional Research). This process will be presented to the SPC on February 15th.
IT Security Audit – Based on successes evident in the SPC’s collective efforts around NIST 800-53 low-security alignment, bring similar consortium-led expertise around an IT security audit of the Unizin GCP information environment. While full consensus appears less favorable, thoughts around third party risk assessment/controls audit (e.g., RiskConnect) will also be discussed.

Marketing

Unizin has inked a new agreement with Inside/Out to provide ongoing content marketing through July 2021. This new agreement brings us into Phase 3 of our marketing effort. Inside/Out and Unizin staff will generate original content (suitable for publishing on our website and other social presence channels) under this agreement. This content will be focused on articulating Unizin’s value proposition, the impact of its solutions, thought leadership – with the goal being to attract new members and refine our identity in higher education’s public discourse.

Vendor Partnership Timeline

This new section (next page) is designed to provide an overview of the current, upcoming, and potential vendor partnerships in Unizin’s pipeline today.