Unizin Reports
March 2021
This report is to be shared by design to communicate the full scope of Unizin impact, work products, and status at your institution.
Along the way, I learned that Unizin, itself, is a big, hairy, prescient idea that has begun to yield real impacts for its institutional members. Unizin has an amazing and emerging portfolio of data services, data marts, hosted applications, and ongoing projects that promise to advance the utilization of learning data to promote student success within higher education.
The most important lessons learned for me are:
- Unizin is not just infrastructure. It is a fantastic set of colleagues, staff, and communities with exceptional talent and work ethic that are more than willing to share their knowledge and opinions. Stop, ask, question, and reflect. I am indebted to so many of you. Thank you for your continued counsel, advice, knowledge, and support.
- Unizin already has unequaled, rare data service offerings that will require a focused partnership between Unizin and our institutions. Strong trust must be built to successfully obtain the insights needed by the diverse constituencies of institutional research, organizational effectiveness, advisors, faculty in various disciplines, students, academic research, academic technologists, and others. The secret to success is systemic, targeted communication across the institutions’ constituencies, not just IT. How do we get the most out of Unizin together?
- Like many start-ups, Unizin harnessed its early passion and investment to explore possibilities and focus on innovation. The outcome of those many journeys is not just the Unizin Data Platform (UDP) and the Engage/Order Tool (now known as Marketplace) systems. That’s only half the journey. Expanding adoption, increasing access and usability, and systemically communicating impact must be our ongoing mission.
- The ala carte offering of the UDP in the Google marketplace is just beginning. Attracting, being vetted, purchased, and ultimately onboarding those customers is a new frontier. The promise of third-party consultants within GCP’s offerings is needed to succeed with adoption.
Like many of you, I anticipated that this budget cycle within higher education would be a tight and uncertain one. Therefore, I expected that potential institutional interest in membership might be low until later in FY22. I am happy to report that interest and inquiries in full membership have exceeded my expectations. More importantly, Unizin’s packaging and marketing efforts are maturing rapidly. And, just a reminder to my Unizin Board Member Partners, as I mentioned in our 1:1’s early on, I’ll be circling back for some warm leads soon. We’re ready.
![Cathy O'Bryan](/wp-content/uploads/2020/10/cathy_obryan-150x150.png)
Cathy O'Bryan
Chief Executive Officer, Unizin
Research & Innovation
This section is the first of several more detailed updates on Unizin-sponsored research projects.
An Update on Unizin-Sponsored Research: U-Behavior Learning and Teaching Method: A UDP Cross-Institutional Project to Improve Student Learning and Success by Dr. James Folkestad, A specialist in technology-enhanced learning, and currently the Director of the Center for the Analytics of Learning and Teaching (C-ALT), James Folkestad teaches courses on digital learning and action research. He has been a member of the faculty at Colorado State University since 1997.
Objectives | Description of Status | Update |
Transfer U-Behavior App Hosting to Unizin | Running on Unizin Server | Done |
Modify U-Behavior Code to Connect to UDP | Identified Data Elements; near-real-time data needed; Using Canvas APIs for now; Awaiting Canvas Data 2.0 data latency improvement. | Done |
Pilot U-Behavior | Piloting with EdOD677 this semester, Spring 2021. Application running well with no needed modifications. | Done |
Conduct Cross-Institutional Research (CSU/IU) | Working with Indiana University to establish a shared Data Handling Agreement. That process has been slow and has slowed due to the demands on administrators due to the Covid-19 crisis. | Ongoing in Year 2 (FY21-22) of Grant |
Contributing to Improved Teaching and Learning at CSU and beyond: U-Behavior’s first experiment was conducted during the fall 2020 semester with a follow-up retention test administered at the start of the spring semester. This first experiment established a U-Behavior condition and a control (no-U-Behavior) condition. This research aims to measure if the U-Behavior method would translate into student study and practice behavior change and if those practice behaviors translate into performance gains.
The behaviors of interest are based on the cognitive science literature of spaced practice
behavior and interleaving (mixing) up practice between subjects. Behaviors of interest include:
- Behavior Change: Results indicate significant differences between the groups with
substantial effects of 1.88 (Cohen’s d). U-Behavior had a significant impact on
how students practiced course content during the fall semester.
- Retention Test Performance: The retention test data indicate significant
differences between these conditions with an effect size of 0.65 (Cohen’s d). This
medium to large effect size indicates an improvement of around one letter grade.
- This performance improvement could significantly contribute to both learning and
teaching at CSU and other Unizin institutions. For example, based on these results, this could help reduce the DFW rates. More students could potentially graduate from difficult STEM majors like microbiology. Microbiology was the first discipline area impacted by the first experiment.
Furthermore, Dr. Folkestad anticipates that other institutions will want to adopt this method once other Unizin institutions see research-proven benefits of changing students’ practice behaviors. It may be interesting to connect with microbiology programs at other institutions to adopt and use the retrieval practice activities (RPAs). This multi-institutional collaboration would lead to larger datasets surrounding the use of these RPA questions and lead to the improvement of the RPA questions across institutions.
Unlocking Additional Resources: CSU is actively seeking funds to continue the U-Behavior research and U-Behavior operations. CSU’s information technology group supports the ongoing help-desk function of U-Behavior. This in-kind contribution of service illustrates CSU’s commitment to further developing the U-Behavior learning and teaching method. The College of Health and Human Services and the School of Education at CSU have committed to continuing the post-doctoral position. That will support continued maintenance and development of the U-Behavior code.
Dr. Folkestad will resubmit the EHR NSF proposal during the fall semester 2021. The original submission received positive feedback on the grant, including the following comment from one of the reviewers. “The study’s overarching aim is the dream of every college instructor – to see students move toward more productive study strategies that produce learning while abandoning pseudo-study methods such as ‘re-reading notes,’ highlighting text, and cramming for exams.” (Reviewer #4). Dr. Fokestad plans to meet with the NSF program director to acquire additional feedback, modify the proposal and resubmit in 2022.
Research Support
- Unizin is working with Colorado State and Indiana University in the ongoing effort to refine data marts and dashboards similar to those delivered to Oregon State, including Course readiness status; Last Activity (student); Tool launches; Course content views, and Canvas tool launches.
Revising Unizin’s UDP Depersonalization (de-identification) documentation to incorporate sections covering “Infrastructure and Access,” setting conditions on where depersonalized data should reside in the course of use, and “Access and End Dates” governing determination and length of access to depersonalized data. Depersonalized datasets have been conveyed to the University of Nebraska, Lincoln (Chad Brassil, Casey Nugent, and Justin Olmanson), and the University of Michigan (Zhen Qian and Chris Brooks). These datasets will introduce UDP data to new audiences, including graduate students conducting research at the University of Nebraska and hackathon participants at the University of Michigan. Depersonalization is a crucial step to simplify multi-institutional UDP-based research.
Unizin Data Platform
Unizin Data Platform (changes only):
Institution |
Status |
Change |
University of Michigan-Dearborn |
Production |
Expanded Production Data: The implementation of additional SIS data in SIT (pre-prod) is complete. UMD is working on moving the changes into production. |
University of Minnesota |
Production |
Expanded Production Data: UMN is testing phase 2 of their SIS data integration, which expands the SIS data they send to the UDP in our pre-prod env. Once that is complete, they will move phase 2 to prod. |
University of Missouri System |
Production |
Mizzou’s production UDP instance is running! The consolidated SIS feed is importing into the UDP, and their UDP-based Canvas import feed is working. This integration enables the production UDP instance to consume Canvas Data from the consolidated canvas instance. |
Unizin Product Releases
Major UDP release – configurable loading schemas. The UDP can now be configured to import any combination of context data loading schemas. This new feature enables institutions to use the UDP with only LMS data while still developing their SIS data integration. Oregon State, for example, has benefited from this feature. It has enabled OSU to build and leverage LMS-only data marts to track course readiness, identify lagging students, measure Canvas tool usage, and measure LTI usage across their institution.
Order Tool minor releases: Unizin released a series of bug fixes to data reporting to institutions and our institutional data import process. As a result, there is higher reporting and import fidelity for the data flowing through Unizin Order Tool.
Marketing
Unizin’s marketing initiatives are proceeding with urgency:
Social media presence. This effort focuses on Unizin’s LinkedIn and Twitter channels. Unizin is now maintaining a social media presence with posts 4-5 times per week that promote original content by Unizin, includes cross-promotion with our partners, and disseminates the Unizin Summit presentation tracks and schedule. Follow Unizin on Twitter and Linkedin. Let’s build our social community!
Original content. Unizin’s D2L press release was posted to Unizin’s revived News section. Future original content postings will include a collaboration piece with TopHat on their UDP data integration and James Russell’s reflections on his first 60 days at Unizin from last month’s Board Update Report.
Products and services. The new Unizin product and services drop-down options are available on the Unizin public site. Unizin will continue to invest time and resources into improving these pages. These pages communicate the value proposition of Unizin’s product and services portfolio. New landing pages exist for the UDP, Order Tool and Engage, and our Hosted application services (I.e., MyLA).
Logo. Throughout March, work continued on the refinement of the Unizin logo and associated branding. This effort will enable Unizin to apply the new branding to Unizin’s website, marketing, collateral, and the presentation of our products/services. Creating, maintaining, and utilizing cohesive brand components will further Unizin’s presence within the higher education market.
Other External Publicity. Educause interviewed Unizin’s CEO, The Future of Learning Begins and Ends with Learning Data: An Interview with Cathy O’Bryan, Educause Review; March 17, 2021.
Summit
Unizin will be hosting its first-ever virtual Summit on April 21 and 22nd from noon until 4 pm CT. Registration is still open. Currently, there are over 500 registrants, including 50+ non-member registrants from 28 institutions. Unizin leadership is reaching out to each of these visitors individually.
All registrants are invited to the Teaching and Learning community social event from 4 to 5 pm on April 21, using spatial chat. Join us, register!
Google Partnership
Unizin and Google’s joint sales engagements for the UDP have expanded since the announcement in March that Unizin is offering the UDP via the GCP marketplace. Unizin is currently working with Google on four separate institutional accounts, all of which are R1 institutions. One of which, ASU, is in the early stages, including a security review of procurement. See later article in this report on IT Security.
Sean DeMonner (University of Michigan) and Cathy O’Bryan will be presenting live during Google’s Student Success with Google Cloud conference on April 20. Join us!
Campus Technology published an article, Unizin Data Platform Now Available to Any Institution Via Google Cloud on UDP and GCP partnership.
Internal Policy Changes
Unizin leadership has finalized 11 HR policies, with significant updates to what was formally titled “Telecommuting” (now Remote Work); revision of the Employee Leave Policies (covering Unizin’s policy on unlimited paid time off), Paid Sick Leave, Paid Parental Leave, and Military Leave; and the creation of the Salary Continuation Policy.
- The Remote Work Policy reflects Unizin’s current remote workforce and sets clear guidelines for areas such as broad eligibility and supply reimbursement. Unizin will remain fully remote post-pandemic.
- The Employee Leave Policy folds all categories of leave into one policy. It articulates clearly scheduling of leave, accrual, and carryover of sick leave, parent leave, and military leave absences.
- The Salary Continuation Policy is new to Unizin and better represents employees’ options under our short and long-term disability plans. Specifically, benefits under our plans commence after an employee has been absent from work for a consecutive 14-day period due to sickness or injury (“Elimination Period”). The maximum benefit period for short-term disability is 11 weeks, and the weekly benefit is 60% of an employee’s weekly earnings up to a maximum of $1,500 per week. After these 11 weeks of continuous disability, the employee may become eligible for long-term disability benefits provided by Unizin.
Other finalized policies but little changed include Travel and Tuition reimbursement; Disciplinary; Discrimination, Harassment, Retaliation, and Background Checks.
IT Security
Over the last three months, Unizin has been steadily verifying and completing policy and process documentation related to security practices. This process began with looking at the NIST 800-53 alignment certification conducted in 2019 by Sera Brynn, a third-party IT security assessment firm. This alignment grew out of a security review and risk assessment of the UDP undertaken by the University of Florida in 2018. Additionally, as part of the risk assessment, the University of Florida conducted a penetration test. Unizin created several policies intended to reflect our compliance with the 124 controls from NIST 800-53 low from the risk assessment. Unizin hired Sera-Brynn to perform independent verification and validation of the University of Florida assessment.
The Sera Brynn verification was completed in April 2020 and found just two controls unimplemented – o involving notice to users accessing the UDP and indicating an insufficient Systems Security Plan. The former is covered within our user agreements, and the latter was completed across the last three months.
Our first step in the maturation of documenting our security posture has been to gather and review all relevant documentation and policy and then to organize it so that it is transparent to current and future clients and members. To this end, Unizin has created a package of documentation that includes the following:
- A general statement on Unizin security and privacy, which includes the Sera-Brynn summary of the NIST 800-53 verification
- The Sera-Brynn report
- A spreadsheet that maps the 124 NIST security controls to Unizin policy documents
- Unizin’s System Security Plan
- A security document summarizing Unizin’s development practices, code vulnerability scanning, and the creation of low-risk images
- The penetration test report conducted in 2018 by the University of Florida
- The Higher Education Community Vendor Assessment Toolkit (HECVAT) for the UDP
While the above represents a solid foundation for the UDP security posture, there is considerable work ahead. Over the past months, Unizin has been discussing with Arizona State University (ASU) the implementation and management of their own UDP instance, purchased through the Google Marketplace. To move this project forward, ASU is conducting a security review of the Google UDP instance. Unizin has been working closely with ASU’s procurement. As a result of this effort, Unizin is reviewing much of the information on the GCP environment. This review focuses on:
- Implementation of routine (weekly) scoped vulnerability scans and daily emerging threat scans. Unizin has begun comprehensively scanning its cloud infrastructure – including its Google Marketplace UDP instance – for over 10,000 vulnerabilities. Reports generated are shared with the Security and Privacy Committee. Additionally, dependent on threat activity, our GCP environments are scanned daily for emerging threats as well.
- The need for annual penetration testing of our UDP package available through Google Marketplace. To this end, we are discussing options with two third parties: NetSpi and REN-ISAC. Preliminary discussions with NetSpi, while providing a state-of-the-art approach to PTaaS, have revealed their services to be expensive. Unizin is scheduled for an initial consultation with REN-ISAC on April 24, 2021.
- Update and revise current IT Policies and approach to policies. Remove the mix of policy elements that address Unizin IT for business processes (primarily around traditional security governing local IT assets and endpoints and the security oversight thereof) and security managing cloud services relative to the UDP/GCP. Our NIST alignment is based on an assessment of the UDP. Existing IT policies should be specific to the UDP/GCP environments. Separate IT policies will be created to cover systems, personnel, data, and workflows unrelated to the UDP – specifically those IT policies that should govern the business process of Unizin, primarily around how we manage fiscal and HR data, and the systems and applications that interact with such data.
- Verify process documentation and documented workflows behind policy and formally build compliance checks around these documented processes. This task will be ongoing, with much in place already. Policy must be partnered with compliance, with sufficient documentation and routine testing where required.
Putting into standard practice annual penetration testing of our UDP marketplace offering (and potentially expanding pen tests to broader targets), conducting routine vulnerability scans of our entire GCP environment, and bringing IT policies into compliance practice alignment, will play a significant role in ensuring the protection of UDP’s systems and data.
Student Savings
The University of Nebraska reports that since 2019, the Successful Teaching with Affordable Resources Initiative has saved students approximately $3.5M.
Vendor Partnership Timeline
Unizin has re-opened an earlier conversation with Panapto that focused on the work that Panapto is doing around usage data. With the increased Panapto consortium usage, Unizin believes that they may be excellent partner candidates. Unizin will be presenting a business case to the Learning Tools Strategy and Operations community soon. See Highlighted Section below.