May 2021

Cross-Institutional Research Process, Service, and Governance Webinar

On July 9th, 10am CST, James Russell will present to the Unizin Board the Cross-institutional Research process. This presentation will include its function as a Unizin service, broader governance that defines the process, some added components as a result of invaluable feedback from members, and it’s formal “packaging” for both researchers and Unizin data stewards at each member institution.

Providing a clear and governed process supporting cross-institutional research is critical to leveraging the power of big data through the Unizin Data Platform (UDP).  In terms of institutional controls, while member institutions must have a completed data sharing agreement on file for their institutional data to be used by member researchers at other member institutions, the Cross-Institutional Research Process provides institutions both a separate final approval process.  This process provides a richer understanding of the research use case in play.  From the perspectives of the researchers and primary investigators initiating data requests, the process lifts the burden of navigating the various documents and components necessary for both the research and Unizin to grant access to data, as well as stewardship in the assessment of the data request from lending institutions.  This webinar will be available to the T&L community soon.

Unizin/Top Hat Sponsored Research Awards Announcement: Unizin and Top Hat are excited to support two proposals as part of the  first, co-sponsored research initiative.  The call for proposals looks to leverage the power of the UDP to determine how learning can be made more inclusive, engaging and effective.  The studies are to enlist faculty who use Top Hat in their courses and potentially other LMS/LTI tool data within the UDP.  Research will focus on success objectives to which Top Hat contributes, such as facilitating engagement, Peer Teaching Learning (PTL), feedback, or objectives where Top Hat plays a more direct focus on student success, such as the impact of custom-designed Top Hat textbooks.

Proposals awarded came from Tracie Redding (University of Nebraska Omaha) and Ben Hellar (Penn State University).  

Tracie’s proposal will use mixed methods approach that includes learning analytics from STEM courses at several different Unizin consortium institutions along with qualitative data from the University of Nebraska-Lincoln (UNL) to help compare student outcomes based on different online delivery platforms through a Communities of Inquiry (COI) perspective. Student outcomes are defined as student retention, relative grades, enrollment in future courses, and sense of course belonging. Learning analytics will be collected through the LMS-supported tools delivered via Top Hat and traditional Canvas-provided platforms.

Ben Hellar’s proposal will Leverage Top Hat data in the UDP to identify faculty instructors at Penn State who are currently heavily engaged users of the Top Hat tool in their courses.  Top Hat specifically, is increasingly being used by Penn State instructors to supplement the educational affordances of Canvas. Many courses take place primarily inside of the Top Hat environment. Currently, these learning activities are a “blind-spot” for a tool like but not limited to PSU’s Elevate, which focuses currently solely on Canvas course activity data. Consequently, we are not able to offer equivalent insights into student activity in courses with a heavy Top Hat presence. For this project Ben proposes to generalize Elevate beyond Canvas data, to incorporate learning activity that occurs in Top Hat.  

Many Classes Research Published: The UDP and consortium driven research of Ben Motz and Emily Fyfe, ManyClasses, has been published online in the journal Advances in Methods and Practices in Psychological Sciences.  Unique in both method and practice, ManyClasses provides a platform to both understand the efficacy of timeliness of feedback, and an experimental framework through which modes of teaching can be investigated. 

In Ben’s words, “ManyClasses is an educational experiment embedded in dozens of classrooms, spanning student populations, institutions, disciplines and course formats. By emphasizing diversity across independent samples, the results of a ManyClasses study will help researchers and teachers infer whether a research finding can reliably generalize.”

The study involved over 2,081 student participants from 38 classes, representing 17 disciplines.  The study spanned 15 campuses from five Unizin member institutions (University of Minnesota, Penn State, University of Michigan, University of Nebraska Lincoln, and Indiana University).

Unizin Sponsored Research – Implementation of two analytics projects utilizing the UDP and GCP

The LARA Project by: Stefano Fiorini, Indiana University

The original UDP-based  LARA project looked for patterns of gendered and racial performance differences in undergraduates, especially in the STEM disciplines, and associated remaining perennial issues in higher education. Regardless of the underlying causes, institutions cannot easily identify courses where patterns of disproportionate difference are apparent, nor their severity. Neither is higher education able to systematically analyze the extent of patterns on a multi-institutional scale, impoverishing public discourse about the nature and remedies of the problem.

In the 2019-20 academic year, Indiana University, the University of Michigan, and the Unizin consortium collaborated to reproduce a well-published study on gender performance differences on the Unizin Data Platform. The UDP aggregates institutional learning data with a common data model, enabling research and inquiry to easily be reproduced across institutions. Unizin also developed an out-of-the-box dataset for any UDP-using institution that would immediately enable them to analyze the patterns of performance difference across their entire institution. This resource illustrates the power of standards-based data integrations and modeling to lower the cost and effort required to reproduce valuable research across institutions.

The evolution of the LARA project established a concordance between the LMS, SIS and analytic data at Indiana University; the comparison of outputs of previous data from the LARA study; completed the assessment of the ease of replicability of the analysis across institutions; created the adaptation of machine learning course performance classifiers;  augmented ML with Canvas data; developed a model of comparison between machine learning (ML) deep learning (DL) tools; built reports in Tableau; and created documentation and videos to facilitate onboarding of new members of the BAR team or for sharing with collaborators on the Unizin platform.

What are highlights and key accomplishments from your project to date? Please be specific and include numbers, if possible?   Establishing access to UDP and connections to RStudio, JupyterLab server for secure deployment of analytics; Identify potential limitations of the data currently available on the UDP, the institutional roots of these limitations and possible workarounds, we also identified issues related to missing Canvas data; Re-designed original LARA analytics to run on the Unizin RStudio server so that they can be adopted across the Consortium – this code provides direct comparison to results published in Koester et al. 2016 and Matz et al. 2017; Expanded the LARA analytics to include other student demographic characteristics – this work is not peer-reviewed and careful consideration before making it available to other Unizin institutions is required; developed course performance models with selected classifier methods using the JupyterLab server environment, based on the work conducted at BAR over the past few years (see e.g. Fiorini et al. 2018 in the proceedings of LAK’18).

How might these accomplishments help advance future utilization of Unizin data products and platforms?   This was foundational work at the University of Nebraska.  While technical teams have placed the data on the platforms, this project enabled an institutional level review of the data by an end-user.  The support Unizin staff provided in troubleshooting and establishing the analytical environment will facilitate adoption of the platform at our institution  and will facilitate collaborations across institutions. For Indiana University Bloomington, this study identified critical data limitations related to the time frame and data made available by our institution in the UDP. Acting on these issues will facilitate IU use of the Unizin platform and participation in collaboration efforts. Access to Canvas data, properly connected to SIS data, expanded the availability and uses of data in order to better understand and improve the student experience.

How will your project potentially contribute to improved teaching and learning at your institution or beyond?  A well connected data and analytical environment provides the ideal setting for facilitating the development, implementation and deployment of analytics to campus partners. The way Unizin is set-up also facilitates the sharing and adoption of analytics developed by other institutions in the consortium. Specifically our project expanded our ability to tap into Canvas data to enrich advising on campus.

At University of Nebraska, like many,  there exists a  faculty learning analytics community.  This community was designed to use data to support institutional changes that will improve student success at the classroom, program and institution. This platform is now more fully accessible to units that provision data to the faculty researchers.  

Do you have any lessons learned or procedural insights to share around the use of Unizin data products and platforms?   Working in the UDP environment and with our Unizin colleagues made it clear that work on this platform is an interactive process that builds on constructive collaboration and adaptations. This aspect offers the advantage of being more adaptable to changes in data and tools available to analysts and researchers using Unizin.

1. Future Instructure Roadmap Webinar, June 30 11 to noon ET Confidential Content, Unizin members only.  Contact mariah.aguilar@unizin.org for an invite.  Designed for Board Members, open to Unizin Community.  Will not be recorded.
2. Presentation and Deep Dive Dialogue with Instructure on LTI 1.3 Webinar, July 16 noon to 1 ET, Instructure has asked that Unizin collect questions in advance. Please contribute.  Contact mariah.aguilar@unizin.org for connection information. Will be recorded and released to the Unizin community when available. 

Changes only: UMN has completed its additional round of SIS data in production. Unizin and Mizzou are testing multi-SIS dataset loads per day.

Minor UDP release – Historical event backfill. Unizin has backfilled all historical event data into each UDP tenant’s expanded event store tables, which make the query and analysis of event data convenient for the end-user. Consequently, we’ve announced a planned deprecation of the legacy “events” table for August 1, 2021.

No Marketplace or Engage releases in May.  The Engage team is heavily engaged in exploring the RedShelf eReader in order to plan for integration.

Unizin and TopHat have completed a joint thought leadership-content marketing piece on the importance of vendor adoption of learning data interoperability standards. Inside/Out, our marketing and publicity partner,  is shopping this article across a few news outlets. Unizin and Inside/Out have begun work on an additional similar piece around the future digital course materials adoption and the importance of an institutionally-governed digital course materials ecosystem. We also continue to produce various social media content in support of member activity, announcements, and news.

Planning and development is underway to achieve the deliverables as defined by the  two learning analytics task forces. Unizin is likely to prioritize the results of the advising-focused task force over the student and faculty-focused task force. The primary reason for this prioritization is the argument that data and analytics are likely more quickly integrated into advising and student coaching workflows than student/faculty workflows. 

Following the recommendations of the task forces, the data marts and services developed in support end-users will: (1) start with descriptive, aggregated student performance and outcomes data at the course level, (2) evolve to providing aggregated student data (e.g., across courses); and finally (3) evolve to indicators (e.g., activity metrics) that report trends and direction.

In May, the new process of initiating payment to Unizin’s vendor partners via the Automated Clearing House (ACH) was launched.  To establish the ACH process, Unizin is required to open a Treasure Management Account at the bank.  Although the banking fees for this type of account exceed the previous expense of a regular checking account, the cost of printer ink, check stock, envelopes, and postage all offset the additional banking fees.  One excellent advantage to processing payments via ACH is that vendors will receive payment the next day. This is all part of Unizin’s new remote offices transition!

The month of April saw significant movement on several inflight vendors.  Along with the progress with Turnitin, Unizin is closing in on completed terms with Sage Publishing to add their courseware platform and have made significant progress with Examity to add reseller terms to their existing agreement.  Unizin has begun reviewing two new vendors to potentially recommend moving forward with: Harmonize from 42 Lines and Boost App.