November/December 2021

On November 4, Brad Zurcher presented at the Teaching & Learning Advisory Group with an update on the digital content program and announcing the formal partnership with RedShelf.   

In the immediate wake of the completion of the contract with RedShelf, the Unizin Services team scheduled time with each institution’s program administrators to walk through the financial details of the agreement and precisely how that will affect institutions.  Also discussed was the timeline for replacing the Unizin reader with the RedShelf reader and early questions about change management and communication plans.

Unizin has also connected the Program Administrator’s Community of Practice (CoP) with the RedShelf team with expertise in All Students Acquire (aka Inclusive Access) and their product manager for accessibility.  See below. Both will be actively engaged with the CoP throughout the transition and provide resources and advice for best transition and train instructors and students.

A new community of practice has been formed within the Unizin consortium.  Noma Maier at Indiana University and Vicky Maloy at the University of Iowa have spearheaded the formation of a CoP for program administrators at member institutions that run digital content programs through Unizin.  The PA CoP is a space for those who oversee these programs to share successes and challenges, focusing on program growth.   While each program is unique, many experiences are common, and this group is eager to grow together.  This group will be instrumental in assisting with the rollout plans and implementation of the new reader (RedShelf).

Some of the most exciting engineering work has been focused on the Engage eReader replacement that will lay the groundwork for a long-anticipated Unizin Content Platform.  This project is an eventual extension of the Unizin Common Data Model and a natural complement to the Unizin Data Platform.

This work started with the re-examination of LTI 1.3 to prepare the (new) Engage Bookshelf to take full advantage as a tool launched from Canvas and as a platform launching the RedShelf eReader.  Work has proceeded with the construction of  a new Publisher metadata pipeline, which will both improve the operation of existing products like Order Tool, as well as support the provisioning, entitlement, and serving of content through the new eReader, RedShelf.

Unizin Data Services and Solutions (DSS) team is gearing up to begin work on the Learning Analytic Taskforces (I.e.,Taskforce for Faculty and Student Success; Taskforce for Advisors and Student Success) in January.  With the base dashboards delivered to member institutions complete by the end of the calendar year, DSS will begin with internal discussions on modeling data based on three common themes.

The first theme centers on student activity measure (which will likely include multiple data marts to realize), including an expansive set of elements that reflect the impact of a variety, of course, activities on student outcomes, as well as student activity within the LMS and accompanying tools that will lead to and build a second set of views around measures of engagement/disengagement.  The framing of the data will then be further focused based on the audience that it serves (faculty/advisor/student), so that, for example, advising or student success groups will have views to student performance across a student’s course history in ways that faculty may not; or demographic information may be incorporated for specific views that provide the basis for surfacing deeper understanding of STEM education or DEI.

The third theme or area of focus will provide broad, aggregated measures on the composition of a course.  This focus will involve thinking about views into the data based on audience (I.e., faculty, advisors, and students). It would look to provide, for example, a faculty view of a particular student profile within a course, such as what courses a student has taken before the class; or, aggregated at the course level, how do students compare to their peers who have strong STEM skills (derived from previous courses taken).

The DSS team will meet monthly between January and June with a small consortium-based working group to review models and align with Taskforce deliverables.  This group will also provide feedback on communicating the structure, scope, and potential applications of the data marts DSS creates to address the foundational use cases each task force defined.

In preparation for the Unizin 2022 Summit in April, we need proposal reviewers for each track.  Ideally, we would like to have 3 or 4 reviewers for each track.  Your perspective and experience would be greatly appreciated.  The tracks are listed below.  Reviewers will have three weeks in Feb. to complete their review of the proposals.  Unizin will provide form-based rubric for scoring.  Proposals are not lengthy.

If you wish to volunteer to be a proposal reviewer, please email cathy.obryan@unizin.org and indicate at least two tracks that you would be willing to review by no later than January 3, 2022.

Unizin’s Summit 2022 will be remote and take place on April 26th and 28th from 11a to 4p CT.    The Call for Proposals is now open.  Proposals are due January 31, 2022

Tracks include:

1. Uniquely, Unizin will focus on those projects, initiatives, and findings empowered by Unizin services, data, and products.  Submissions can also include solutions enabled by the power of the Unizin consortium and its member institutions.  Proposals on the innovative uses of digital learning tools, analytics, and other products resulting from T&L sub-committee work are also encouraged.  Please include identifying, developing, and using collaboratively developed resources and unique use cases with students, faculty, instructors, advisors, administrators, and other audiences. Please note the emphasis on active audience engagement in the formats below. 

2. Digital Learning Data Ethics, Policy, Governance will focus on the essential components of data policy, governance, and the increasing need for professional development and awareness of data ethics best practices. As learning data becomes increasingly valuable and available, institutional policy and data governance must consider and address the ramifications of applied learner analytics. Submissions can cover social norming, communication campaigns, transparency awareness, and privacy policies, among other related topics.   Proposals for this track can focus on any or all of these components by sharing campus practices, initiatives, processes, and products.  Discussions of future needs and how best to address them as a community is also most welcome.  Please note the emphasis on active audience engagement in the formats below. 

3. Informed Research and Decisions through Applied LA will focus on utilizing learner analytics to operational and strategic challenges on your campuses.  The utilization of learner analytics varies in scope, scale, and context. This track aims to demonstrate, illustrate, and inform the community of the vast diversity of these possibilities. Proposals are encouraged to include background on the problem, the challenges, and the journey of selecting the learner analytics combined with other data as needed to address the needs and the current resulting state.  Proposals are encouraged to include next steps and envisioned future states or stages.  Please note the emphasis on active audience engagement in the formats below. 

We highly recommend that your proposals actively engage the audience.  Like last year we anticipate that the audience will be both Unizin members and non-members.  The reviewers will prioritize those formats (see below) that are most engaging. (i.e., speed-dating, panels, and guided discussions).  There are four formats:

• Speed Dating (10 to 15 minutes per topic) Each presenter shall have 10 to 15 minutes of the whole session to provide a quick orientation to their topic or sub-topics.  Proposals should include the overarching theme that types the collection of topics or ‘speed-dates’ together.  
• Panel-Style Presentations are a group of persons gathered to discuss a given topic publicly.  Typically they represent a variety of perspectives on this said topic and together can illustrate a complete understanding of the given topic.  All panel proposals are encouraged to have a facilitator for their panelists, prepare questions, and allow for spontaneous questions from their audiences. 
• Guided Discussions are carefully designed to promote brainstorming through active collaboration and discovery with the audience.  Small group breakout sessions are typical to encourage individual participation and harvest information shared with the larger group.  Collaborative digital tools are frequently essential in this type of session.  Results are commonly shared both with the audience and externally.
• Presentations are the more traditional style of proposals. They typically include a visual (PowerPoint) and a verbal component (lecture). However, submissions of this type are strongly encouraged to encourage more active audience participation through surveys, group discussions (potentially using small group breakouts), and Q&A.

The University of Minnesota Libraries will partner with Unizin, IMS Global, Ex Libris, and UMN Office of Information Technology to implement the Caliper Library Profile in the course reading list system Leganto (Ex Libris) and provide caliperized event data for the Twin Cities for deposit into the Unizin Data Platform (UDP).  

Leganto usage data is well suited for inclusion into the UDP since it is already tied to Canvas, specific courses, assigned course materials accessed in those courses, and student usage of those materials.  There are several benefits to this pilot:

• Students will benefit from including Leganto data in student-facing dashboards such as My Learning Analytics (MyLA) and future learning analytics projects where the inclusion of Leganto-based course materials usage data is appropriate. Better data from Leganto will also help faculty improve course content and design, potentially leading to more student success.
• Caliper is a preferred standard for depositing learner data into the Unizin Data Platform. This effort will create valuable data for current and future learning analytics projects, and it prepares the University of Minnesota Libraries to have course-related data to offer where appropriate.
• Implementing the Caliper Library Profile will help establish the profile and promote it out of “candidate status.” Information about the implementation and depositing data in the UDP will also be shared with other Unizin libraries.
• The University of Minnesota Libraries also plans to implement an “opt-out” option for students, allowing pilot partners to test and evaluate student consent functionality and workflows. In addition, the University of Minnesota Libraries will investigate data security and control options within the UDP.

This pilot will create and strengthen relationships between several internal and external stakeholders and organizations.

Indiana University has registered an entry, Terracotta, in the XPRIZE Digital Learning Challenge. Terracotta is an open-source LTI 1.3 web app, designed, managed, and hosted at Indiana University, that integrates with the Canvas learning management system (LMS). Terracotta’s core feature is to enable experimental research on assignments in Canvas. Terracotta was initially funded by a gift from Schmidt Futures and is now supported by the Institute of Education Sciences grant. We believe Terracotta is highly competitive for the XPRIZE Digital Learning Challenge. The purpose of this prospectus is to seek involvement from Unizin member institutions in this competition.

The XPRIZE Digital Learning Challenge aims to “enable experiments of frequency, scope, and scale not possible through traditional methods used in education research or commercial EdTech processes.” The winning team must have built a system that can “conduct rapid, reproducible experiments and demonstrate the resilience and rigor of this infrastructure in a formal learning context.”

We propose a collaborative distributed study to be conducted across Unizin institutions during Fall 2022. The study will be analogous to the ManyClasses effort but smaller in scale and time commitment. We seek roughly 20 classes to participate in this effort.

At this time, we are preparing a local pilot at IU: an experimental research study executed in Terracotta during the Spring 2022 semester. After this pilot phase, if successful, we may be one of 5 teams selected to proceed to the demonstration phase. If we move to the demonstration phase, we would be interested in running the same pilot experiment in as many as 20 classes across Unizin member institutions. Participation will require the member institution to integrate Terracotta with its Canvas instance. Indiana University will provide free hosting.

We propose to share 30% of any prize funds with Unizin, which will then disburse 2/3rds of these funds to the 20 teachers at participating Unizin institutions. If we reach the demonstration phase, teachers will receive $500 simply for participating. If we are chosen as the runner-up, each of the 20 teachers will receive $2,500. If we are selected as the grand prize winner, each of the 20 teachers will receive $5,000.

Research Background:  Pre-questions are ungraded questions that appear before a learning activity. Though learners rarely know the answers to these questions, presenting pre-questions before a lecture, a reading, a discussion, or an online video reliably improves learning from the subsequent activity compared to activities not preceded by pre-questions. One study recently found evidence that the benefit of pre-questions exceeds the benefit of questions after the learning activity, suggesting that the pre-question effect may be greater than the testing effect (Pan & Sana, 2021).

Why do pre-questions improve learning? Laboratory research suggests that pre-questions cause learners to increase their focus on the learning task, enhancing initial memory encoding of the material (Pan et al., 2020). However, given that this research has been conducted in a laboratory, one wonders whether one would observe similar effects on attention in authentic education contexts where learners have a grade incentive to learn the material. The current study will critically examine the hypothesis that pre-questions change how students process learning materials in authentic self-directed learning settings. 

In each participating class, a teacher will identify or create two instructional videos on YouTube and generate two sets of multiple-choice questions that cover material, facts, or content presented in each video. These videos will be presented in Canvas assignments, deployed using Terracotta, where the question sets, either do or don’t, appear before the video. Each student will experience an assignment with pre-questions and an assignment without pre-questions, but Terracotta will randomize the order of this exposure in a within-subject crossover design. Teachers will also be asked to include the same pre-questions in a subsequent course assessment of their choosing (e.g., an exam, a review quiz, a midterm, etc.). 

Our primary measure will be the video playback analytics captured from the embedded iframe player in Terracotta. We will examine the effect of pre-questions on students’ behaviors (playback speed, jump ahead, player size) when watching instructional videos, and secondarily, their delayed memory performance on questions that had been included in the videos.

About Terracotta

Member institutions, such as Penn State, whose Top Hat funded project will hinge on adding Top Hat data to enrich their Elevate reporting. The University of Iowa will be leaning on Top Hat context data the feed to the UDP.  To this end, Unizin Engineering will be finalizing the implementation of the Top Hat loading schema for context data.  There remains some fine-tuning around mapping between the way Top Hat and the UDP represent Course Offerings and Course Sections, respectively, so that Course Section Enrollments can be represented, but should not extend the integration implementation past March 2022.

For Tracie Reding’s research, there are some challenges are ahead.  First, Tracie is leaving the University of Nebraska for a new position and has assigned a new principal investigator,  Sache Nedungadi, for the Digital Interactions to Enhance Student Belongingness and Engagement, Unizin/Top Hat funded research project.  Data Services and Solutions will work with Sache to finalize data requirements and the Cross-institutional research process.  Additionally, because this project hinges on Top Hat discussions and neither event nor a publicly accessible API provide discussion data (including discussion thread, author, and responders), we are working with Top Hat to provide tailored data extracts for targeted STEM courses in January.

The Unizin Services and Data Services and Solutions teams are beginning the final laps of our independent UDP pilots for ASU EdPlus and Continuing and Professional Education (CPE):

• ASU (ASU Online) serves admitted students in residential and online education.
• Learning enterprise (LE) serves non-credit-bearing courses for students. LE serves students who seek earned admission to ASU via Open edX. LE also serves a Continuing and Professional Education (CPE) via a single Canvas instance.

The CPE UDP is an LMS-only instance that is in full implementation.  Unizin’s Kyle Unruh and Sara  Bolf completed (12/10/2021) the second  (prepaid) training for ASU, which included investigation of quiz results and enrollments to construct views helpful in analyzing student performance in courses; dwell time in courses; data exploration, visualization, and modeling using Python; and  a specific use case brought by ASU that asked: “how do we create a list of learners that have enrolled in classes, that are still enrolled in classes for the current and future term.”

Following the training, Jonathan Carroll, Director of Learner Insights and Opportunities, had high praise for the Unizin team: “I wanted to express my gratitude to you and your team, particularly Sarah, James, and Kyle at this time for hosting a UDP training for our continuing and professional education unit today. We had 14 people from ASU on the call from three different teams across two business units within ASU. I am excited to see the growth and adoption that the UDP can have at ASU as we value the enormous benefit Unizin brings with preparing our LMS+SIS data so we can focus on using our data and not maintaining it.”

Additionally, Lynette Mercado and Tracy Thompson from Unizin Services have invested tremendous efforts in stewarding ASU toward their SIS mapping and implementing the ASU Online, UDP instance.  Their work continues as they help bring and validate the required SIS entities to completion and provide consultation for ASU’s efforts to implement the remaining recommended entities.

One more training and a final “open discussion” session around the ASU OL UDP instance are planned for early January.  Licensing for both instances of the ASU UDP will begin in June 2022 when this first pilot is complete.

• Unizin Data Platform ingest of SIS data can now happen multiple times a day, and the accompanying SIS data manifest files are easier to provide – Release Notes.
• Order Tool now emails reminders about the starting and ending of ordering periods to Instructors and Coordinators – Release Notes.
• Order Tool has made improvements to the user experience and resolved minor bugs reported by member institutions – Release Notes.

Unizin will be closed for the holidays beginning Monday, December 20th, and will open on Monday, January 3rd. For our consortium members, if you require urgent assistance, please visit our Jira support portal to request support. Unizin staff will be monitoring this channel throughout the holiday break. For non-urgent requests, please contact services@unizin.org and we will be in touch after the break. Happy holidays to everyone!