Summer 2021

Unizin hosted a UDP boot camp for the University of Minnesota on August 20 and August 23.  These boot camps are designed to:

1. provide a broad overview of the concepts animating the UDP and the types of event and context data that comprise the rich learner and learning environment ecosystem available in the UDP.
2. provide a foundational understanding of how to analyze this data within the Google Cloud, BigQuery (BQ) environment.
3. guide participants through building SQL queries and working with R to analyze and target areas of inquiry utilizing this data.
4. develop a better understanding of the unique data concerns of member institutions like Minnesota.

Kyle Unruh and Sara Bolf led the UDP boot camp sessions. This session was attended by 14 participants from Minnesota, including service owners/managers, systems and business analysts, and student success professionals. The boot camp began with an overview of the UDP, describing its various purposes and features. Focus then moved toward how to connect and manage projects within the UDP, BQ environment.  Discussion on best practices for working within BQ, connecting through VPN, and using assigned Service accounts were covered extensively.  

Three foundational use cases were explored. Two use cases focused on using context and event data in BigQuery to look at quiz results and dwell time in courses, respectively. The third use case focused on how to use context data in R to predict final grades.  

Use case 1 – Quiz Results with demographics – modeled data around evaluating student performance on assignments utilizing the UDP’s Context Store, drawing on quiz results, person demographics, and enrollments to build views for analyzing student performance in courses.

Use case 2 – Dwell-time in Course – looked to understand student activity and, by extension, engagement of students based on time spent in the LMS.

Use case 3 – Data Exploration, Visualization, and Modeling in R- introduced participants to a learning analytics project focused on final grades and other learning attributes’ effect on them, utilizing the Context Store.  The analysis incorporated attributes related to students, course offering, academic terms, quizzes, and final grades, centering on creating a linear model built on relationships between the multiple variables employed and a student’s final grade in a course.

Participants can continue to work in the projects developed through boot camp and return to a recording made of the boot camp in its entirety to review topics and techniques covered.

The Unizin Data Solutions team partnered with Jose Silva-Lugo, Center for Instructional Technology and Training at the University of Florida, to model student success using R Studio and the UDP. The Unizin Data Services and Solutions (DSS) team worked closely with Jose on delivering a clean, comprehensive data mart that included the following features:

• Person demographics
• Quiz results (submissions, time taken, etc.)
• Event timestamps for time-on-task calculations
• Cumulative GPA (including and excluding an individual course grade)
• Current GPA (including and excluding a particular course grade)

The DSS meetings allowed Jose to present challenges around interpreting SQL results and collaboration around building marts and SQL best practices. DSS also introduced a test instance of the “interaction session” data marts, which calculate time-on-task for 10, 20, or 30-minute functional cut-off windows for student events in University of Florida courses.  

Jose continues his analysis, and the output will be a predictive model in R characterizing student success using in-term activity features.

Educause published an article that focuses on My Learning Analytics,  “Show Students Their Data: Using Dashboards to Support Self-Regulated Learning,” by Jennifer Love, Sean DeMonner, and Stephanie Teasley at the University of Michigan.

Inside Higher Education published “Higher Education Has a Data Problem” as their lead story by Bhavin Shah, CTO Top Hat, and Cathy O’Bryan, CEO Unizin.  Please alert your teams and encourage them to interact with the content on the IHE site: tweet, post, share from the actual article. Strong engagement for our byline will help next time we submit for consideration. 

New support for multi-SIS dataset loads per day nears release (after support tested in June proved inadequate), as well as an “Entity Status Table” surfacing the most recent successful ingest time for each Context Store Entity.

New Context Store Entities for the UDP- New UCDM entities are available in the Context Store, providing additional data from Canvas immediately and more support for integrations from the deprecated UDW.

The following features have been released since June 2021. For more information, please see the Unizin resources site. 

• Delivered a reordering feature that streamlines the ordering process by enabling Instructors, Course Coordinators, and Program Administrators to reorder for any course where an order was placed for another term.
• Delivered a student price feature that describes how much a student will pay (or another party on their behalf) to purchase digital course materials. This allows institutions to apply a percent markup to the offer price available through vendor agreements. Institutions may choose to implement a markup fee to fund their digital content program or digital teaching and learning technologies ecosystem. 
• Delivered a historical entitlements import feature that backfills entitlements for purchases that students made outside of Order Tool. For example, this import might be used to backfill entitlements for a new learning tool to an institution’s catalog. This ensures that students will not be charged again for a learning tool that they already have access to through a prior purchase. The University of Iowa utilized this feature to manage the adoption and student billing of Top Hat Classroom, where the cost of an enterprise agreement was passed on to students. This reduced the cost of Top Hat Classroom to students for the Fall 2021 term. 

The summer months have brought with them significant conversations with the vendor of the chosen eReader replacement.  Unizin is nearing the completion of the contract negotiation. As a part of that process, the Engineering and Product teams from both vendors are meeting to put the finishing touches on the scope of work that will guide us through the technical replacement work.

This contract is expected to be completed in September, with implementation work starting in earnest immediately.

During these technical dives and conversations with the vendor, along with the departure of the former CTO, we have slightly adjusted the delivery timeline. The current plan is to pilot soft launch the new eReader experience in the summer 2022 term with a formal launch for Fall 2022.

Meeting the charge of both the Taskforce for Faculty and Students, and Advisors & Student Success Staff is multifaceted.  Essential in Data Services and Solutions delivery of Taskforce imperatives is the completion for all member institutions of the data marts and resulting dashboards that Unizin began to deploy to member institutions in the Spring of 2021.  Equitably distributing the knowledge gained through project work that resulted from a singular engagement with a member institution across all member institutions represents one of many vital assets of the consortium – leveraging learning analytics knowledge and insights gained from singular engagements across all members. These dashboards provide tangible value for institutional reporting and intelligence and clear examples of what can be created from the data marts that DSS will develop as part of the Taskforce deliverables.

Over the last eight months, DSS has been steadily delivering five base data marts and accompanying dashboards that address, through our engagements with student success teams, what we believe to be a core set of questions to member institutions, beginning with Indiana University, Colorado State University, Oregon State University, Penn State University, and Minnesota University.  The University of Florida and the University of Wisconsin are currently in the process of development. The University of Missouri, Rutgers University, and The Ohio State University to follow once the work required within their institution’s UDP is complete.  

These dashboards and data marts, built-in Data Studio within the Google BigQuery project space within each member’s UDP tenant, reflect each member institution’s learner activities and learning environments and include the following views.

1. Course Readiness (which courses in the LMS are published and unpublished)
2.  Last Activity (when a student has last engaged with the LMS)
3.  Tool Launches (what tools outside of the LMS were launched)
4.  Content Usage (what content is being most frequently viewed by course and which students haven’t looked at particular content)
5.  Canvas Tool Usage (what Canvas-based tools are being launched, by course, and by student)  

These dashboards – purposed data marts visualized through Data Studio – were created to give member institutions who needed a “jump start” into what is possible from an institutional reporting or intelligence standpoint.  These dashboards both provide insight into the learning environment and give solid examples for member institution’s analysts what can be done both in terms of tangible examples of refreshed queries that constitute these marts and dashboards that are populated with by those underlying marts.  The goal in providing these “proof of concept” marts and visualizations is to provide member institutions with foundational examples for their independent development of learning analytics and real and tangible resource for institutional research and decision-making.  As part of DSS outreach, consultations, institutional and academic research support, and boot camp and hackathon training, these marts and dashboards further fill the gap between the vastness of the UDP and supporting growth and implementation of learning analytics in all member institutions.

DSS broadly sees two primary “buckets” of data, through which data marts will be modeled and created.  The first might be considered a student activity bucket (which will likely include multiple data marts to realize), including an expansive set of elements that reflect the impact of a variety, of course, activities on student outcomes, as well as student activity within the LMS and accompanying tools as a measure of engagement/disengagement.  The data framing will then be further focused based on the audience that it serves (faculty/advisor/student). For example, advising or student success groups will have views of student performance across a student’s course history in ways that faculty may not.  Demographic information may be incorporated for specific views that provide the basis for surfacing a deeper understanding of STEM education or DEI. As an important side note, each member institution will have to decide how to utilize these marts best, what data to surface, and what audience is best suited to view that data.

The second area of focus of the Taskforce was to provide broad, aggregated information on the composition of a course.  This focus would involve again thinking about views into the data based on audience (faculty, advisors, and students) and would look to provide, for example, a faculty view of a particular student profile within a course, such as what courses a student has taken before the class; or, aggregated at the course level, how do students compare to their peers who have strong STEM skills (derived from previous courses taken).

Once the Taskforce data marts have been created, the DSS team will provide documentation, including Entity Relationship Diagram (ERD) and a discussion of the underlying queries that reveal and discuss how the data marts were built and the rationale behind their construction. Ideally, these data marts will focus on the Taskforce imperatives and be flexible for broader application, depending on institutional priorities and data governance.

Unizin anticipates completing the five foundational marts and dashboards for all member institutions whose UDP can accommodate these marts by the end of the calendar year and beginning rollout on the Taskforce marts in January 2022.

Lauren Marsh (UMN) will be the incoming chair-elect for the Unizin Teaching & Learning Advisory Group. Jennifer Smith (UF) will become the new Unizin T&L chair in mid-September after serving in the chair-elect role last year.

Lauren is a Senior Academic Technologist at the University of Minnesota. Since joining the University of Minnesota’s academic technology unit in 2003, Lauren has served in many roles, including Manager, Service Owner, and Senior Academic Technologist. She managed the University’s 18-month Faculty Fellowship Program for ten years; the program focused on cultivating faculty leaders in academic technology.

Currently, Lauren supports the University’s efforts to create a culture of learning analytics: she co-chairs the Learning Analytics DiaLOG group, which guides the operationalization of learning analytics as educational practice at the University of Minnesota. She coordinates learning analytics pilot efforts  (MyLA and ECoach). Lauren has been active in Unizin since 2015 and has been co-chairing the Unizin CTL Subcommittee with Gwen Gorzelsky.  The CTL Subcommittee is developing “Pathways to Learning Analytics,” a learning analytics curriculum for faculty.  They also support efforts to identify faculty questions and priorities for learning data through structured conversations, which will begin this fall. A proud English major, Lauren earned her doctorate at the University of Minnesota in 2003 and still keeps a tall stack of novels on her nightstand.

For more on Jennifer Smith, please see the following article.

Maggie Jesse, Unizin’s current chair emeritus, will remain the Unizin T&L Advisory Group’s Board contact. Many thanks to Maggie for all of her advice and encouragement during this last year.  

Jennifer K. Smith is the Director of the University of Florida’s Center for Teaching Excellence, where she co-chairs UF’s Teaching Innovations committee. She has served as the Associate Director of Course production for UF Online and as manager of Instructional Design Services at UF’s Center for Instructional Technology and Training (CITT). Before her work at CITT, Jennifer was an Associate Professor with the University of Florida Theatre and Dance Department, where Jennifer supervised costume construction for over 80 productions. She earned her Master of Fine Arts degree in Theatre Production at the University of North Carolina Chapel Hill.  You need to ask her how she learned men’s tailoring while working in the costume shop at the Guthrie Theatre in Minneapolis. This skill served her well when she was hired to create costumes for Prince and The Revolution (before The Revolution was disbanded). The costume shop made the broad-shouldered “gangster-style” silk suits a staple of Prince’s wardrobe during Jennifer’s time.

Jennifer has been very engaged with Unizin and looks forward to furthering Unizin’s ability to work collectively towards quality teaching and learning tools and practices. The foundation of the T&L committee has been one of sharing and collaboration since the beginning. During the coming year, Jennifer would like to cultivate the cooperative spirit of the group, including:

• Facilitating the connections between individual members and Unizin staff.
• Identifying time-efficient ways for the subcommittees to “cross-pollinate” regarding ideas and projects.
• Supporting the development of “welcome wagon” resources and activities for new T&L members.
• Continuing work on the organization and documentation of the T&L committee resources and processes.

Jennifer shares her appreciation and “the opportunity to connect with folks in the Unizin consortium who are as passionate about teaching and learning as I am.  The sharing of ideas, resources, and challenges have been priceless during the past year and a half. I am honored to help the Teaching and Learning group to continue to grow.”  Please join us in welcoming Jennifer in this crucial role!

Unizin launched a wide recruitment net and generated over 180 applicants.  Twelve early finalists were then screened in the first round of interviews with Unizin’s leadership team.  The following steps include:  

1. Five or six candidates meet with four board members who will provide feedback to Unizin’s CEO in September.
2. Two to four applicants in October will:
1. Meet with Unizin’s CEO and CFO for a final interview.
2. Present to all of Unizin’s staff.
3. References will be screened

At this stage, Unizin is quite confident that a new CTO will be joining us by November at the latest.

All the Unizin membership fees for the fiscal year 2022 have been collected.  Historically it has always been mid-fall before the majority of membership fees have been posted.  Thank you all!