The UDP: Where the Rubber Meets the Road for Student Success
The Unizin consortium uses the power of the collective to create opportunity from digital disruption. Unizin institutions are utilizing the vast quantities of data streaming from digital learning environments to drive leading-edge student success initiatives.
The Unizin Data Platform (UDP) is the single-largest living teaching and learning laboratory within higher education, catalyzing research, teaching and learning tool development, and pedagogical innovation across Unizin member campuses.
The UDP represents a massive exercise in data standardization. Data within the UDP is modeled to be tool-agnostic, leveraging the Unizin Common Data Model to standardize context data from student and learning information systems, as well as 1EdTech’s Caliper specification to harmonize learning event data from a growing list of vendor partners. This standards-based approach has enabled the UDP to grow exponentially, now containing hundreds of terabytes of learner data – with more added daily from across 60 Unizin-member campuses serving more than 700,000 students.
A Platform-as-a-Service (PaaS), the UDP is designed for flexibility within a data ecosystem. Residing in the Google Cloud platform, the UDP supports various analysis, business intelligence, data science, and data virtualization solutions. Simple integrations with other cloud providers allows UDP data to be accessed and made available in the appropriate formats and environments to suit the needs of each member institution.
The examples below, which have been presented at Educause and 1EdTech and can be viewed on Unizin’s YouTube channel, demonstrate just a handful of the many ways Unizin members are utilizing the UDP to inform decisions and impact the learning lives of students, their academic advisors, and faculty every day. As these members, and others like them, move from theory to practice, they are sharing the real-world impact of their student success initiatives.
Penn State University
Penn State’s Data Empowered Learning (DEL) team, part of Teaching and Learning with Technology (TLT), is a research and development group that leverages data science methods to analyze large institutional data sets in order to generate individual scale insights. These insights provide a near real-time view into a student’s learning activity, supporting day-to-day sense-making and decision-making for both advisors and faculty. This comprehensive and robust approach to student data – with the UDP at the center – is providing support for data empowered student success initiatives at Penn State University.
The UDP provides Penn State with Caliper event data (student and course interactions) and context data (student and course metadata) from key learning tools such Canvas, Top Hat, and Kaltura; and the team hopes to include additional Caliper data from newly supported learning tools in the UDP including Zoom LTI and Packback. Refreshed every 24 hours, this data is pulled from the UDP and is transformed through data science processing to generate analytic insights, which are then delivered to different audiences across the University:
Elevate is an adviser-facing application that leverages course-level activity data to help inform advisers when students are experiencing disproportionate decline in their online activity compared to their peers in the same course. These comparisons drive different levels of alerts to draw an adviser’s attention to specific students who may need proactive outreach during the semester. This approach enables advisers to tailor outreach to the specific circumstances of individual students and strengthens advisers’ ability to support the right students at the right time.
Course Insights: takes the data architecture and activity calculations of Elevate and repurposes them in an LTI application designed for instructors with additional analytics added to support reflection on practice throughout the semester. Course Insights helps instructors understand where students are in their learning, how they are interacting with online course materials, and what courses they take next. The online activity data, available in week two of the semester, has proved to be particularly useful during the early progress reporting window to help instructors identify students who are struggling prior to the first midterm and grade checkpoints.
Both Elevate and Course Insights are having a measurable impact on students, advisers, and instructors at Penn State:
- More than 400 academic advisers actively use Elevate to support students on their advising rosters.
- Course Insights is currently being used by a diverse set of 145 instructors, representing 47 academic departments, and 12 Commonwealth campus locations.
- 50% of all Penn State students are currently being advised by an adviser using Elevate or instructor using Course Insights.
Beyond the statistics – these initiatives, and others like them are shifting the culture at Penn State. Leveraging data in a way to provide proactive outreach to support student success has never been more important. Learning data can form the central part of a growing student success network and through shared insight into this data, instructors, advisers, and students can work together to achieve a better, more inclusive learning experience. This approach at Penn State has generated significant results and promoted a culture of data-informed decision making for student success.
UNIVERSITY OF IOWA
The University of Iowa is taking a student-centric approach to data-informed decision-making. The research and application development team, in conjunction with faculty, set out to address a common learning challenge experienced by students in the early stages of their academic careers. Large introductory courses can often prove vexing to first- and second-year students. Many students coming out of high school simply have unrealistic expectations of how they’ll perform at the college level. They are unfamiliar with grading methodologies like curves. And young students often have not yet developed mature metacognitive monitoring capabilities, which means they will likely miss opportunities to improve their own performance.
Without tools to recognize and monitor their own academic performance, and make adjustments accordingly, these young students can become frustrated and demotivated, missing opportunities to improve their grades in foundational courses which can have long-term consequences.
Research supports the notion that students can achieve better outcomes when they develop skills and insights to self-regulate their own learning. The team at the University of Iowa recognized the opportunity to use the UDP to aggregate, analyze and visualize student activity data to provide students with confidential insights directly connecting their actions and outcomes. They are using activity data to create effective visualizations and communications tools to help students throughout the cycle of self-regulated learning: from planning and setting goals, to identifying strategies to monitor their performance, reflecting on their achievements to adapting to new challenges.
ELEMENTS OF SUCCESS at the University of Iowa provides real-time feedback on performance to help students achieve their desired outcomes. Visualized data shows students their performance in real time, in comparison to their peers. The platform can project an estimated grade if they maintain their current trajectory, and provides ways to improve their performance, from utilizing available campus resources, to tips on actively participating in lectures and discussions, better listening, and note taking.
The easy-to-understand visual format provides insights into their overall course progress and progress by grade category. The system has proven particularly useful in courses with high enrollment and in courses graded on a curve, as well as for courses with historically established grading procedures and those in which there is a frequent disconnect between student expectations and actual performance. Elements of Success is being deployed within the finite window of opportunity early in the semester to provide enough time for students to make adjustments.
The links to student visualizations are made available within the LTI within the first two weeks of a course, immediately after the first graded assignment. The visualization provides an at-a-glance view of the student’s weekly progress and performance and projected final grade if they maintain their current performance. Students are regularly encouraged to set goals that include a set of actions – chosen by the instructor – to achieve those goals. ONLY the students see their goals, and they are offered regular opportunities to reflect on and revise those goals. Instructors never see the goals chosen by the students, as the system is designed to build their metacognitive monitoring capacity. Instructors have the option to provide detailed feedback intended to encourage students to take actions with proven results. Campus resources are highlighted and tuned to each student profile.
Elements of Success has been evaluated and researched extensively by staff and faculty at the University of Iowa. Among students with a projected class performance of D/F, 71% of those who used Elements of Success achieved a better grade than projected while only 37% of those who did not use the platform performed better than projected. What’s more, only 9% of students using Elements of Success performed more poorly than projected, compared to 24% of their non-using counterparts. These results hold when researchers control for GPA, and when evaluating a subset of underrepresented and minority students.
Indiana University – Purdue University Indiana (IUPUI)
Without 100% persistence or graduation, IUPUI needed a comprehensive, truly early-alert system to focus its limited advising resources on the right students, at the right time, with the right proven strategies. Previously, IUPUI had investigated a vendor-deployed predictive analytics solution to support student success initiatives. However, the lack of data transparency in the black-box approach of that solution did not engender trust from advisors. Without transparency, the platform was underutilized by advising staff who were reticent to follow the recommendations and actions prompted by the system.
With access to the UDP, IUPUI piloted a homegrown solution utilizing UDP data and in a randomized, controlled trial to determine whether the solution warrants the investment in time and resources to support wide-spread deployment.
The randomized control trial for a learning analytics intervention evaluated the population of students with the lowest activity in the IUPUI LMS across two-week time spans during a six-week period in the fall 2020 semester. Trial participants were selected from the University College student population of pre-major students – which included a disproportionate population of first-generation, low income, and students of color – and randomized into treatment and control cohorts.
Students in the treatment group were flagged to IUPUI advisors for targeted advising intervention, while students in the control group were not flagged. Students in the treatment group achieved a significantly higher term GPA on average than their control group peers (2.01 vs 1.66). More importantly, IUPUI researchers documented a 10-percentage-point difference in persistence to the next semester (67% treatment group vs. 57% control) and a 14-percentage-point difference in persistence to the following year (64% treatment group vs. 50% control).
The IUPUI team carefully selected the variables used to determine LMS activity, to provide advisors with the insight into the score that they could not access with the previous vended, black-box solution. The LMS activity score used for flagging was calculated using easily understandable variables: assignments submitted out of assignments due, total active minutes in the LMS across two weeks, and total active LMS sessions across two weeks.
WHAT’S NEXT?
Penn State, Iowa and IUPUI are just three examples of how Unizin members are utilizing the UDP to rethink the dynamics of student success initiatives. Whether it’s providing students direct insight into their own performance or helping advisors and faculty provide better guidance and intervention at the optimal moments, the UDP is informing decision making and having a direct impact on student success.
The tools, applications, and capabilities of the UDP will only continue to evolve and grow. And with it, our ability as data scientists, educators, and innovators will expand as we chart new courses for the students we serve.