Study advisors dashboard
One of the problems students may face is study delay. Study delay can have negative consequences for a student, such as financial problems. The study advisors at the Utrecht University (UU) noticed that the number of students with study delay was increasing as well as the length of study delay. Therefore they asked if it would be possible to help students at an earlier stage and reduce (or even prevent) potential unwanted study delay.
Background and hypotheses.
On average only 69% of the bachelor students get a degree within the n+1 study years. The overall goal of this project is to try to identify which students have a higher risk of having a study delay. However, it is not clear yet which predictors relate to study delay. So the first aim of this project is to find the indicators that relate to potential study delay.
Currently, study advisors manually create lists of students to contact, email them one by one, and have to keep track of who they already spoke to. It would be great to have a system that, once we know which predictors to use, would automatically alert the study advisors and support them in sending and keeping track of the emails. Hence, the second aim is to make the process of identifying and contacting students more efficient by developing a study advisors dashboard that provides an overview of the students under the care of a study advisor.
We expect that the dashboard fulfills a useful role in study advisors’ monitoring of students and decision making of which students to invite for a meeting. We additionally hypothesize that study advisors will be able to pinpoint cases of study delay at an earlier stage.
Realization of the dashboard
The dashboard is developed in powerBI. The predictions that are visualized in the dashboard are based on data from the student information system, such as ECs.
Timeline
Academic year 2023 - 2024.
A pilot was run in this academic year to test the usability of the dashboard.
Evaluation
For the evaluation of the dashboard, we focussed on:
- Usability and use in practice
- Usefulness
Participation in the evaluation was based on consent.
Usability.
Usability of the dashboard was evaluated with the Usefulness, Satisfaction, and Ease of use (USE) questionnaire (Gao et al, 2018 and Lund, 2001) and the system usability scale (SUS)(Brooke, 1996). Additionally, an interview was held with the study advisors to get more detailed information about the usability of the dashboard and how study advisors used the dashboard in their daily practice.
Usefulness.
For this topic we used the system usability scale and interviews with the study advisors. During the interviews, we asked them if they thought the predictors gave useful (new) information and if the dashboard helped them with prioritizing students. We additionally wanted to analyze if the use of the dashboard will lead to reduced study delay.
Results 2023 - 2024
Usability scored an average of 3.3 out of 5, but advisors initially found the dashboard overwhelming. Adjustments, such as reducing visible information, were made in Round 2.
The interviews revealed three main characteristics that influenced the usability of the dashboard: dashboard characteristics (usability and accuracy), student advisor characteristics (including data literacy) and institutional factors (including available time and resources).
The recommendations for follow-up projects are: explore the role of learning analytics throughout the counseling chain, invest in adequate resources and training around data-informed work, and potentially explore additional data sources as predictors of study delay.
Possibility for internship
No