DACE - a dashboard for course evaluation
DACE is a dashboard that provides a comprehensive overview of course evaluations, student activities in a course, course development history, and notes from course developers. By visualizing this information in one place, it will be easier to update and improve courses.
Background and hypotheses
In 2021, the decision was made to start using Caracal as a university-wide evaluation system. This required a technical redevelopment of the system. Prior to this, an inventory was made within the faculties of the needs with respect to course evaluations. Discussions with various stakeholders revealed that in addition to Caracal outcomes, there was also a need for process and policy information, information about (student) experiences, opportunities for improvement and educational quality. Also mentioned were panel discussions, key figures, test analyses, correspondence with the program committee, teacher reflection and unsolicited feedback. Looking at multiple sources in context gives a much richer and more complete picture of teaching. The proposal in the advisory report was therefore to purchase or develop a platform that serves as a dashboard and brings together various sources of information. More and more sources can be linked to this platform step by step, so that the complete quality assurance file of an educational unit (such as a course or program) is kept together. In this way the information about that particular education can be viewed in context.
In this project, a first pilot will be started with such a platform: the Dashboard for Course Evaluation (DACE) with the following core functionalities: 1) visualizations of Caracal evaluations, 2) visualizations of student activity in the learning management system (LMS), 3) historical progression, and 4) a link to notes and other documents.
The previously written advisory report identifies several long-term goals. In this pilot, we are making a start on realizing those goals by trying to achieve the following: 1) the course coordinator has a richer and more complete picture of the course than without DACE; 2) the course coordinator gets additional information; information needs are met (usefulness) and this information is easy to use (usability); 3) the course coordinator gets concrete insights (“action perspectives”) on how the course can be modified; 4) by creating an evaluation plan, and communicating this to students, we hope to have a higher response rate to Caracal (we are aiming for 75%).
Realization of DACE
The dashboard has been developed in powerBI and combines data from Caracal, Blackboard and documents with course notes stored in OneDrive.
Evaluation
Course coordinators will start working with DACE in blocks 1 and 3 of academic year 2024-2025. The evaluation will be aimed at investigating whether course coordinators find the dashboard usable and useful, in terms of generating actionable insights for (re)designing courses. The course coordinators will be interviewed and will fill in several questionnaires to this end.
Timeline
Academic year 2024 - 2025.
The pilot will run in academic year 2024 - 2025.