The F-ACT dashboard
The F-ACT dashboard has been specifically developed to address individual differences in academic writing competencies and support needs among students. The dashboard collects all the received assessments and feedback conveniently in one place and shows which competencies and related skills a student scores high and low on. This way the students know what they are doing well and where they need to pay more attention. In addition, the F-ACT dashboard provides a number of suggestions for resources the students can consult for each skill.
Background and hypotheses.
There are several examples of competency dashboards for students available in literature that demonstrate the feasibility and usefulness of such systems (Villamane et al, 2018; Nussbaumer et al, 2015;Grann & Bushway, 2014). For example, there are indications that students who use a competency dashboard demonstrate competencies at higher levels and persist in their program at greater rates (Grann & Bushway, 2014).
The aim of the F-ACT dashboard is to support students’ development of academic writing competencies and student ownership of feedback practices and study strategies. The F-ACT dashboard is available over multiple courses so that competency development becomes visible over a longer time. The F-ACT dashboard aggregates the competency and skill score based on teacher and peer feedback, as well as self-assessments (but also enables splitting these scores) and provides resources for further competency and skill development.
We hypothesize that the use of the F-ACT dashboard may increase students’ ownership of their own study behavior, resulting in more targeted requests for feedback from teacher and peers, and more targeted division of study time over the competencies and skills that require further development (Winstone et al, 2017; Little et al, 2024).
The project will run in the pre-master and bachelor programs of pedagogical sciences.
Realization of the F-ACT dashboard
The F-ACT dashboard is developed in powerBI. The data that are visualized in the dashboard are extracted from the educational application FeedbackFruits and consist of textual feedback and Rubric scores. A list of useful resources is provided in the dashboard as well (extracted from an Excel file provided by the teachers), such as links to videos, pdfs or websites.
Evaluation
In the evaluation of the F-ACT dashboard we will focus on three topics:
- Usage level
- Usability
- Usefulness
Participation in the evaluation will be based on consent.
Usage level.
We consider a good usage level of the F-ACT dashboard 80% of the students that participate in the academic writing skills courses in the premaster and bachelor program pedagogical sciences, which is roughly 280 students. Based on how the dashboard will be introduced in the program (see description under timeline), we aim to achieve this number in the academic year 2025 – 2026.
Usability.
We are aiming for an average score of at least 5 on the technology acceptance model 4 (TAM4; Hornbæk & Hertzum, 2017) at the end of the academic year 2025 – 2026. By including students and teachers in the development process, we believe it will be feasible to reach this score.
Usefulness.
The dashboard will be considered useful if students who use the dashboard show better competencies, have better grades on their academic writing assignments, are able to request more specific feedback, are more capable in handling and giving feedback, and managing their study time (e.g. focus on competencies that need further development). We will assess the usefulness by combining data about usage level of the dashboard with outcomes of the feedback literacy behavior scale (Dawson et al, 2023) and the feedback orientation scale (Linderbaum & Levy, 2010).
Results 2023 - 2024
In the academic year 2023 - 2024, we piloted the dashboard in an undergraduate (second year bachelor + premaster) course. The dashboard was introduced to 350 students. Most students did not use the dashboard on a regular basis. For example, in the last three months only 12 students logged in (+/- 3.5% of the students). On average, these 12 students accessed the dashboard 6 – 7 times in these three months. In combination with the low response rate (1 - 6% of the students) on the usability and usefulness questionnaires we could not properly evaluate the F-ACT dashboard.
Based on the limited information we received, we identified the following reasons for the limited uptake:
- The dashboard relied on data that were extracted from FeedbackFruits. When we introduced the dashboard at the beginning of the academic year, there was no FeedbackFruits data available from previous years. It only started to contain information after the first half of the course. This may have demotivated students to consult it;
- FeedbackFruits had been newly introduced and not fully adopted by all teachers. Therefore, there was no academic writing skill information available in the dashboard for some of the students;
- Using the dashboard was voluntarily and not incorporated into the curriculum, which led to low awareness of the existence of the dashboard despite the introduction session.
In the upcoming academic years, we will focus on further integrating FeedbackFruits in the academic writing skill courses. Next, we will gradually embed the dashboard more in the courses, so students must use it to complete an assignment. Lastly, we will promote the dashboard by creating a promotional video and sharing that video with the students.
Timeline
Academic year 2024 - 2025.
We will focus on further embedding FeedbackFruits in all the academic writing skills courses of the pedagogical sciences program. We will introduce the F-ACT dashboard in block 3 & 4 when the bachelor thesis is written.
Academic year 2025 - 2026.
We expect that FeedbackFruits is now integrated in the academic writing skills courses of pedagogical sciences and we will roll out the F-ACT dashboard over all academic writing skill courses.
Academic year 2026 - 2027.
The dashboard is expected to be fully embedded in the academic writing skill courses of pedagogical sciences program.
Possibility for internship
Yes. You can contact team LA or the project leaders about internship possibilities.