Programmatic assessment, how can I use it in my own course?

In recent years, the popularity of programmatic assessment has increased significantly. More and more programmes are implementing a programmatic approach to assessment (Baartman, van Schilt-Mol, & van der Vleuten, 2020). But how do you translate this concept to your own course?

Lubberta de Jong (Educate-it), Marlies van Beek & Lonneke Schellekens (Onderwijsadvies & Training) discuss the principles of programmatic assessment at the curriculum level, and give you concrete tips on how to apply elements of this in your own course.

Principles of programmatic assessment

Programmatic assessment (Van der Vleuten et al., 2012; Van der Vleuten, Schuwirth, Driessen, Govaerts, & Heeneman, 2015) is a new approach to assessment that focuses on optimising the learning and decision-making function.

No decision moment based on a single assessment

The main focus is the student’s learning. Instead of the ‘jumping through summative assessment hoops’, assessments should promote learning. This is done by disconnecting the pass/fail decision from one single assessment moment. Instead, the student is provided with meaningful and rich feedback to help them develop themselves. Ultimately, multiple moments of assessment over a longer period of time lead to a pass/fail decision. This decision is made to decide if the student has met the desired learning outcomes. As such, there is no decision moment based on a single assessment; the decision is made based on rich information on the student.

Additionally, the student is seen as the owner of their own learning process. The student is challenged to be aware of their own strengths and weaknesses. A mentor or coach facilitates this process, helping the student formulate learning goals and monitor progress.

Implementation in your own course

The principles are designed on the curriculum level; however, this does not mean that you as a lecturer cannot apply them in some fashion. For example, in a course you can consider using assessment to promote learning and when doing so focus on the development of the student.
Below we discuss four different themes with concrete tips that you can use in your own course.

Theme 1: Generating useful information

Assignments, professional products, presentations, feedback from other students, but also test results provide both you as a lecturer and the student with information on the learning process. By providing rich and meaningful feedback on these moments you can give students insight into their learning process (where am I now) and in the needed subsequent steps (where am I going and how do I get there).

Individual grades are very poor sources of information

Feedback can be quantitative (for example an overview of scores) but also qualitative wherein you as provider of feedback put your own findings into words (narrative feedback). If, for instance, you want to work with short quizzes you can start by thinking about what the student needs to improve their presentations instead of only providing the information on whether a question is answered correctly or incorrectly. We know that individual scores or grades are very poor sources of information (Butler, 1987; Chan & Lam, 2008; Narciss & Huth, 2004) and that specifically qualitative feedback promotes learning as it provides a direction for improvement.

To be able to concretely think about subsequent steps (where am I going and how do I get there) it is important that the goals of the assignment/quiz/presentation are clear and that the feedback is aimed towards these goals (Nicol & Macfarlane-Dick, 2006; William, 2005).

In addition to thinking about what information you can give to the student on an assessment moment (or students can give each other), it is also important to think about when to use it. Ensure that you have ample opportunity for the student to receive the feedback and apply it in an exercise/presentation. This way you can bring the moment in which you provide feedback forward in the course to, for instance, halfway through the course. If you provide proper feedback (written or verbal) at that moment and the student is able to apply it, it is more likely that you will need to spend less time on the final assessment.

Finally, also consider the ‘’less is more’’ principal (Watlin et al., 2012). This entails that it is better to provide occasional but good quantitative feedback rather than regular but poor qualitative feedback (not enough information). 

Theme 2: Defining your role as a lecturer

To be able to use the feedback effectively in the learning process, it is important that the student reflects upon it (Sargeant, Mann, van der Vleuten, & Metsemakers, 2009). This process can be supported by a mentor or coach (Knowles, 1975). This requires a different definition of the role of a teacher, more along the lines of a coach. It is not necessary to appoint mentors or coaches; lecturers, too, can give substance to this role in their daily teaching practice.

So, how do you apply this role? If we once again use the example of a short quiz, you could ask your students to reflect on their mistakes, and provide information on the underlying theories, concepts, or learning goals that are at the basis of the quiz question.

You can also link the reflection to a larger assignment or project in your course. Invite students before an assignment (for instance a group assignment which results in an advisory report) to formulate one or more personal learning goals and share these with their fellow students and the supervising lecturer. The fellow students and the supervising lecturer provide feedback during the process which the student can use to develop themselves. After finishing the advisory report, the learning goals are once again consulted, and the development of the student is discussed. Optionally, the development of personal learning goals can be part of the final assessment.

Theme 3: Student involvement  

A third perspective on how you can optimise your students’ learning process in your own teaching, is the active involvement of students in the assessment and their own learning. By considering the student as an active partner in the assessment process, the student learns how to make complex decisions regarding the assessment of their own and other’s work (Boud & Falchikov, 2006).

You can stimulate active involvement of students by, for example, including activities such as self and peer assessments in your course. Another option is having students make an assessment form (rubric) for assessing a certain product or skill, or by discussing existing rubrics. Through these kinds of activities, the student is actively processing knowledge, and this increases their understanding of the criteria that contribute to a good result and how they can be applied (Tai et al., 2018).  

Students are actively involved in their learning when they are given the opportunity to take responsibility for directing their own learning (Earl, 2013). For example, by offering assignments in which they can plan, monitor, and evaluate their own learning, or by offering multiple assignments for them to choose from, so that students have the choice to adapt the assessment to their own goals and interests.

An important condition for successfully implementing these kinds of activities is that there also needs to be time for familiarising students with these kinds of assessment. As such, it must be what is expected of the student and new skills may have to be learned for assessing and managing their own assessment and learning process.

Theme 4: Beyond your course

Finally, also look outside of your own course for optimising your students’ learning processes. How does your course build upon previous courses and what can students take from your course to other courses? Perhaps it is possible to repeat the study material of a previous course in your course, so that it becomes more memorable, and the student becomes more familiar with the structure and composition of the curriculum.

If, for example, the same skills will be assessed or practiced you could check which feedback the students received so that you can use this as a starting point in your own course. If the same skill is covered (for example, writing skills or presenting), you could make the rubric reach across courses, thus allowing the learning process to be monitored over multiple courses.

Auteurs

• 

  drs. Marlies van Beek

  Email: m.e.vanbeek@uu.nl

  Phone: 06 49 880 229

  Social and Behavioural Sciences - Educational Development & Training

Sources

Baartman, L., van Schilt-Mol, T., & van der Vleuten, C. (2020). Programmatisch toetsen: Voorbeelden en ervaringen uit de praktijk. Amsterdam: Boom.

Boud, D., & Falchikov, N. (2006). Aligning assessment with long‐term learning. Assessment & evaluation in higher education, 31(4), 399-413.

Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. Journal of Educational Psychology, 79, 474-482. Doi:10.1037//0022-0663.79.4.474

Chan, J. C. Y., & Lam, S. (2008). Effects of different evaluative feedback on students’self-efficacy in learning. Instructional Science, 38, 37-58. doi:10.1007/s11251-008-9077-2

Earl, L. M. (2013). Assessment as learning: Using classroom assessment to maximize student learning. Corwin Press.

Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers.

Narciss, S., & Huth, K. (2004). How to Design Informative Tutoring Feedback for Multimedia Learning. In

Niegemann, H., Brunken, R., & Letner, D. (Eds.), Instructional Design for Multimedia Learning (pp. 181-196). Munster: Waxmann.

Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in higher Education, 31, 199-218. 

Sargeant, J. M., Mann, K. V., van der Vleuten, C. P., & Metsemakers, J. F. (2009). Reflection: a link between receiving and using assessment feedback. Advances in Health Sciences Education, 14(3), 399-410.

Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481.

Van der Vleuten, C., Schuwirth, L., Driessen, E., Dijkstra, J., Tigelaar, D., Baartman, L., & van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205-214. 

Van der Vleuten, C., Schuwirth, L., Driessen, E., Govaerts, M., & Heeneman, S. (2015). Twelve tips for programmatic assessment. Medical Teacher, 37(7), 641-646. 

William, D. (2005). Keeping learning on track: Formative assessment and the regulation of learning. Making Mathematics Vital: Proceedings of the Twentieth Biennal Conference of the Australian Association of Mathematics Teachers. 

Watling C., Driessen E., van der Vleuten C.P., Lingard L. (2012). Learning form clinical work: The roles of learning cues and credibility judgments. Medical Education, 46, 192-200.