Schubert, Sebastian; Ortwein, Heiderose; Dumitsch, Antje; Schwantes, Ulrich; Wilhelm, Oliver; Kiessling, Claudia:
A situational judgement test of professional behaviour: development and validation
In: Medical teacher: an international journal of education in practice for educators in the health science, Vol. 30 (2008), No. 5, pp. 528 - 533
2008article/chapter in journal
PsychologyFaculty of Educational Sciences » Institut für Psychologie
Title:
A situational judgement test of professional behaviour: development and validation
Author:
Schubert, Sebastian;Ortwein, Heiderose;Dumitsch, Antje;Schwantes, Ulrich;Wilhelm, Oliver;Kiessling, Claudia
Year of publication:
2008

Abstract:

Aims: To describe the development of a single best answer multiple choice situational judgement test (SJT) to assess medical students' ability to judge professional behaviour; to present results of an expert validation of the SJT; to describe experiences with two different validation formats. Methods: Based on educational objectives and work situations concerning professional behaviour, a SJT with 17 vignettes and 35 questions was developed. Best answers were developed according to available evidence. Forty-four experts validated the answers using either a rank order or a rating scale. Results: For sixteen questions in the rating group, thirteen questions in the ranking group and twelve questions in both groups more than two thirds of the experts agreed on a best answer. In the ranking group some experts found it difficult to commit themselves to a single best answer as instructed. In the rating group some experts did not mark any of the proposed answer options as adequate. Discussion: The definition of the construct of and experts for professional behaviour remain a challenge. Both validation scales have advantages and disadvantages. The two thirds criterion is somewhat arbitrary and lower or higher agreement may be acceptable or necessary depending on the purpose of the assessment.