Evaluation of the proficiency test for dentists educated outside the European Union and EEA

The Proficiency test is the most common pathway for dentists, educated outside the European Union and the European Economic Area (EEA) who want to be licensed to practice dentistry in Sweden. The test involves passing a series of examinations designed to assess whether a dentist meet Swedish standards for licensure. Since exams directly affect the individuals who take the exams and patient safety, the assessments must be robust and provide credible evidence of candidate competency. The test consists of three main parts: the first is a theoretical examination, which evaluates candidates' knowledge and ability to apply that knowledge to dental scenarios, which is assessed through a computer-based examination. The second is a clinical skills examination, which focuses on clinical and communication skills in two parts: a 4-station Objective Structured Clinical Examination (OSCE) and manual skills on a dental manikin. Lastly, candidates must complete a six-month clinical practice under supervision, which assesses their professional practice in a real-world clinical setting. Given the high stakes of these exams the assessments must be robust and provide credible evidence of competency. There is limited research on proficiency tests for dentists educated outside the European Union and their outcomes, in Europe and in Sweden. The recognition and licensure testing processes are vital not only for ensuring high-quality patient care and safety within individual countries but also from a wider European perspective as the current lack of standardisation can lead to unequal recognition processes, hindering harmonisation across the European Union and the EEA.

The theoretical and clinical examination's ability to adequately discriminate between candidates who meet the qualifications of the dental degree, from those who do not, and factors effecting results was a tested in Study I and Study Il. In Study I, test results from seven theoretical examinations between 2018- 2019 (n = 316) were collected, along with demographic data. In Study II, test results from theoretical and clinical skills examinations between 2018-2022 was combined with self-reported language proficiency. Based on the tests we found that the assessments positively differentiate candidates who do not meet the requirements of the professional qualifications of the dental degree in the theoretical examination and in the Objective Structured Clinical Examination (OSCE) assessment tested in the clinical skills examination.

Licensure examinations are limited to a sample of a full curriculum, and although participants are proficient enough to pass - the examinations cannot ensure that passing participants are equally proficient within all dental disciplines. The tests showed that age had a significant influence on results in both theoretical and clinical examinations. Language difficulties were also identified as a potential barrier to performance in the theoretical and practical examinations.

The reliability and consistency of setting pass-mark in theoretical examinations using the three-level Angoff methodology was tested In Study I and Study III. In Study I the Angoff item ratings by panels between 2018-2019 were collected. In Study III, Angoff ratings across three theoretical examinations were collected from a reference panel at The Department of Dental Medicine, Karolinska Institutet and compared to item ratings from an invited Angoff panel for a dental program from another Swedish university. The tests of inter-rater reliability and credibility indicated that a three-level Angoff method is a defensible method for setting the pass mark for the theoretical examinations. The results from the comparison of the two independent panels showed that the three-level Angoff ratings produced similar standards for minimal qualification.

In Study IV, we explored the experiences of dentists who participated in the theoretical and clinical assessments. Survey responses from six theoretical examinations (n=486) and four clinical skills examinations (n=190) were collected between 2022-2024. Systematic text condensation was used for analysis of written responses from post-examination surveys to describe participants' experiences of participation in the proficiency test. The descriptions from dentists revealed a need for guidance in preparation, language barriers, stress, and a perceived misalignment between exam content and clinical practice.

The studies highlight the necessity of using multiple assessment formats to provide enough evidence that can be used to decide whether a participant has adequate competent for licensure. The assessments must be seen as part of a system of assessments where the three components of the Proficiency test are all important as they complement each other. The findings of this study are relevant for Sweden and other nations seeking to improve re-certification.

List of scientific papers

I. Dalum J, Christidis N, Myrberg IH, Karlgren K, Leanderson C, Sandborgh Englund G. Are we passing the acceptable? Standard setting of theoretical proficiency tests for foreign-trained dentists. Eur J Dent Educ. 2023 Aug;27(3):640-649.https://doi.org/10.1111/eje.12851

II. Dalum J, Christidis N, Häbel H, Karlgren K, Leanderson C, Englund GS. Clinical skills examination as part of the Swedish proficiency test of dentists educated outside of the EU/EEA. Eur J Dent Educ. 2024; 28: 880-888. https://doi.org/10.1111/eje.13022

III. Dalum J, Paulsson L, Christidis N, Andersson Franko M, Karlgren K, Leanderson C, Sandborgh Englund G. Consistency between inter- institutional panels using a three-level Angoff-standard setting in licensure tests of foreign-trained dentists in Sweden: A cohort study. [Submitted]

IV. Dalum J, Brodén J, Johannsen A, Sandborgh Englund G. Barriers to showing competence: foreign-trained dentists' experiences of re- certification examinations in Sweden. [Manuscript]