Correlation between clinical clerkship achievement and objective structured clinical examination (OSCE) scores of graduating dental students on conservative dentistry

Article information

Abstract

Introduction

Assessment is an important issue in determining the outcome of students' educational courses in view of their learning objectives.1,2 Observation is the main method of assessment, which is customary and widespread in education of health professionals.3 Dental students' clinical competency is usually rated in terms of the number of case observations and/or assistance with a professor's treatment, and their performance of procedures in the student dental clinic. Nevertheless, as stated by the well-known proverb, "Practice makes perfect. The more you practice, the better your skills are." This simple, quantitative and unstructured type of learning and assessment has numerous deficiencies in terms of reliability, objectivity, and variability.4

The objective structured clinical examination (OSCE) is a widely known assessment method that aims to overcome these limitations. OSCE was developed and introduced by Harden in 1975 in a Scottish medical school to assess students' clinical competence.5 The United States Medical Licensing Examination introduced step 2, Clinical Skill, in June 2004, in much the same way as the OSCE, which was previously known by different names, such as Clinical Skill Assessment (CSA) or Clinical Performance eXamination (CPX).6 Generally, OSCE is a series of standardized assessment stations, requiring students to perform a number of tasks based on clinical situations, in which either standardized or simulated patients (SPs), or alternative simulators, are used. The examinees are assessed using the same stations and rating scheme within the same timeframes. Examiners assess examinees' application of knowledge to practice, such as history taking, physical examination, communication, interpersonal skills, and performance of appropriate procedures. This contrasts with traditional examinations that assess the recall of knowledge, which may be irrelevant to clinical practice.

Dental schools in various countries use OSCE for formative and/or summative assessment of students' clinical competence.7-8 The National Dental Examining Board of Canada introduced OSCEs into the certification process in 1994.9 The Korean National Health Personnel Licensing Examination Board recently discussed adding clinical skill tests to the board examination for the Doctor of Dental Surgeon degree on the basis of its success in the Medical Doctor board examination.10 However, although clinical competency tests, such as OSCE, have been used by dental schools in Korea, few reports have been published.

This study aimed to investigate the clinical skill level of final-year students using OSCE for restorative dentistry, and to determine the associations among OSCE results, achievement of clinical requirements, and attitudes towards clerkships.

Materials and Methods

The Curriculum Commission of Kyung Hee University School of Dentistry designed OSCEs for their fourth-year students. We focused on the levels of students' clinical skills at graduation, so the tests comprised assessments of core competencies, based on the students graduating from Kyung Hee University School of Dentistry. The OSCEs was conducted over 5 days, from October 29 to November 2, 2012. All clinical departments participated in the OSCEs, with the sole exception of Orthodontics. Each department established two assessment stations. The OSCEs consisted of 14 stations in total, each of which was managed by the respective departments.

The Department of Conservative Dentistry aimed to assess students' clinical competencies regarding cavity preparation for a class II gold inlay and a class IV composite restoration. The checklist for the former consisted of 14 items that for the latter comprised 11 items, based on the laboratory guidebook for operative dentistry.11 The score scales comprised three steps: unsuitable, intermediate suitable, and suitable. Students were allotted 20 minutes for the first station and 40 minutes for the second. Four professors and four residents rated clinical performance and each rater assessed approximately 10 students.

Student clinical clerkship achievement scoring was based on the number of observations of procedures performed by professors or residents, and clinical procedures performed by the students themselves at the student dental clinic, which were related to the OSCE cases. Attitudes towards clerkships were graded on the basis of a portfolio, case presentation, and attendance of conservative dentistry classes.

Statistical analyses were conducted using the SPSS software (Korean version 21 for Windows, SPSS Korea Data solution, SPSS Inc, Chicago, IL, USA). Descriptive statistical measures of the mean scores and standard deviations of each OSCE checklist were used. Pearson's correlation analyses of OSCE scores, hands-on clinical procedures, observations, and attitude towards clerkship scores were conducted. Multiple regression analyses using the OSCE scores as the dependent variable and the other factors as independent variables were also performed. In the correlation and multiple regression analyses, p values < 0.05 were considered to indicate statistical significance.

Results

In total, 79 fourth-year students participated. Two restorative dentistry OSCE stations were completed. The Cronbach's alpha coefficient of all assessed OSCE items was 0.516.

The first station assessed students' clinical skills regarding cavity preparation for a class II gold inlay. The OSCE scores of cavity width (2.038 ± 1.145) and flatness of floor (1.785 ± 1.077) were lower than the those of other items in the assessment of the occlusal step. The secondary flare (2.519 ± 1.000) and gingival margin bevel (0.722 ± 0.511) scores were lower than those of the other items in the proximal box category. Likewise, the finishing margin score was lower than those of the other assessment items (Table 1).

Table 1

Objective structured clinical examination (OSCE) scores for cavity preparation for a class II gold inlay

The second station was intended to assess clinical skill in terms of a class IV composite restoration. The OSCE score of lingual bevel width (2.835 ± 1.220) was the lowest of all cavity preparation items assessed. The lingual contour score (3.329 ± 1.058) was also lower than those of the other items in the restoration assessment category (Table 2).

Table 2

Objective structured clinical examination (OSCE) scores for class IV composite restoration

To evaluate students' clerkship achievements, the numbers of students' clinical procedures and cases observed (which were related to the OSCE), and clerkship attitude scores (which included case presentation, portfolio and attendance), were used. The numbers of clinical procedures (17.557 ± 2.669) and cases observed (20.063 ± 3.333) scores were associated more strongly with class IV composite restoration than cavity preparation for a class II gold inlay (Table 3).

Table 3

Scores of clerkship achievement related Objective structured clinical examination (OSCE)

Correlation analyses of each OSCE score and the student clerkship achievement scores of the related procedure were conducted. The correlation coefficient between the OSCE scores for cavity preparation for a class II gold inlay and clerkship attitude score was 0.241 (p < 0.05). Regarding class IV composite restorations, significant correlations with both the observation (r = 0.344, p < 0.01) and attitude (r = 0.303, p < 0.01) scores were found (Table 4).

Table 4

Correlation between Objective structured clinical examination (OSCE) and related clerkship achievement scores

These OSCEs were administered at the end of the fourth year. The students' experiences and efforts to reach the requirements for clerkship may have influenced the results of the OSCEs. In the multiple regression analysis, student clinical procedures (p = 0.804) and observation (p = 0.537) were eliminated from the stepwise norm, but attitude towards clerkship (p = 0.033) was associated with the OSCE result for cavity preparation for a class II gold inlay (Table 5). In contrast, regarding class IV composite restoration, student observation (p = 0.002) was associated with the OSCE result, while student clinical procedures (p = 0.272) and attitude towards clerkship (p = 0.061) were not (Table 6).

Table 5

Clinical clerkship requirement with the greatest effect on the Objective structured clinical examination (OSCE) result for cavity preparation for a class II gold inlay

Table 6

Clinical clerkship requirement with the greatest effect on the Objective structured clinical examination (OSCE) result for class IV composite restoration

Discussion

The purpose of this study was to investigate the relationship between graduating students' OSCE performance and their achievement of the three clinical clerkship requirements, that is, performance of clinical procedures, observation, and attitudes, in the field of restorative dentistry. The OSCE is a summative assessment of students' graduation outcomes. The OSCE results are thought to predict the extent to which students had learned the related course material and their performance as general dentists. According to Gerrow et al., there was a positive correlation (r = 0.46, p < 0.001) between the OSCE portion of the Canadian Dental Certification Examination and students' final-year grade point average (GPA).12 Brown et al. found that OSCE assessment of the clinical competence of fourth-year dental students was a better predictor of performance in written and case-based interdisciplinary final examinations than either concurrent fourth-year examinations or their university grades.13

Scoring of students for a clinical clerkship is generally conducted by means of observation, performance of clinical procedures, and the students' attitudes, such as case presentation, portfolio, and attendance. Through these processes, students are expected to qualify as general dentists. As mentioned above, OSCE is suitable for assessment of graduating dental students' degree of readiness and completion of their educational course. Contrary to expectations, our results indicated no correlation between OSCE scores and the number of clinical procedures performed by students. Fulfillment of the observation requirement also showed a weak correlation with the OSCE result for class IV composite restoration, but the correlation with cavity preparation for a class II gold inlay was stronger. These results suggest a weak association between OSCEs and students' attitudes towards clerkships. However, the multiple regression analysis showed that each attitude and observation has little effect on the various OSCE stations. It is difficult to understand and reconcile the fact that students' attitudes towards clerkships had more effect than their performance of clinical procedures in terms of achieving higher OSCE scores as a final clinical skill examination.

There may be legitimate reasons for these results. Provision of only two OSCE stations for each clinical department assessment could be a limitation. The requirements for student clinical procedures might be inadequate. However, inducing many patients to visit a student dental clinic in a university hospital is problematic in Korean society. Undergoing dental procedures performed by students can be subject to extensive delays. The student dental clinic is staffed largely by adjunct professors, who are present at only irregular intervals.

Considering these points, it could be a matter of clerkship education that includes teaching, learning, and assessing. That is, the well-known proverb "The more practice, the better the skills" may not be applicable to dental student education. To improve students' clinical skills, blindly emphasizing the number of clinical procedures performed may not be appropriate. The numbers of procedures that dental students should observe in the clinic and perform themselves in the student dental clinic may not be the most representative measures of achievement. Our results suggest that use of quantitative norms to evaluate passing or failing of a clerkship course likely hinder students reaching the level of general dentists.

If the clerkship course could be made more effective, the resulting greater clinical procedure experience would lead to improved clinical skills and OSCE scores. Then, our results might be different. In this sense, attention should be paid to improving the clinical teaching and the clinical learning environment for clerkships. Irby's definition of an excellent clinical teacher, based on numerous references, is exemplary: the excellent clinical teacher 1) serves as a positive role model of a competent and compassionate healthcare provider, 2) provides effective supervision and mentoring of learners, 3) employs a varied and dynamic approach to teaching, and 4) is supportive.14 McGrath et al. pilot-tested an instrument known as the Effective Clinical Dental Teaching (ECDT) score to measure student views of clinical instruction. According to McGrath, creating a positive learning climate, controlling the clinic, communicating goals, promoting understanding, forming lasting relationships, self-directed learning, and providing feedback are the important attributes.15

Various teachers, such as professors, fellows, residents, and adjunct professors, participate in student pre-clinical courses and clerkship education in a typical Korean dental hospital. Teaching guides, such as the ECDT, should be acknowledged by the faculty. In particular, prompt feedback is vital for improving student competency in treating patients at student clinics and related clinical skills. The responsibility for teaching students at the student clinic and preclinical courses, in which basic clinical skills should be provided, is commonly shifted to adjunct professors. This occurs because the professors and clinical teachers, such as residents, who are actually responsible are likely occupied with other matters, e.g., performing dentistry and/or research with insufficient staff numbers. Thus for effective application of teaching guidelines, adequate numbers of clinical teachers and professors are necessary.

Conclusions

The scores based on the numbers of patients students had treated did not show any correlation with OSCE scores in this study and the scores of observation were also unrelated. Rather, the scores of clerkship attitude had correlation with OSCE. More effective teaching with feedback and assessing methods need to be developed and applied for the teachers at student's clinic.

Article information Continued