OSPE as an Assessment Tool for Medical Students

Dr. Sushma BJ^1* and Mr. Sumit Parashar²

1. Professor and Head, Department of Biochemistry, National Institute of Medical Sciences and Research, Jaipur, India

2. Tutor, Department of Biochemistry, National Institute of Medical Sciences and Research, Jaipur, India

Received: 10 April 2024; Accepted: 20 May 2024; Published: 25 May 2024

Citation: Sushma BJ “OSPE as an Assessment Tool for Medical Students“. J Chem Analyt Biochem (2024): 103. DOI: 10.59462/JCAB.1.1.103

Copyright: © 2024 Sushma BJ. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Keywords

Objectively structured practical examination • Conventional practical examination • Observer station • Responsive station • Student perceptions and faculty perceptions

Introduction

The Objectively Structured Practical Examination (OSPE) is a novel tool introduced by the National Medical Commission used for assessment in competency based Undergraduate Curricula [1]. Competency based education has been defined as an outcome-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies [2]. Greater emphasis should be placed on setting up an ongoing and longitudinal assessment so that teachers can identify the stage of the learner and decide whether they need further or different learning opportunities to acquire competency. Assessment of competency-based curricula plays a crucial role in its implementation [1]. The major characteristics of competency-based assessment [CBA] is its longitudinal nature, provision of developmental feedback and use of authentic settings, all of which result can lower the stakes in individual assessments. The utility of assessment is traditionally expressed as a notional concept represented as the use of a product of validity, reliability, acceptability, feasibility and educational impact [3]. For CBA, validity and educational impact are the major determinants of its utility. Despite subjective judgments being involved, the reliability of these assessments can be improved by increasing the number of assessors, assessments, and tasks and by involving all teachers in the department during the CBA process. This approach is a simple intervention for not only taking care of subjectivity but also improving ownership of teaching-learning and assessment [4]. Traditional assessments are easy to design, administer, score and analyze compared to CBA but may not be able to provide complete information about the stage of the student. Traditional assessments are snapshot observations of learning, are generally not linked to instructions or outcomes and promote test taking behavior. They are fragmented and mainly focus on knowledge (sometimes skills). CBA, on the other hand, provides more comprehensive information about not only the current stage of the student but also about his progression and ascendency [1].

It is a well-known fact that assessment drives learning. A single examination does not fulfill all the functions of assessment, such as assessing knowledge, comprehension, skills, motivation, and feedback [6]. Written examinations (essays and multiple choices) test cognitive knowledge, which is only one aspect of competency. The structuring of questions and assessments through highlighting objectivity has been emphasized and gained importance in the practical evaluation [7]. The objective structured practical examination (OSPE) is now an accepted tool for assessing practical skills in both preclinical and paraclinical subjects. However, there are no strict or limiting guidelines on the types of scenarios that are used in the OSPE examinations. In the UK, the USA, Canada, and indeed the most reputable colleges of medicine, the OSPE is the standard mode of assessment of competency, clinical skills, and counseling sessions that satisfactorily complement cognitive knowledge testing in essay writing and objective examination [8]. The present study was designed to assess students’ performance using the OSPE method and conventional method. Student’s perceptions of OSPE were also obtained via the Google Form Questionnaire and were assessed using a 5-point Likert scale as used by the previous authors [5].

Aim and Objectives

The objectives of the present study were

1. The OSPE and conventional practical examination were used to compare student performance based on the scores obtained.

2. To explore the student’s perceptions and faculty members perceptions of objectively structured practical examination as an assessment tool or regular practical examinations.

Materials and Methods

The present study is a prospective interventional educational research study conducted with undergraduate medical students in the Department of Biochemistry, National Institute of Medical Sciences & Research, Jaipur. The present study included a total of 212 phase I MBBS students in the batch 2023-24 cohort after providing informed consent. Conduction of OSPE pattern induction was announced one month prior to the conducted during second internal assessment. The confidentiality of the stations used for OSPE was maintained until the end of the examination. Adequate instructions regarding conventional practical examination and OSPE were given prior to the internal assessment. A total of 212 students were divided into 4 groups -A, B, C and D-of 53 students for the assessment. In the conventional practical examination, all the students were provided with the abnormal urine samples at the respective seat number allotted to the student using the chit method. The student had to perform all the tests related to abnormal urine analysis and produce a urine report at the end of the analysis and Table viva was used for each student, the score recorded was out of 20 marks. In OSPE, we had kept five stations out of which two were observer type and three were responsive type (unobserved responsive type, responsive type and coupled OSPE) time allotted was three minutes at each station. At the observer station one test related to normal and abnormal urine analysis was kept for performing, and the psychomotor skills were assessed. Similarly at responsive station, the cognitive skills were assessed. Each student has to attend 4 stations, one observer station and 3 responsive stations. Both assessments were performed using a standardized checklist (Tables 1-4), and the scores were subsequently indicated. The scores were recorded out of 20 points. The mean scores obtained by the students were compared using Student’s t test and a p value <0.05 was considered to indicate statistical significance. The student’s perceptions and faculty perceptions about both conventional practical examination and OSPE were assessed using a validated questionnaire in the Google Forms link sent to the students in the Watts app group. The data were downloaded from the Google Forms and tabulated in tables and graphs.

Table 1. OSPE checklist for observer station Benedicts Test

Table 2. OSPE checklist for observer station Heat Acetic Acid

Table 3. OSPE checklist for unobserved responsive station

Table 4. OSPE checklist for responsive station and Coupled OSPE

Ethical Consideration and Consent

This study was conducted after obtaining Institutional Ethics Committee approval (Reference No NIMSUR/ IEC/2023/770 and Proposal Number IEC/P-427b/2023. Informal written consent was obtained from all the students involved in the study

Results

In the present study, a total of 212 students provided informed verbal and written consent. Student performance was assessed using conventional practical examination and objectively structured practical examination, in both the assessments the maximum score was out of twenty marks. Table 1 represents the frequency distribution of the students based on the scores obtained for both the CPE and OSPE. In CPE 5.66% of the students had scores <5 out of 20, 25.94% had scores 5 to <10 out of 20, 59.9% had scores 10-<15 out of 20 and 8% had scores between 15 to 20 in OSPE. In addition, 1.88% of the students had scores <5 out of 20, 11.79% had scores 5 to <10 out of 20, 70.75% had scores 10-<15 out of 20 and 15.56% had scores between 15 and 20.

Discussion

In the present study a total of 212 students of batch 2023-24 phase I MBBS participated. The frequency of distribution of students as per the scores obtained out of twenty marks revealed that out of 212 students in CPE 67 (31.6%) students had scores less than 50% and 145 (68.39%) students had scores greater than 50%. According to OSPE method 29 (13.67%) students had scores less than 50% and 183 (86.3%) students had scored more than 50% presented in (Table 5). A comparison of the scores obtained by students for both examination patterns revealed that OSPE score was significantly greater than the CPE score and the OSPE served as a better tool for assessment because students had better performance as presented in (Table 6). Evaluated marks of question stations and check list of procedure stations were made available to the students, who appreciated what they achieved and identified and where they needed to improve [6]. Students perceptions about OSPE was assessed, using 5 point Likert’s Scale using validated questionnaire, 75.9% agree that OSPE is good form of examination and learning process, 67.1% agreed that the checklist provides a fair and unbiased system of marking, 75.9% agreed that OSPE covers learning objectives of important topics, 77.9% agreed that OSPE is more easy and better for scoring compared to CPE, 79.1% agreed that OSPE was useful for retention of knowledge in biochemistry, data is presented in (Figure 1). 74.5% agreed that OSPE reduces luck factor in the examination, 76.6% agreed that OSPE should be the method of assessment in all practical examinations for phase I MBBS students, 77.2% agreed that OSPE encourages to pay attention to practical examination, 74.1% agreed that attitude of teachers is good in OSPE as compared to CPE and 77.9% agreed that OSPE is a better pattern than CPE, as presented in (Figure 2). The vast majority of the students enjoyed OSPE because of lack of teacher bias, non-partiality, uniformity, decreased consumption and student-centred and studentfriendly. The majority of the faculty felt that OSPE was not time consuming, eliminated observer bias, and should be implemented on regular basis during all types of assessments, better assessment of practical skills, and also majority of the faculty agree that OPSE is a good learning method for students, as represented in (Table 7).The majority of the time following the examinations, students often complain that they have performed well in the practical examinations but they do not obtain their expected scores, irrelevant questions or differences in the level of questions asked to them simple to difficult: some students were asked basic questions and some were asked difficult questions and on the other end the external examiners during university examinations complain about the extensiveness of the examination pattern during university examinations. OSPE definitely minimizes all the factors encountered during CPE. Some students mentioned that OSPE gives them better confidence and defines them with what exactly needs to be studied and prepared for practical examinations as objectives and competency skills are well-defined in the OSPE. These findings were in accordance with the findings of the studies conducted by Ananthkrishnan [9] Watson and Houston [10], Vijaya and Alan [11], KL Bairy et al. [12], and Mokkapati et al. [13], these authors noted that the OSPE was a well-structured, easy assessment format in which the students were well organized and easy and less stressful that it covered the learning domains and syllabus appropriately than did conventional examination. Faldessai et al. [14], Praveen Singh et al. [15] reported that 90% of student participants expressed OSPE as a better alternative to conventional examination and it was better structured and uniform. The students were clear that the OSPE assessed the relevant practical skills and it covered the appropriate knowledge consistent with the learning objectives.

Figure 1. Students’ perceptions of the OSPE

Figure 2. Students’ perceptions of the OSPE

Conventional Practical Examination		Objectively Structured Practical Examination
Number	Percentage	Number	Percentage
12	5.66	4	1.88
55	25.94	25	11.79
127	59.90	150	70.75
18	8.49	33	15.56
212	100	212	100

Table 5. Shows the frequency distribution of scores obtained by conventional practical examination and OSPE expressed in number and percentage (no=212) out of maximum marks 20

	Conventional Practical Examination	Objectively Structured Practical Examination	t value
Mean ± SD	10.655 ± 3.477	12.091 ± 2.55	8.4655 x 10-7
P value <0.001, Highly Significant

Table 6. Shows the compassion of mean scores obtained by conventional practical examination and OSPE expressed in number and percentage (no=212) out of maximum marks 20

	Yes (Agree)		No (Disagree)
	Number	Percentage	Number	Percentage
Not time consuming	9	100	0	0
Eliminates observer bias	8	88.88	1	11.11
OSPE should be implemented on regular basis during all type of assessments	9	100	0	0
Better assessment of practical skills	8	88.88	1	11.11
Apart from Assessment OSPE is a good learning method for students	7	77.77	2	22.22

Table 7. Faculty perceptions about OSPE (no=9)

Conclusion

The present study showed that the majority of the students performed well scored more than 50% marks in OSPE as compared to CPE and also there was significant differences in the mean scores obtained in OSPE in comparison with conventional practical examination. The OSPE is a better tool for the assessment of students during practical examinations, it should be used regularly in all types of assessments and also it can also be used for learning purposes. Student and faculty perceptions were highly positive towards OSPE as a better assessment tool and learning tool.

Declaration

Ethical approval and consent to participate: Institutional ethical committee clearance was obtained from Office of the Institutional Ethics Committee, NIMS University, Rajasthan, Jaipur for this study entitled “Objectively Structured Practical Examination (OSPE) as a Tool for Assessment during Internal Practical Examination in Biochemistry” with the Proposal Reference Number NIMSUR/IEC/2023/773(b). Verbal and written informed consent were obtained from all the students interested in participating in this research study on a voluntary basis.

Consent for Publication

This study does not contain any individual person data in any form. It is not applicable.

Competing Interests

There are no competing interests and no conflicts for publication.

Availability of Data and Materials

Data collected during the research study is available in the excel sheet is shared in the supplementary files.

Funding

Once the study is accepted for publication APC for this article will be provided by NIMS University, Jaipur, Rajasthan

Acknowledgement

We are highly thankful to NIMS University, Jaipur, Rajasthan for all the essential support throughout the study.

References

1. Patra, Somdatta, Amir Maroof Khan, Madhu Kumari Upadhyay and Sanjiv Kumar Bhasin, et al. “Module to Facilitate Self-Directed Learning Among Medical Undergraduates: Development and Implementation.” J Educ Health Promot (2020).
2. Frank, Jason R., Linda S. Snell, Olle Ten Cate, Eric S. Holmboe and Carol Carraccio, et al. “Competency- Based Medical Education: Theory to Practice.” Medical Teacher 32 (2010): 638-645.
3. Van Der Vleuten, Cees PM. “The Assessment of Professional Competence: Developments, Research and Practical Implications.” Ad Health Sci Educ 1 (1996): 41-67.
4. Sood, Rita, and Tejinder Singh. “Assessment In Medical Education: Evolving Perspectives and Contemporary Trends.” Natl Med J India 25 (2012): 357-364.
5. Shenoy, Preethi J., Priyanka Kamath, Vinaykumar Sayeli, and Sunil Pai. “Standardization and Validation of  Objective  Structured  Practical  Examination in Pharmacology: Our Experience and Lessons Learned.” Indian J Pharmacol 49 (2017): 270-274.
6. Abraham, Reem Rachel, Rao Raghavendra, Kamath Surekha, and Kamath Asha. “A Trial of The Objective Structured Practical Examination in Physiology at Melaka Manipal Medical College, India.” Adv Physiol 33 (2009): 21-23.
7. Kundu, Dipankar, H. N. Das, Gargi Sen, Manish Osta, T. Mandal, and Divyendu Gautam. “Objective Structured Practical Examination in Biochemistry: An Experiece in Medical College, Kolkata.” J Nat Sci Biol Med 4 (2013): 103.
8. Abraham, Reem Rachel, Subramanya Upadhya, Sharmila Torke, and K. Ramnarayan. “Student Perspectives of Assessment by TEMM Model in Physiology.” Adv Physiol Educ 29 (2005): 94-97.
9. Ananthakrishnan, N. “Objective Structured Clinical/ Practical Examination (OSCE/OSPE).” J Postgrad Med 39 (1993): 82-84.
10. Watson, A. R., I. B. Houston, and G. C. Close. “Evaluation Of an Objective Structured Clinical Examination.” Arch Dis Child 57 (1982): 390-392.
11. Vijaya, D. S., and S. Alan. “A Comparative Study to Evaluate Practical Skills in Physiology Among 1st Phase Medical Under Graduates at JNMC Belgaum: Traditional Practical Examinations Versus Objective Structure Practical Examinations (TPE V/S OSPE).” Int J Educat Res Technol 5 (2014): 126-134.
12. Bairy, K. L., and Mohan Amberkar. “OSPE In Pharmacology: Comparison with The Conventional Method and Students’ Perspective Towards OSPE.” Br Biomed Bull (2013).
13. Mokkapati, Anuradha, G. Pavani, S. Manick Dass, and M. Srinivas Rao. “Objective Structured Practical Examination as A Formative Assessment Tool for Iind MBBS Microbiology Students.” Int J Res Med Sci 4 (2016): 4535-4540.
14. Faldessai, Nitin, Archana Dharwadkar, and Shruti Mohanty. “Objective-Structured Practical Examination: A Tool to Gauge Perception and Performance of Students in Biochemistry.” Asian J Multidiscip Stud 2 (2014): 32-38.
15. Singh, Praveen R., Raksha Bhatt, and Suman Singh. “Perceptions Towards Implementation of OSPE As an Assessment Tool in Anatomy for Undergraduates at A Rural Medical College in Western India.” Natl J Basic Med Sci 2 (2010): 54-60.