Review Article: 2021 Vol: 24 Issue: 1S
Summative Assesment Based on Objective Structured Practical Examination (OSPE) in Forensic Medicine and Toxicology in a Medical College in South India- An Experience and Students Perspective.
Shankar M Bakkannavar, Manipal Academy of Higher Education
Vinod C Nayak*, Manipal Academy of Higher Education
Keywords
Objective Structured Practical Examination, Summative Assessment, Practical Examination
Abstract
Summative assessment is an assessment administered to the students at the end of a teaching period. This assessment is intended to assess students’ learning ability and knowledge gained during the period based on the specific comparison of their performance with the standard learning outcome specified in the course. Various assessment formats are used in the summative assessments including class tests, end of semester examination, online assessments etc. In these formats the pattern would be either multiple choice questions or descriptive type questions are asked. To assess the practical knowledge gained during the teaching period, the conventional practical examination system is followed in most of the medical colleges. However, many of the medical colleges are now using Objective Structured Practical Examination (OSPE) pattern which is becoming more common method of practical assessment among the medical students.
In the present study, pattern of OSPE employed in the center was described and the students’ perception about the OSPE examination was evaluated.
Introduction
It is a common practice among all the medical colleges to assess the students for their knowledge and learning ability at the end of every teaching period. This evaluation either could be theoretically assessed or practically. The theoretical assessment is usually a simple, uniform and easy way of assessment with less time consumption. But the conventional practical examination is not only time consuming, but also it is not a uniform assessment pattern as the different questions may be asked by the examiner. Hence a uniform and reliable practical evaluation of medical students is always desirable. In order to ensure objectivity, and make it more comprehensive and unbiased, the Objective Structured Practical Examination (OSPE) was introduced in medical education. The OSPE is a method of objectively testing the practical knowledge and skills acquired during the preclinical years. This method was standardized by the All India Institute of Medical Sciences (Nayar, 1983). It is a modification of Objective Structured Clinical Examination (OSCE) used for clinical evaluation (Harden, 1979). At present OSPE is conducted in a few medical colleges in association with other conventional methods, and is being allotted a small percentage of marks.
Objectives
The objectives of the present study are to assess the student performance based on OSPE in the Forensic Medicine examination and their view towards it. This study is meant to give us an idea regarding the effectiveness and students’ opinion regarding the OSPE based evaluation method.
Materials and Methods
The study was conducted during the practical sessional examination in the department of Forensic Medicine and Toxicology at a Medical College, situated in South India affiliated to private deemed to be university. All 240 phase II medical students appearing for the practical examination were involved in the study. Prior to the conduction of OSPE examination, every student was briefed about the study and informed consent to participate in the study was obtained. It was also informed them that their performance and evaluation remain same irrespective of their participation in the study. During the summative assessment practical examination, the students were assessed with the 8 structured practical exercises kept at 8 different stations. All 8 stations with the respective questions assigned is shown in Table 1. In each station, specimen pertaining to the subject was kept and students were asked to identify the specimen citing the reasons and answer the questions pertaining to it mentioned at the station. Seven minutes were given to students to complete each exercise. After completion of a station, the student should move to next station to attempt the exercise. Two dummy stations were kept at the end of 4th and 8th stations to accommodate 10 students in a cycle and to allow them to have a relook and correct their answers of previous station exercises. The faculty members were allotted to each station to observe the students. At the end of the practical examination, the students’ feedback was obtained using the questionnaire prepared containing 14 questions. Students were required to indicate their agreement by ticking appropriate response. Completed response sheets were received from all the students appeared for the examination. The students' performance of OSPE was evaluated and analysed. The marks obtained by the students were made known to them by displaying on the notice board. Mean scores of all the stations has been obtained. The questions and tasks in each station were checked for coefficient of reliability by calculating Cronbach’s alpha. The data was tabulated and analysed for descriptive statistics using Statistical Package for Social Sciences (SPSS) version 21.
| Table 1 Aall (8) Stations Corresponding Practical Exercise and Mean Score of Students in Each Station |
|||
|---|---|---|---|
| Station No | Practical Exercise | Maximum Score allotted for the exercise | Mean Score obtained by student |
| 1 | Skeletal Examination | 3 | 2.36 (78.67%) |
| 2 | Age estimation based on Radiological examination | 4 | 3.18 (79.5%) |
| 3 | Post-mortem summery | 4 | 3.27 (81.75%) |
| 4 | Post-mortem certificate | 4 | 3.08 (77%) |
| 5 | Weapons & Soft specimens | 4 | 3.05 (76.25%) |
| 6 | Photographs & Dentition | 4 | 2.79 (69.75%) |
| 7 | Weapons and Post-mortem instruments | 4 | 3.08 (77%) |
| 8 | Toxicological specimens & Appliances | 3 | 2.57 (85.67%) |
| 30 | 23.38 (77.93%) | ||
Results
Total 240 students were included in the study. Out of 240 students, maximum number of students (65%) were happy with the examination pattern and scored more than 75% marks. Only 6% of students were unsuccessful in the OSPE practical examination as shown in Figure 1.
Figure 1: Performance of Students in OSPE
Station no.1 and Station no. 8 were carrying 3 marks each and rest of the stations were carrying 4 marks each. Average score of the students in all the 8 stations is shown in Table 2. The maximum marks (85.67%) were scored in station no. 8 exercise, comprises of Toxicological specimens and appliances followed by station no. 3 (Interpretation of post-mortem summery) in which they scored (81.75%). Least marks were scored in the exercise containing photographs and dentition (station no. 6) which amounts to 69.75%.
| Table 2 Feedback Analysis of Responses to Questionnaire Regarding OSPE |
||||||
|---|---|---|---|---|---|---|
| Sl No | Criteria | Choice 1 | Choice 2 | Choice 3 | Choice 4 | No response |
| 1 | OSPE System compared to other Practical exams | Very good (79) 33% | Good (85) 35% | No difference(7) 3% | Bad (2) 1% | (67) 28% |
| 2 | Difference between OSPE and other forms of practical examinations | Obvious difference (110) 46% | Not much difference (48) 20% | No difference (1) 0.5% | Can't comment (5) 2% | (76)31.5% |
| 3 | Structure of Examination | Very good (68) 28% | Good (96) 40% | Average (6)2.5% | Can't comment (1)0.5% | (69)29% |
| 4 | Environment of OSPE | Very much Comfortable (72) 30% | Comfortable (80) 33% | Somewhat Comfortable (14)6% | Uncomfortable (3)1% | (71)30% |
| 5 | Is it confusing to switch over from one station to other in short time? | Yes (42) 17.5% | No (100) 42% | Somewhat (30)12.5% | Can't comment (0) | (68)28% |
| 6 | Relevance to syllabus | Yes (215) 90% | No (10) 4% | Somewhat (15)6% | Can't comment 0 - (0) | 0 |
| 7 | Questions in Response Station | Relevant (216)91% | Irrelevant (16)6.5% | Can't comment (8)2.5% | - | 0 |
| 8 | Does it reduce element of luck? | Yes (117)49% | No (74)31% | Somewhat (25) 10% | Can't comment (21)9% | (3)1% |
| 9 | Time in each Station | Excess (27)11% | Adequate (182)76% | Inadequate (25)10.5% | Can't comment (4)1.5% | (2)1% |
| 10 | Effect of OSPE | Helps to improve (125) 52% | Somewhat helps (85)35.5% | Does not help (13)5.5% | Can't comment (15) 6% | (2)1% |
| 11 | Application as Assessment System for | Only Evaluation (77)32% | Only Learning (26)11% | Learning & Evaluation (119)49% | Can't comment (14) 6% | (4)2% |
| 12 | Application to exams | Sessional & University exams (198)82.5% | Only University exams (12)5% | Only sessional exams (19)8% | Don't want (10) 4% | (1) 0.5% |
| 13 | Introduction in Final exams | Completely (150)62.5% | Partially (64)27% | Not at all (8)3% | Can't comment (17)7% | (1)0.5% |
| 14 | Your preference for OSPE over Conventional practical examinations | Like it very much (101)42% | Like it (107)44.5% | No difference (26)11% | Don't like (6)2.5% | 0 |
The questionnaire introduced to the students at the end of the examination and their responses recorded are tabulated in the Table 2.
Most of the students (68%) agreed that the OSPE System was very good compared to other practical examination systems. But only 46% students accepted that there is the difference between OSPE and other forms of practical examinations. The students were happy with the structure of Examination which constituted 8 stations.
Majority of the students (60%) in our study agreed that the sufficient or excess time was given during the OSPE. But 31% of students were not agreeing with it.
Most of the students (91%) agreed that the questions in each stations were relevant and are in consistent with the syllabus. 49% of students accepted that the OSPE system reduces the element of luck. When effect of OSPE is considered, majority of students (87.5%) opined that it helps to improve the knowledge. According to 49% of students, OSPE can be applied for both learning and evaluation purposes whereas 32% of students opted it only of evaluation purposes. As far as applying the method across the examinations is concerned, 198 (82.5%) of students opined that it should be applied to both sessional & university examinations. Overall 86.5% of students liked the OSPE system over traditional method of practical examination.
Discussion and Conclusion
OSPE has been evolved over the years as one of the methods to assess the students’ performance. Its use has eliminated the subjectivity of examiners towards the students through identical questions and predefined marking system (Roy, 2004).
Among 240 students, 157 students performed exceptionally well in the OPSE who scored >75% of marks which suggests the effectiveness of the system. It is confirmed that the reliability of the assessment method with good consistency (Cronbach’s alpha score of 0.8). Similar observations were found in study done by Mard & Ghafouri (Mard, 2020). The feedback regarding using OSPE given by students was constructive and highly accepted by them. Similar feedback was received by (Feroze et al., 2002; Krishna et al., 2011; Menezes et al., 2011; Nayar et al., 1986; Singh & Singh, 2020; Bairy et al., 2014)
68% of students felt that the OSPE System was good compared to other conventional practical examinations. 60% of students’ agreement in our study regarding the sufficient or excess time given during the OSPE is in agreement with the study done by Mard & Ghafouri (Mard, 2020).
Our finding regarding the agreement by the students to the relevant and consistent questions with the syllabus, was similar to the other studies.
49% students agreed that the system reduces the element of luck. This was not in consistence with the earlier studies (Mard, 2020; Feroze, 2002; Abraham, 2009; Kundu, 2013; Abraham, 2005).
Among the new parameters included in our study, overall 49% of students wanted OSPE to be a method for both learning and evaluation processes and also highest no. of students (82.5%) opined that the OSPE can be used for both formative as well as summative assessment.
Acknowledgement
Conflict of Interest: None to declare
Funding: No source of funding
Ethical Approval: Obtained from Institutional Ethics Committee
Conclusion
The OSPE has many advantages over conventional methods of formative and summative assessments. The objectivity, eliminated examiner bias, fixed time period, identical questions and predefined marking system have made it more acceptable pattern of assessment. During OSPE assessment, all students’ exposure to similar types of questions with the same difficulty is an important aspect and makes the system an unbiased fair system of examination.
References
- Nayar, U. (1983). Objective structured practical examination, in: R.L. Bijlani and U. Nayar (Eds) Teaching Physiology, Trends and Tools. All India Institute of Medical Science, New Delhi, 151-159.
- Harden, R.M., & Gleeson, F.A. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13, 41-54.
- Roy, V., Tekur, U., & Prabhu, S. (2004). A comparative study of two evaluation techniques in pharmacology practicals: Conventional practical examination versus objective structured practical examination. Indian J Pharmacol, 36, 386-8.
- Mard, S.A., & Ghafouri, S. (2020). Objective structured practical examination in experimental physiology increased satisfaction of medical students. Advances in Medical Education and Practice, 11, 651–659.
- Feroze, M., & Jacob, A.J. (2002). OSPE in Pathology. Indian J of Pathology & Microbiology, 45, 53–57.
- Krishna, M.N., Ashakiran, S., Mendez, D., Kundar, M., Chatterjee, S., Ganesh, G., & Nandini, T. (2011). OSPE as a learning & evaluation tool for biochemistry: First experience. J Clin Biomed Sci, 1(2), 64–69.
- Menezes, R.G., Nayak, V.C., Binu, V.S., Kanchan, T., Rao, P.P.J., Baral, P., & Lobo, S.W. (2011). Objective Structured Practical Examination (OSPE) in forensic medicine: Students’ point of view. Journal of Forensic and Legal Medicine, 18, 347–349.
- Nayar, U., Malik, S.L., & Bijlani, R.L. (1986). Objective structured practical examination: a new concept in assessment of laboratory exercises in preclinical sciences. Med Educ, 20(3), 204-9.
- Singh, G., & Singh, R. (2020). Objective structured practical examination: A tool design to gauge the perception and performance of the undergraduate medical students for gram stain. J Med Sci Health, 6(2), 19-24.
- Bairy, K.L., Adiga, S., Shenoy, S., Magazine, B.C., Amberkar, M., & Kumari, M.K. (2014). OSPE in pharmacology: Comparison with the conventional method and students. Perspective towards OSPE. Br Biomed Bull, 2, 218-22.
- Abraham, R.R., Raghavendra, R., Surekha, K., & Asha, K. (2009). A trial of the objective structured practical examination in physiology at Melaka Manipal Medical College. India. Adv Physiol Educ, 33(1), 21–23.
- Kundu, D., Das, H.N., Sen, G., Osta, M., Mandal, T., & Gautam, D. (2013). Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata. J Nat Sci Biol Med, 4(1), 103–107.
- Abraham, R.R., Upadhya, S., Torke, S., & Ramnarayan, K. (2005). Student perspectives of assessment by TEMM model in physiology. Adv Physiol Educ, 29(2), 94–97.