Journal of Postgraduate Medicine
 Open access journal indexed with Index Medicus & ISI's SCI  
Users online: 902  
Home | Subscribe | Feedback | Login 
About Latest Articles Back-Issues Article Submission Resources Sections Etcetera Contact
 
  NAVIGATE Here 
  Search
 
  
 RESOURCE Links
 ::  Similar in PUBMED
 ::  Search Pubmed for
 ::  Search in Google Scholar for
 ::Related articles
 ::  Article in PDF (316 KB)
 ::  Citation Manager
 ::  Access Statistics
 ::  Reader Comments
 ::  Email Alert *
 ::  Add to My List *
* Registration required (free) 

  IN THIS Article
 ::  Abstract
 :: Introduction
 :: Methods
 :: Results
 :: Discussion
 :: Conclusions
 ::  References
 ::  Article Figures
 ::  Article Tables

 Article Access Statistics
    Viewed972    
    Printed33    
    Emailed0    
    PDF Downloaded20    
    Comments [Add]    

Recommend this journal


 


 
  Table of Contents     
ORIGINAL ARTICLE
Year : 2017  |  Volume : 63  |  Issue : 2  |  Page : 100-105

Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students


1 Department of Surgical Disciplines, All India Institute of Medical Sciences, New Delhi, India
2 Department of Paediatrics, CMCL-FAIMER Regional Institute, Medical Council of India Nodal Centre for Faculty Development, Christian Medical College and Hospital, Ludhiana, Punjab, India
3 Department of Pharmacology, CMCL-FAIMER Regional Institute, Medical Council of India Nodal Centre for Faculty Development, Christian Medical College and Hospital, Ludhiana, Punjab, India

Date of Submission31-Jul-2016
Date of Decision05-Dec-2016
Date of Acceptance20-Dec-2016
Date of Web Publication10-Apr-2017

Correspondence Address:
M K Joshi
Department of Surgical Disciplines, All India Institute of Medical Sciences, New Delhi
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0022-3859.201411

Rights and Permissions


 :: Abstract 

Background: Despite an increasing emphasis on workplace-based assessment (WPBA) during medical training, the existing assessment system largely relies on summative assessment while formative assessment is less valued. Various tools have been described for WPBA, mini-clinical evaluation exercise (mini-CEX) being one of them. Mini-CEX is well accepted in Western countries, however, reports of its use in India are scarce. We conducted this study to assess acceptability and feasibility of mini-CEX as a formative assessment tool for WPBA of surgical postgraduate students in an Indian setting. Methods: Faculty members and 2nd year surgical residents were sensitized toward mini-CEX and requisite numbers of exercises were conducted. The difficulties during conduction of these exercises were identified, recorded, and appropriate measures were taken to address them. At the conclusion, the opinion of residents and faculty members regarding their experience with mini-CEX was taken using a questionnaire. The results were analyzed using simple statistical tools. Results: Nine faculty members out of 11 approached participated in the study (81.8%). All 16 2nd year postgraduate surgical residents participated (100%). Sixty mini-CEX were conducted over 7 months. Each resident underwent 3–5 encounters. The mean time taken by the assessor for observation was 12.3 min (8–30 min) while the mean feedback time was 4.2 min (3–10 min). The faculty reported good overall satisfaction with mini-CEX and found it acceptable as a formative assessment tool. Three faculty members (33.3%) reported mini-CEX as more time-consuming while 2 (22.2%) found it difficult to carry the exercises often. All residents accepted mini-CEX and most of them reported good to high satisfaction with the exercises conducted. Conclusions: Mini-CEX is well accepted by residents and faculty as a formative assessment tool. It is feasible to utilize mini-CEX for WPBA of postgraduate students of surgery.


Keywords: Educational assessment, formative assessment, mini-CEX, workplace-based assessment


How to cite this article:
Joshi M K, Singh T, Badyal D K. Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. J Postgrad Med 2017;63:100-5

How to cite this URL:
Joshi M K, Singh T, Badyal D K. Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. J Postgrad Med [serial online] 2017 [cited 2017 Jul 26];63:100-5. Available from: http://www.jpgmonline.com/text.asp?2017/63/2/100/201411



 :: Introduction Top


Workplace-based assessment (WPBA) has emerged as an attractive concept in the field of medical education. It has many advantages as the assessment is conducted under direct supervision in a real environment, thus resembling the day-to-day clinical encounters more closely than traditional ways of assessment. It focuses more on actual performance rather than mere knowledge. Although it can be used both for formative and summative assessment, the strength lies in its use for formative assessment to promote learning. Since its introduction many tools have been developed and added to armamentarium of WPBA, namely, mini-clinical evaluation exercise (mini-CEX), direct observation of procedural skills, case-based discussions, clinical work samples, and multi-source feedback.

Mini-CEX is a useful tool that can be used for formative assessment and hence for WPBA. First introduced by the American Board of Internal Medicine in 1995 for assessment of postgraduate students, mini-CEX has been designed to provide feedback following direct observation of a clinical encounter.[1] The assessor scores the performance using a generic form that is applicable to most clinical scenarios which is followed by a feedback session. The duration of encounters varies from 10 to 15 min including the feedback. Thus, mini-CEX appears promising as it is conducted in a real environment, in a short time with feedback incorporated. The traditional way of assessment does not allow opportunity to provide feedback and thereby avoids the use of this important influence on learning. Mini-CEX is being used extensively for formative assessment in West [2],[3],[4],[5] but has been used sparingly in India.[6],[7],[8],[9],[10],[11]

Although appealing, WPBA has not enjoyed a uniform level of acceptability and success,[8] and there are concerns regarding the feasibility of carrying out multiple assessments amidst busy clinical schedules by multiple assessors to ensure reliability and simultaneously justifying the formative nature of the exercises being conducted.[12],[13]

We attempted to assess the feasibility of carrying out mini-CEX as a formative assessment tool in Indian setting and also tried to find out whether it is acceptable to our students and faculty in surgery.


 :: Methods Top


The study was carried out at a Government Medical College and Tertiary Care Teaching Hospital after approval from the ethical clearance committee and consent from the participating faculty and residents. Purposeful sampling including all 16 2nd year postgraduate surgical residents (total population sampling) was done. Focus group technique was employed to evaluate the shortcomings of the existing system of internal assessment (IA) and to sensitize the residents to mini-CEX. Two focus group discussions (FGDs) were conducted for residents. In the first FGD, the views of the residents regarding the existing system of IA were taken including the responses to their reaction to the marks displayed after the assessment. The advantages and disadvantages of the existing system were discussed in detail in a friendly atmosphere. They were encouraged to suggest alternatives to offset the disadvantages of the existing system of IA. Mini-CEX was then introduced as a formative assessment tool that could overcome some of the disadvantages inherent to the traditional way of IA. At the end of the first FGD, the students were provided with printed material on mini-CEX to help it understand better.

In the second FGD, residents were exposed to mini-CEX in detail and were told how the exercises would be conducted and what is expected out of them. The mini-CEX evaluation form was discussed in detail. Simultaneously, one to one discussion was carried out with the faculty members to sensitize them toward mini-CEX with emphasis on how to provide feedback. They were also provided with the printed material on mini-CEX, and the mini-CEX evaluation form was discussed in detail with them.

Informal feedback was taken from the students and faculty members at regular intervals to assess difficulties in running the project, and appropriate measures were taken to address them. After achieving the requisite number of exercises, feedback from residents and faculty members was taken using a predesigned and validated questionnaire on a five-point Likert scale [Figure 1] and [Figure 2]. The questionnaire was used to generate responses to assess the acceptability of mini-CEX as a formative assessment tool for WPBA.
Figure 1: Feedback questionnaire form for faculty on their experience with mini-clinical evaluation exercise

Click here to view
Figure 2: Feedback questionnaire form for residents on their experience with mini-clinical evaluation exercise

Click here to view



 :: Results Top


The results of the study are as follows.

Analysis of first focus group discussion

The responses of postgraduate students on display of performance following (conventional) IA varied from “works as a stimulant to study hard” to the feeling of “unpleasantness, depression and being worthless.” Many of them admitted being “surprised with the result” as they thought that the “case presentation” went fine and they wanted to know “what went wrong” to which they had no opportunity.

The following shortcomings of the existing conventional IA were identified during the conversation.

  • The assessment is grossly inadequate as it is conducted once in 6 months as a single case presentation
  • There is chance of getting an easy case and performing well by the candidate and vice versa
  • There is chance/luck of getting a strict or lenient examiner that may affect the score
  • There is no provision of feedback to help the candidate know about his performance.


Feasibility to conduct mini-clinical evaluation exercise

A total of 60 mini-CEX were conducted by 9 faculty members for 16 2nd year postgraduate students of the department over the period of 7 months. Each resident underwent at least three and a maximum of five encounters during this period. This amounted to nearly seven exercises conducted by each faculty member over 7 months, i.e., one mini-CEX per month per faculty. This was therefore not difficult for the faculty members and does not seem to increase their workload significantly. The initial plan of distribution of students among faculty members for mini-CEX was found not working well as the students or faculties were not available on same day and time because of their professional commitments. The distribution was then rearranged to ensure that the student remained available to the assigned faculty member.

No extra resources were required to carry out the exercises except printed evaluation sheets. Despite difficulties, mainly limited to the motivation of faculty to continue taking mini-CEX and some unforeseen circumstances such as attrition of few faculty members and therefore their unavailability for the project, it was observed that it is feasible to conduct mini-CEX without stretching the resources.

Complexity of mini-clinical evaluation exercise conducted

Forty-four (73.3%) clinical encounters were considered of moderate complexity by faculty while the number of low- and high-complexity clinical encounters was 12 (20%) and 4 (6.67%), respectively.

Competencies assessed

The competencies assessed during mini-CEX included interviewing skills 15 (25%), physical examination skills 15 (25%), counseling skills 4 (6.7%), clinical judgment 2 (3.3%), and an overall impression 24 (40%). None of the residents was assessed on professionalism alone [Table 1].
Table 1: Responses of faculty on mini-clinical evaluation exercise on a five-point Likert scale

Click here to view


Time taken for observation and feedback

The mean time taken by the assessor for observation was 12.3 min (8–30 min) while the mean feedback time was 4.2 min (3–10 min).

Acceptability of mini-clinical evaluation exercise by faculty

All faculty members found mini-CEX acceptable as a formative assessment tool. Most of them expressed that it can be used to supplement the traditional way of assessment. The feedback from the faculty regarding their experience with mini-CEX is presented in [Table 1].

As is reflected in the table, some of the faculty members reported that mini-CEX was more time-consuming than initially thought. Although all of them agreed that the exercises were easy to carry out and sample wider areas as compared to the traditional IA, some found it difficult to conduct the exercises often. The following are some of the favorable comments in the faculty feedback questionnaire regarding their experience with Mini-CEX:

  • It is better way to assess students than traditional way
  • It is interesting to interact with different students frequently.


However, one faculty member found mini-CEX similar to the case presentation by residents during routine ward and emergency rounds.

Acceptability of mini-clinical evaluation exercise by residents

All residents well-accepted mini-CEX for IA and most of them reported good to high satisfaction with the exercises conducted. A few of them liked the relaxed environment during the exercises, but a few of them found that being supervised by faculty while they were interviewing/examining the patient made them nervous. All were convinced that mini-CEX could be incorporated for IA. Some of them came up with innovative ideas to further enhance the quality of IA using mini-CEX. The feedback from the students is summarized in [Table 2].
Table 2: Responses of residents on mini-clinical evaluation exercise on a five-point Likert scale

Click here to view


In the words of residents:

  • Mini-CEX is a great opportunity to get assessed by different examiners and know the shortcomings from time to time so that we can improve gradually
  • This type of assessment is better than our exam where we are assessed for our learning of 3 years in just 1 day on few cases
  • It was more relaxed than routine exams, which was inherently different for our set up
  • The exercises appeared very casual
  • The exercises should have been more frequent and at the end of 6 months should have been graded by a point-based system as is done in IITs.


Performance of residents

There was consistent improvement in the performance of the residents with the conduction of the exercises, however, we did not observe any significant differences in their performances when different assessors assessed them.


 :: Discussion Top


We observed that conducting mini-CEX is feasible in the existing setting without stretching the resources, this has also been observed by other authors.[6],[9],[10] We did not had any difficulty in ensuring the workforce, clinical material, and stationary required to carry out the exercises. The biggest hurdle was the motivation of the faculty members for participating in the study. Two of the senior faculty members expressed their inability to participate citing their unavailability due to other professional commitments. This was despite the assurance that mini-CEX was not time-consuming and can seamlessly be incorporated in their existing schedule. Rest of the faculty accepted to participate although there was some difficulty initially to keep their motivational levels up. Once the faculty became comfortable with mini-CEX, the task became progressively easier.

The exercises conducted during this study were not scheduled with respect to time, considering the first exposure of mini-CEX and also to provide faculty with the liberty to complete the exercises at their own comfort. This was found to be a limiting factor in conducting the desired number of exercises in time. We learned that it is better to schedule the exercises at the outset for their smooth conduction. Mini-CEX was well acceptable to the faculty and students alike despite few apprehensions. The primary concern of some faculty members that mini-CEX requires more time was apparent as in few exercises the time taken was nearly 30 min. This reflects inexperience of the faculty members with mini-CEX as they tried to assess multiple skills at the same time and considered it comparable to a directly supervised traditional clinical case presentation. This can be overcome by providing the faculty with a small training on mini-CEX. Although we deliberated them about mini-CEX at the outset, more training in conducting the exercises would have offset this apprehension. In a review on perceptions on WPBA, improper training of assessors has been cited as one of the major shortcomings in acceptance of WPBA.[14] Other authors have also expressed the importance of proper training of assessors.[6],[15],[16],[17] Another concern of faculty regarding difficulty in conducting the exercises often also reflects the low motivation on their part. The time taken for conduction of each exercise was small, so we believe that this apprehension is more a mental block than a real problem. Massie and Ali have also cited this as one of the shortcomings that affect the acceptance of WPBA adversely.[14] In a system, once some form of WPBA is firmly incorporated, and faculty assignments are well laid down, there will be less inertia in conducting the exercises.

The students accepted the tool very well. This was reflected by their high level of satisfaction with the exercises performed. Some of our residents reported anxiety as the teacher was watching them during their performance. This is not surprising as in the present system of assessment; there is usually dissociation between the examiner and the student while the student is actually “performing” on the patient. This has also been observed by other authors [6],[18] except Goel and Singh.[10] With more exposure on 'being observed' and the recognition that it enhances learning, we have sincere hope that this will not disturb the student as also conveyed by Malhotra et al.[7]

Mini-CEX provides a unique opportunity to discuss the weaknesses of the candidate identified at the conclusion of the exercise. This empowers them for consistent improvement in the weak areas so identified. We observed consistent improvement in the performance of our residents over the period when mini-CEX was conducted.

This is the first study on the use of mini-CEX for formative assessment of postgraduate students of surgery form India. The present study is also unique as we could overcome some of the shortcomings reported by other Indian authors on mini-CEX, like the varying status of the assessors,[6] unable to complete the requisite number of exercises,[6],[11] exercises conducted in limited settings,[6],[11] less number of exercises,[9] each candidate examined by only one examiner,[9] and evaluation of a single competency.[11] The findings from this small study may not be generalized and more similar studies need to be conducted to support the evidence generated from this initial work.


 :: Conclusions Top


Mini-CEX is acceptable to the faculty members and residents as a tool for formative assessment for WPBA. It is feasible to carry out mini-CEX without stretching the existing infrastructure and humankind. Mini-CEX has a potential to supplement or replace the existing system of IA.

Acknowledgment

The authors would like to thank Faculty colleagues and participating students from UCMS, Faculty of CMCL-FAIMER, and CMCL-FAIMER fellows of 2014 batch.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 
 :: References Top

1.
Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): A preliminary investigation. Ann Intern Med 1995;123:795-9.  Back to cited text no. 1
    
2.
Weston PS, Smith CA. The use of mini-CEX in UK foundation training six years following its introduction: Lessons still to be learned and the benefit of formal teaching regarding its utility. Med Teach 2014;36:155-63.  Back to cited text no. 2
    
3.
Kogan JR, Bellini LM, Shea JA. Implementation of the mini-CEX to evaluate medical students' clinical skills. Acad Med 2002;77:1156-7.  Back to cited text no. 3
    
4.
Pelgrim EA, Kramer AW, Mokkink HG, van den Elsen L, Grol RP, van der Vleuten CP. In-training assessment using direct observation of single-patient encounters: A literature review. Adv Health Sci Educ Theory Pract 2011;16:131-42.  Back to cited text no. 4
    
5.
Oliphant R, Drummond R, Jackson A, Ross J, Blackhall V, Ridley-Fink E, et al. The use of mini-CEX in UK foundation training six years following its introduction: Lessons still to be learned and the benefit of formal teaching regarding its utility. Med Teach 2014;36:916.  Back to cited text no. 5
    
6.
Singh T, Sharma M. Mini-clinical examination (CEX) as a tool for formative assessment. Natl Med J India 2010;23:100-2.  Back to cited text no. 6
    
7.
Malhotra S, Hatala R, Courneya CA. Internal medicine residents' perceptions of the mini-clinical evaluation exercise. Med Teach 2008;30:414-9.  Back to cited text no. 7
    
8.
Behre BP. Introduction of mini clinical evaluation exercise (MINI-CEX) as an evaluation tool in psychiatry residency: A comparative study in JNMC, Sawangi, Wardha. JHSE 2014;1:9-12.  Back to cited text no. 8
    
9.
Behere R. Introduction of mini-CEX in undergraduate dental education in India. Educ Health (Abingdon) 2014;27:262-8.  Back to cited text no. 9
    
10.
Goel A, Singh T. The usefulness of mini clinical evaluation exercise as a learning tool in different pediatric clinical settings. Int J Appl Basic Med Res 2015;5 Suppl 1:S32-4.  Back to cited text no. 10
    
11.
Chandra M. Initiating formative assessment of postgraduate students in obstetrics and gynecology. Natl J Integr Res Med 2013;138:476-81.  Back to cited text no. 11
    
12.
Fokkema JP, Teunissen PW, Westerman M, van der Lee N, van der Vleuten CP, Scherpbier AJ, et al. Exploration of perceived effects of innovations in postgraduate medical education. Med Educ 2013;47:271-81.  Back to cited text no. 12
    
13.
Fokkema J, Teunissen PW. Assessing the assessment of interventions: We're not there yet. Med Educ 2013;47:954-6.  Back to cited text no. 13
    
14.
Massie J, Ali JM. Workplace-based assessment: A review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Theory Pract 2016;21:455-73.  Back to cited text no. 14
    
15.
Foley T, Walsh E, Sweeney C, James M, Maher B, O'Flynn S. Training the assessors: A mini-CEX workshop for GPs who assess undergraduate medical students. Educ Prim Care 2015;26:446-7.  Back to cited text no. 15
    
16.
Daelmans HE, Mak-van der Vossen MC, Croiset G, Kusurkar RA. What difficulties do faculty members face when conducting workplace-based assessments in undergraduate clerkships? Int J Med Educ 2016;7:19-24.  Back to cited text no. 16
    
17.
Liao KC, Pu SJ, Liu MS, Yang CW, Kuo HP. Development and implementation of a mini-clinical evaluation exercise (mini-CEX) program to assess the clinical competencies of internal medicine residents: From faculty development to curriculum evaluation. BMC Med Educ 2013;13:31.  Back to cited text no. 17
    
18.
Weller JM, Jones A, Merry AF, Jolly B, Saunders D. Investigation of trainee and specialist reactions to the mini-clinical evaluation exercise in anaesthesia: Implications for implementation. Br J Anaesth 2009;103:524-30.  Back to cited text no. 18
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2]



 

Top
Print this article  Email this article
 
Online since 12th February '04
2004 - Journal of Postgraduate Medicine
Official Publication of the Staff Society of the Seth GS Medical College and KEM Hospital, Mumbai, India
Published by Wolters Kluwer - Medknow