Journal of Postgraduate Medicine
 Open access journal indexed with Index Medicus & ISI's SCI  
Users online: 735  
Home | Subscribe | Feedback | Login 
About Latest Articles Back-Issues Articlesmenu-bullet Search Instructions Online Submission Subscribe Etcetera Contact
 
  NAVIGATE Here 
  Search
 
  
 RESOURCE Links
 ::  Similar in PUBMED
 ::  Search Pubmed for
 ::  Search in Google Scholar for
 ::Related articles
 ::  Article in PDF (1,015 KB)
 ::  Citation Manager
 ::  Access Statistics
 ::  Reader Comments
 ::  Email Alert *
 ::  Add to My List *
* Registration required (free) 

  IN THIS Article
 ::  Abstract
  ::  Introduction
Programmatic Ass...
Online Assessmen...
Online Assessmen...
Blended Programm...
Blended Programm...
  ::  Conclusion
 ::  References
 ::  Article Figures
 ::  Article Tables

 Article Access Statistics
    Viewed1359    
    Printed76    
    Emailed0    
    PDF Downloaded0    
    Comments [Add]    

Recommend this journal


 


 
  Table of Contents     
EDUCATION FORUM
Year : 2021  |  Volume : 67  |  Issue : 1  |  Page : 18-23

Blended programmatic assessment for competency based curricula


1 Department of Pharmacology, Adesh Institute of Medical Sciences and Research, Bathinda, Gujarat, India
2 Department of Physiology, Smt. NHL Municipal Medical College, Ahmedabad, Gujarat, India
3 Department of Community Medicine, Adesh Medical College and Hospital, Mohri, Kurukshetra, Haryana, India
4 Department of Pharmacology, Pramukhswami Medical College, Karamsad, Anand, Gujarat, India
5 Department of Medical Education, Sri Guru Ram Das University of Health Sciences, Amritsar, Punjab, India

Date of Submission13-Sep-2020
Date of Decision06-Dec-2020
Date of Acceptance16-Dec-2020
Date of Web Publication03-Feb-2021

Correspondence Address:
T Singh
Department of Medical Education, Sri Guru Ram Das University of Health Sciences, Amritsar, Punjab
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/jpgm.JPGM_1061_20

Rights and Permissions


 :: Abstract 


The uncertainty in all spheres of higher education due to the COVID-19 pandemic has had an unprecedented impact on teaching–learning and assessments in medical colleges across the globe. The conventional ways of assessment are now neither possible nor practical for certifying medical graduates. This has necessitated thoughtful considerations in making adjustments to the assessment system, with most institutions transitioning to online assessments that so far have remained underutilized. Programmatic assessment encourages the deliberate and longitudinal use of diverse assessment methods to maximize learning and assessment and at present can be utilized optimally as it ensures the collection of multiple low-stake assessment data which can be aggregated for high-stake pass/fail decisions by making use of every opportunity for formative feedback to improve performance. Though efforts have been made to introduce programmatic assessment in the competency-based undergraduate curriculum, transitioning to online assessment can be a potential opportunity if the basic tenets of programmatic assessment, choice of online assessment tools, strategies, good practices of online assessments and challenges are understood and explored explicitly for designing and implementing online assessments. This paper explores the possibility of introducing online assessment with face-to-face assessment and structuring a blended programmatic assessment in competency-based medical education.


Keywords: Assessment strategies, assessment tools, competency-based assessment, medical education, online assessment, programmatic assessment


How to cite this article:
Mahajan R, Saiyad S, Virk A, Joshi A, Singh T. Blended programmatic assessment for competency based curricula. J Postgrad Med 2021;67:18-23

How to cite this URL:
Mahajan R, Saiyad S, Virk A, Joshi A, Singh T. Blended programmatic assessment for competency based curricula. J Postgrad Med [serial online] 2021 [cited 2021 Mar 8];67:18-23. Available from: https://www.jpgmonline.com/text.asp?2021/67/1/18/308636





 :: Introduction Top


Effective assessment requires tasks that assess cognitive, psychomotor, and communication skills, while also assessing professional attributes. Students' learning depends largely on how they think they will be assessed, especially when embedded within “authentic” learning activities; hence, the assessment takes the shape of a strategic program for enriching teaching and learning in higher education, thus making way for the implementation of programmatic assessment.

As online education takes center stage, one question continues to linger in the minds of medical educators: “How do I know what my online students have learnt?” For which there indeed are no simple answers, just as there are not in face-to-face teaching–learning; however, with a little creativity and flexibility, an online learning environment can open up a host of new student assessment possibilities. And, just as with conventional courses, the answer lies in finding the right combination that works best for a particular course.

The rise of e-learning and web-based education to provide or improve educational experiences has led to a growth in the use of online assessment, which will further increase as the use of e-learning becomes more widespread. E-assessments when compared with conventional methods possess distinct characteristics due to the asynchronous nature of interactivity among the online participants. Therefore, teachers need to rethink the strategies of online pedagogy and develop effective assessment techniques that can support meaningful learning. Meaningful exchange and cooperation between students, peers, and teachers with a common purpose can create opportunities for ongoing and ample learner support. This can ultimately foster impactful engagement and deep learning in online education. Effective application of assessment in an online learning environment can offer a creative pedagogical strategy to facilitate such opportunities.

Are there any ways and means of introducing programmatic assessment through online modes for competency-based medical curriculum; if not fully at least partially? This review tries to explore the possibilities.


 :: Programmatic Assessment and Competency-Based Assessment—The Linkage Top


Programmatic assessment (PA) is a program of assessment, wherein information about learner's competence and progress is collected, analyzed, and acted upon longitudinally. It shifts the focus away from individual tools and assessments, which serve as data points, which are used to provide feedback and mentoring but not for summative decision making. Decisions are made on the collection of data points, proportional to the stakes.[1] Since individual assessments are not used for high-stake decisions, it allows tools with lesser reliability but high educational impact to be used. It also allows lowering the stress of examinations. Delinking of decision making from each assessment by using it for formative purposes and making high-stake decisions on the collection of such data points can be considered the hallmark of PA.[2]

PA encompasses enough opportunities for assessment for learning and not merely depends upon the assessment of learning. Originally proposed by van der Vleuten and Schuwirth (2005),[3] PA assessment gels well with the concept of competency-based medical education (CBME). CBME advocates the longitudinal development of the learner by mastering competencies and by incorporating more formative assessment strategies, feedback, and continuous monitoring.[4] Principles of PA and Competency-Based Assessment (CBA) are congruent to a large extent.[5],[6],[7],[8] Under these circumstances, it would not be an overstatement to claim that CBME can be implemented properly, only when supplemented by PA [Figure 1].
Figure 1: The mutual supplementing nature of the programmatic assessment and competency-based assessment

Click here to view



 :: Online Assessment—Context Top


Online assessment can be liberally used for the assessment of knowledge through online multiple-choice questions, assessment of skills by using online objective-structured clinical examination or virtual patients, assessment of competency and performance by using e-logbooks and e-portfolios, and assessment of affective domain through contributions and discussion in online team project works.[9],[10]

As CBME is being adopted in the undergraduate medical curriculum in many institutes, the unparalleled fall-out of the COVID 19 pandemic has dealt a heavy blow on teaching–learning and assessments in medical colleges across the globe. The conventional ways of assessment based on high-stake clinical examination(s) are neither possible nor practical in the face of such adversity, as institutions today are faced with a dilemma of certifying medical graduates in the absence of summative examinations.[11]

The current testing times due to COVID-19 have necessitated exceptional modifications to the assessment system, with most institutions adopting e-learning and e-assessments as an indispensable mode of learning and assessment that so far remained underutilized. The triage of resources in medical education at all levels has recently been suggested as a policy to tide over the COVID-19 pandemic.[12] Even in the post-pandemic era, online modes of learning and assessment are going to stay in medical education. We just need to devise ways to tap their full potential.

In such challenging circumstances, a programme of assessment incorporating frequent online assessment activities is the most feasible option fit for the purpose of assessment. The programmatic approach to assessment encourages the deliberate and longitudinal use of diverse assessment methods to maximize learning and assessment. Such an assessment approach ensures the collection of multiple low-stake assessment data which can be aggregated for higher stake pass/fail decisions by making use of every opportunity for formative feedback to improve performance.[13] But can both CBA and PA be connected through online mode, via e-assessment strategies, particularly so in the face of the COVID 19 crisis? For this to happen, the basic principles of programmatic assessment, choice of online assessment methods, and their relevance to the curriculum need to be prioritized while developing e-assessment strategies for CBME[14] that can enrich the quality and range of student assessment [Table 1].
Table 1: Principles of programmatic and/or competency-based assessment as addressed by e.assessment strategies

Click here to view



 :: Online Assessment – Practical Considerations Top


We need to reconsider and re-review the assessment modalities in medical education looking into the current pandemic.[15] Though the prospects of introducing online assessment may look bright, some practical aspects do need consideration.

First, online assessment cannot be introduced for the entire undergraduate medical course. As experiential learning through patient contact is the focus of medical training, so is the assessment. Second, online teaching must precede online assessment, which literally means one need to develop modules for imparting online teaching and assessment in areas of feasibility. Third, online assessment must be aligned with competency objectives and online teaching-learning methods. Fourth, while implementing online assessment at the institutional level, the opportunities provided by the CBME curriculum, like self-directed learning and facility of virtual classrooms and skill labs must be tapped.

The emphasis within online assessment should be placed on the employment of multimodal tools for formative and summative assessments.[9] These tools must focus upon mastering clinical reasoning, problem-solving, and decision-making skills, enabling the transition to a competency-based curriculum. Effective assessment in a digital epoch offers a comprehensible foundation to the applicability of technology for assessment and feedback. A list of online tools specific for online assessment in different domains is given in [Table 2].
Table 2: Online assessment tools with respect to domains of learning for a program of online assessment

Click here to view



 :: Blended Programmatic Assessment—The Buy-In Top


As stated earlier, introducing an entire program of online assessment in the CBME curriculum is neither feasible nor practical; nonetheless, it must be made part of the traditional PA of any institute and areas must be identified to develop online modules, thus making the introduction of online assessment feasible in that particular content/expertize area. Since online clinical assessment is a challenge, innovative tools have to be generated like using zoom video communications for virtual clinical examination. However, it is very difficult to replace the tests of clinical, communication, and practical skills with formats that do not allow the examiner to observe a student's performance directly.[16]

Considering all such arguments, the “Blended Programmatic Assessment” will definitely give leverage to the teachers to have a multimodal face-to-face assessment interspersed with online assessment activities [Figure 2].
Figure 2: A framework for blended programmatic assessment

Click here to view


As individual assessments are never used for making final decisions in PA and low-stake assessments are very much the part of PA, the immense potential of online assessment can be used to plan low-stake assessments during the entire course. The whole program needs to be supplemented with opportunities for online feedback and reflections. The key task shall be the timely creation and monitoring of a balanced face-to-face and online assessment program that focuses on resource availability, faculty buy-in, and support of regulatory and licensing bodies, fortified with the provision of continuous mentoring and quality assurance. As such, “Blended Programmatic Assessment” will be a new dimension in the implementation of CBME curriculum for medical training.


 :: Blended Programmatic Assessment—Fundamental Concerns Top


The statement, “Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment” signifies the importance of effective assessment very well.[17] The broad perspectives laid down by van der Vleuten and Schuwirth (2005)[3] to defend different forms of assessment are in congruity with the aforementioned statement including blended programmatic assessment. These perspectives address two key concerns—intellectual and logistics.

Intellectual concerns

Given the importance assigned to students' assessment, it is critical to examine the fundamental concerns of assessment within the blended context, i.e., validity, reliability, acceptability, feasibility, and educational impact, to control the associated risks and realize its desirable outcomes. The model of the utility of assessment given by van der Vleuten implies that it is possible to compensate for any lacunae in assessment tools by their strengths.[18] The same model needs to be kept in mind when using online assessment tools in medical education. When we translate the concept to the entire program of blended assessment, it will be better to look at the utility or for that matter, individual characteristics like the validity and reliability of the blended PA itself, rather than looking at the utility/validity/reliability of individual online assessments and/or traditional assessment tools. This allows us to use a variety of online and face-to-face methods to assess competencies.[19]

Though content and construct validity are important, face validity needs to be accorded equal weightage, particularly when introducing online assessment for the first time to students who may be unfamiliar with its processes and may require reassurance. The content validity of online assessment can be enhanced by introducing additional features: like animations, videos, and sound (with headphones) added to online questions and “Hotspot” questions (which require students to place a mark anywhere on an image or diagram dragging labels directly over an image) etc.[20] The online nature and technological aspects of such assessments can significantly impact the authenticity in comparison to other forms of paper-based assessment media.[21]

The reliability of an assessment refers to the dependability of results. It can be seen in terms of marker consistency, temporal stability, and context specificity.[22] The reliability attributes of blended PA are detailed in [Table 3].
Table 3: Reliability attributes of blended programmatic assessment

Click here to view


Acceptability of an online assessment method by all stakeholders, vis-a-vis teachers, students, institutions, and society is a key attribute of blended PA that influences the faculty motivation to use the method and enhances the student's trust of the results. Similarly, it is pertinent to recognize the various feasibility issues like affordability that govern the use of an assessment method. The acceptability of blended PA will definitely vary at the level of different stakeholders—both interclass and intraclass variations will be seen. Similarly, feasibility will be an issue across institutes, depending upon infrastructure.

For blended PA, the educational impact must be vouched for. During online learning and assessment, the ease of reverting to previous steps while using and being assessed on virtual platforms, thus providing an opportunity for corrections and skill enhancement must be weighed against the possibility of development of careless behavior and actual translation while dealing with real-life situations. The best way to ensure the utility of blended programmatic assessment is to have multiple online tests on multiple content areas by multiple examiners using multiple online tools [as given in [Table 2]] in multiple settings.

Logistic concerns

Just as the intellectual concerns warrant due consideration there are many legal, technical, and economic issues that require rigorous and comprehensive planning to cater to the design and implementation of quality assessment in e-learning.[23] thus having an impact on the blended programmatic assessment [Table 4].
Table 4: Legal, technical, and economic concerns having impact on blended programmatic assessment

Click here to view


Tips for implementation

Though COVID-19 pandemic has provided opportunities to the medical teachers to innovate and use hitherto underutilized online assessment methods and entwines them with existing assessment methods, and the opportunities thus provided can be utilized to develop a framework of “Blended Programmatic Assessment” particularly when in many countries competency-based medical education and programmatic assessment are newer concepts and are still in their introductory phase and are amenable to experimentation; the paradigm shift is never easy.

For implementing such a program, immense mobilization of resources will be required, besides investing time and efforts for faculty development and capacity building. Faculty development for implementing blended programmatic assessment will have a dual focus—”training in teaching technology” and “training in information technology.” We feel that initially, this will certainly add to the cost of the assessment but a sustained program will eventually lead to better allocation and utilization of resources, thus nullifying the enhanced cost factor. The tips as outlined in [Table 5] will help in introducing the “Blended Programmatic Assessment.”
Table 5: Tips for implementing Blended Programmatic Assessment in Competency.Based Undergraduate Curriculum

Click here to view


Tools focusing more on designing realistic digital simulations should also be developed. Still further, with the advent of technology, augmented and virtual reality may be incorporated with online assessment for highly engaging experiences. Efforts must be made to make online assessment suitable for effective as well as skills domain, and using it along with traditional means so as to compute a blended program of assessment in the competency-based medical curriculum.


 :: Conclusion Top


COVID pandemic has forced the use of online modes for teaching and assessment. Lockdown and closure of institutes even in the post-lockdown phase have made the use of online tools for teaching, learning, and assessment activities inevitable. Teaching–learning and assessment through this mode is likely to continue, though with lesser encounters, even during the postpandemic phase. A framework of “Blended Programmatic Assessment” will provide ways and means to the medical teachers and stakeholders for the seamless incorporation of online assessment with face-to-face traditional assessment in competency-based curricula.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
 :: References Top

1.
Wilkinson TJ, Tweed MJ. Deconstructing programmatic assessment. Adv Med Educ Pract 2018;9:191-7.  Back to cited text no. 1
    
2.
Singh T, Gupta P, Singh D. Student assessment. In Principles of Medical Education. 5th ed. N Delhi: Jaypee Brothers; 2021. p 69-80.  Back to cited text no. 2
    
3.
van der Vleuten CPM, Schuwirth LWT. Assessing professional competence: From methods to programs. Med Educ 2005;39:309-17.  Back to cited text no. 3
    
4.
Frank JR, Snell L, Englander R, Holmboe ES. Implementing competency-based medical education: Moving forward. Med Teach 2017;39:568-73.  Back to cited text no. 4
    
5.
Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach 2010;32:676-82.  Back to cited text no. 5
    
6.
Schuwirth LWT, van der Vleuten CPM. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach 2011;33:478-85.  Back to cited text no. 6
    
7.
van der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. Twelve tips for programmatic assessment. Med Teach 2015;37:641-6.  Back to cited text no. 7
    
8.
Torre DM, Schuwirth LWT, van der Vleuten CPM. Theoretical considerations on programmatic assessment. Med Teach 2010;42:213-20.  Back to cited text no. 8
    
9.
Ellaway R, Masters K. AMEE guide 32: E-learning in medical education Part 1: Learning, teaching and assessment. Med Teach 2008;30:455-73.  Back to cited text no. 9
    
10.
Gordon JA, Campbell CM. The role of ePortfolios in supporting continuing professional development in practice. Med Teach 2013;35:287-94.  Back to cited text no. 10
    
11.
Sahi P, Mishra D, Singh T. Medical education amid the COVID-19 pandemic. Indian Pediatr 2020;57:652-7.  Back to cited text no. 11
    
12.
Tolsgaard MG, Cleland J, Wilkinson T, Ellaway RH. How we make choices and sacrifices in medical education during the COVID-19 pandemic. Med Teach 2020;42:741-3.  Back to cited text no. 12
    
13.
Humphrey-Murto S, Wood TJ, Ross S, Tavares W, Kvern B, Sidhu R, et al. Assessment pearls for competency-based medical education. J Grad Med Educ 2017;9:688-91.  Back to cited text no. 13
    
14.
Bala L, van der Vleuten C, Freeman A, Torre D, Heeneman S, Sam AH. COVID-19 and programmatic assessment. Clin Teach 2020;17:420-2.  Back to cited text no. 14
    
15.
Sabzwari S. Rethinking assessment in medical education in the time of COVID-19. MedEdPublish 2020;9:80.   Back to cited text no. 15
    
16.
Fuller R, Joynes V, Cooper J, Boursicot K, Roberts T. Could COVID-19 be our 'There is no alternative' (TINA) opportunity to enhance assessment? Med Teach 2020;42:781-6.  Back to cited text no. 16
    
17.
Boud D. Assessment and learning: Contradictory or complimentary. In: Knight P, editor. Assessment for Learning in Higher Education. London: Kogan Page; 1995. p. 35-48.  Back to cited text no. 17
    
18.
van der Vleuten CPM. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract 1996;1:41-67.  Back to cited text no. 18
    
19.
Singh T. Student assessment: Moving over to programmatic assessment. Int J Appl Basic Med Res 2016;6:149-50.  Back to cited text no. 19
    
20.
Dennick R, Wilkinson S, Purcell N. Online eAssessment: AMEE Guide no. 39. Med Teach 2009;31:192-206.  Back to cited text no. 20
    
21.
Sim G, Holifield P, Brown M. Implementation of computer assisted assessment: Lessons from the literature. ALT-J: Res Learn Technol 2004;12:215-29.  Back to cited text no. 21
    
22.
Singh T. Basics of assessment. In: Singh T, Anshu, editors. Principles of Assessment in Medical Education. 1st ed. New Delhi: Jaypee Publishers; 2012. p. 1-13.  Back to cited text no. 22
    
23.
Raaheim A, Mathiassen K, Moen V, Lona I, Gynnild V, Bunæs BR, et al. Digital assessment – how does it challenge local practices and national law? A Norwegian case study. Eur J Higher Educ 2019;9:219-31.  Back to cited text no. 23
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4], [Table 5]



 

Top
Print this article  Email this article
 
Online since 12th February '04
2004 - Journal of Postgraduate Medicine
Official Publication of the Staff Society of the Seth GS Medical College and KEM Hospital, Mumbai, India
Published by Wolters Kluwer - Medknow