Journal of Postgraduate Medicine
 Open access journal indexed with Index Medicus & ISI's SCI  
Users online: 1496  
Home | Subscribe | Feedback | Login 
About Latest Articles Back-Issues Article Submission Resources Sections Etcetera Contact
 
  NAVIGATE Here 
  Search
 
  
 RESOURCE Links
 ::  Similar in PUBMED
 ::  Search Pubmed for
 ::  Search in Google Scholar for
 ::Related articles
 ::  Article in PDF (501 KB)
 ::  Citation Manager
 ::  Access Statistics
 ::  Reader Comments
 ::  Email Alert *
 ::  Add to My List *
* Registration required (free) 

  IN THIS Article
 ::  Abstract
 :: Introduction
 ::  Materials and Me...
 :: Results
 :: Discussion
 :: Acknowledgements
 ::  References
 ::  Article Figures
 ::  Article Tables

 Article Access Statistics
    Viewed2579    
    Printed76    
    Emailed1    
    PDF Downloaded28    
    Comments [Add]    

Recommend this journal


 


 
  Table of Contents     
ORIGINAL ARTICLE
Year : 2015  |  Volume : 61  |  Issue : 3  |  Page : 163-168

Short-term outcomes of a program developed to inculcate research essentials in undergraduate medical students


1 Department of Pharmacology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India
2 Department of Pathology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India
3 Department of Physiology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India
4 Department of Microbiology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India
5 Department of Community Medicine, Kasturba Medical College, Manipal, Karnataka, India
6 Department of Pharmacology, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India

Date of Submission03-Aug-2013
Date of Decision08-Sep-2013
Date of Acceptance06-Mar-2015
Date of Web Publication26-Jun-2015

Correspondence Address:
V Devi
Department of Pharmacology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0022-3859.159315

Rights and Permissions


 :: Abstract 

Background: Participation in research during undergraduate studies may increase students' interest in research and inculcate research essentials in them. Aims: The purpose of this study was to evaluate the effectiveness of the mentored student project (MSP) program. Settings and Design: In the MSP program, students in groups (n = 3 to 5) undertook a research project, wrote a scholarly report, and presented the work as a poster presentation with the help of a faculty mentor. To begin with, the logic model of the program was developed to identify short-term outcomes of the program on students, mentors, and the institution. A quasi-experimental design was used to measure the outcomes. Materials and Methods: A mixed method evaluation was done using a newly-developed questionnaire to assess the impact of the MSP on students' attitude, a multiple-choice question (MCQs) test to find out the impact on students' knowledge and grading of students' project reports and posters along with a survey to check the impact on skills. Students' satisfaction regarding the program and mentors' perceptions were collected using questionnaires. Evidence for validity was collected for all the instruments used for the evaluation. Statistical Analysis: Non-parametric tests were used to analyze data. Based on the scores, project reports and posters were graded into A (>70% marks), B (60-69% marks), and C (<59% marks) categories. The number of MSPs that resulted in publications, conference presentation and departmental collaborations were taken as impact on the institution. Results: Students' response rate was 91.5%. The students' attitudes regarding research changed positively (P = 0.036) and score in the MCQ test improved (P < 0.001) after undertaking MSP. Majority of project reports and posters were of grade A category. The majority of the items related to skills gained and satisfaction had a median score of 4. The MSPs resulted in inter-departmental and inter-institutional collaborations, 14 publications and 15 conference presentations. An area for improvement noted was to have the MSP implemented in the curriculum without increasing students' overall workload and stress. Conclusion: The study identified strengths and weaknesses of the MSP program. Our model of undergraduate research project may be incorporated in undergraduate medical programs to foster positive attitude and knowledge base about scientific research and to instil research skills among students.


Keywords: Attitude, knowledge, mentored, outcome, research, skills, student


How to cite this article:
Devi V, Ramnarayan K, Abraham R R, Pallath V, Kamath A, Kodidela S. Short-term outcomes of a program developed to inculcate research essentials in undergraduate medical students. J Postgrad Med 2015;61:163-8

How to cite this URL:
Devi V, Ramnarayan K, Abraham R R, Pallath V, Kamath A, Kodidela S. Short-term outcomes of a program developed to inculcate research essentials in undergraduate medical students. J Postgrad Med [serial online] 2015 [cited 2019 Sep 15];61:163-8. Available from: http://www.jpgmonline.com/text.asp?2015/61/3/163/159315



 :: Introduction Top


The evolution of the medical field requires that medical educators constantly and systematically look at better ways to prepare future physicians. [1] Despite the absence of definite data, it is believed that student participation in research during undergraduate studies during medical school leads to better clinicians or physician scientists who will be able to create and apply new knowledge related to biomedicine. [1] Studies have shown that research experience during medical school training increased students' interest in research, [2],[3],[4] improved their confidence in conducting research, [4] influenced students' choice of clinical specialty, [5],[6],[7],[8],[9],[10] improved knowledge regarding research principles, [2],[3],[6],[11] fostered abilities to critically evaluate literature [7],[12],[13] and skills to write scientific papers. [14] The knowledge and skills gained while undertaking research work may further increase students' participation in research thereby increasing students' publications and presentations. [15],[16],[17]

Despite the above evidences on educational benefits of research projects in medical school, whether such programs should be required as a part of the medical school curriculum is still debatable. [1] Studies have shown that these activities caused stress on students. [2],[4],[18] Their improper implementation in the curriculum may have several negative effects on students' learning. [1] When educational interventions become an extra burden on students, it may adversely affect their performance in examinations by affecting their learning and distracting them from studying for examinations. Bierer and Chen (2010) in their systematic review on the impact of scholarly concentration on medical students, recommend the requirement of more rigorous evaluation designs to show the positive and negative effects of such programs. [19]

The aim of this study was to initiate program evaluation of an introduced curriculum change, mentored student project (MSP) at the institution. The primary objective was to measure the short-term outcomes of MSPs on students' knowledge, attitude, and skills regarding research. This study also describes outcomes of MSP on faculty and the institution. During the study we also developed instruments to measure the same, as systematic instrument design and use of reliable instruments are vital if the results of education testing are to be considered valid.


 :: Materials and Methods Top


Educational context and subjects

The MSP program was implemented in the curriculum in 2007 with the intention of developing research skills essential for the career development of medical students. In MSPs, medical students (n = 141) were required to undertake a research project involving in-depth study of an area of their interest, write a scholarly report, and present the work as a poster presentation in 6 month duration during the second year of the course. The MSP coordination committee with two faculty members as coordinators and Dean as the chairman functioned to monitor the smooth conduct of the MSP program. Faculty coordinators oriented students regarding the MSPs; research methodology; preparation of project report; and the poster preparation and presentation. The coordinators supervised student projects done under the guidance of faculty mentors and also ensured that the time line was followed.

Keeping in mind the cultural background and specific needs of the international students, our medical school took the initiative in developing a mentorship scheme, in which the mentors (faculty members who are basic scientists) were required to supervise the academic progress of those students who were assigned to them. Mentors must review and approve the project proposal and final report. Mentors guide the students during all phases of the project from research topic selection, ethical consideration, writing the proposal, data collection, analysis and interpretation, writing up of final report and designing poster for poster presentation. Feedback taken from faculty and students of the institution in 2010 indicated the need for evaluation of the MSP program. To begin with the logic model [20] of the MSP program was developed to identify short-term outcomes of the program [Figure 1].
Figure 1: Logic model of MSP program. ISO: International organization for standardization

Click here to view


The ethics committee of the institution approved present quasi-experimental study. The convenience sampling method was used. Written informed consent was obtained from the students and faculty before responding in instruments.

Assessing outcomes

All the instruments required for assessing the outcomes of the MSP were developed using standard protocols and involving four experts with the Foundation for Advancement of Medical Education and Research (FAIMER) fellowship degree. To develop questionnaires, experts checked content validity individually. The questionnaire was further refined using a Delphi process in a series of meetings with a panel having above experts. All closed-ended questions in questionnaires were in a 5-point Likert scale (1 = disagree and 5 = strongly agree).

Students' knowledge

Fifteen multiple-choice questions (MCQs) were developed by coordinators based on standard literature on research methodology [21],[22] and standard processes to establish validity and reliability. To test students' knowledge, the finalized 15 MCQ administered to students before (n = 153) and soon after (n = 140) completing MSP with poster presentation being the final requirement for the completion.

Students' attitude

To test the impact of the MSPs on students' attitude, a questionnaire was developed after a literature review. [19] To assess test retest reliability, students were asked to respond in the questionnaire twice with a week interval in between. Following this, items with low a Cronbach's α (CA) value were deleted. The final questionnaire with 13 items was used to assess students' (n = 140) attitude by doing a retrospective pretest. Students' responded in the questionnaire in a 5-point Likert scale.

Students' skills

Quality of students' project reports and posters along with students' perception on skills gained gives evidence for the impact of MSP on research skills of students.

Students' project report and poster evaluation instruments (with 35 and 20 criteria respectively) were developed based on Glassick's six standards for assessing scholarships [23] and review criteria for scholarly manuscript. [24] The questionnaire developed to assess students' perceptions regarding skills gained had 15 items with a Likert scale. After the completion of the MSP, two evaluators independently evaluated students' posters (n = 41) and project reports (n = 39). The percentage marks scored was used for grading the reports and posters as A (>70% marks), B (60-69% marks), and C (<59% marks) categories.

Impact on faculty

A questionnaire with 13 Likert-type statements and 4 open ended questions was developed. Faculty responded in this questionnaire after the completion of the MSP.

Impact on the institution

The number of research projects resulted in publications, conference presentations, and research collaborations were noted. A questionnaire with 17 Likert type statements was developed to measure students' satisfaction regarding the program. Perceptions regarding the factors that facilitated and hindered the conduct of the MSPs and measures taken to overcome those obstacles were collected using open-ended questions.

Analysis

Data analysis of attitude test was done using SPSS version 16. Change in students' total attitude score was done using Wilcoxon Signed Ranks test. The item wise responses were cross-tabulated and subjected to McNemar's test. The item response analysis for difficulty index and discrimination index of MCQs was done using Excel worksheets. Comparison of MCQs scores in pre- and post-test was done using the Independent Sample T test. Concurrent validity between the project report and poster evaluating criteria was assessed by Pearson's product-moment correlation coefficient (PPMCC). The students' and faculty's perceptions collected using other questionnaires were expressed as the median and the interquartile range (IQR). The students' and faculty responses to open-ended questions were tabulated in the decreasing order of frequency of their appearance. Later, two authors independently grouped the responses into three categories. Finally, the categories and responses were finalized by consensus among authors.


 :: Results Top


The response rate was 91.5% (140/153) during post-test of knowledge and retrospective pre-test of attitude.

The CA value for test retest reliability of attitude questionnaire was 0.879. The average total median score in attitude scale was 37 (32,40), which was significantly increased to 38 (32,44) after undertaking MSP (P = 0.036). After undertaking MSPs, statistically significant and positive changes in attitude were observed regarding most of the items except items number 5 and 10, where the change was negative [Table 1].
Table 1: Impact of mentored student project on students' attitude

Click here to view


The majority (87%) of MCQs was within the difficulty index of 20% to 80%, whereas 73% of MCQs had the discriminating index of ≥0.2. The students' scores in MCQ test showed statistically significant improvements (P < 0.001, 95% CI -1.62 to -0.71) following the MSPs (pre-test: 6.95 ± 1.95; post-test: 8.04 ± 2.15).

The majority of posters (68%) and project reports (62%) were of A grade. However, 20% and 18% of project reports were of B and C grade, respectively, whereas 25 % and 7% of posters were of B and C grade respectively. The PPMCC between scores obtained in posters and project reports was 0.418.

The CA values for questionnaires related to satisfaction and skills gained were 0.747 and 0.738 respectively. Median scores for most of the items denoting students' satisfaction and perceptions regarding skills gained were ≥3 [Table 2] and [Table 3].
Table 2: Students' responses [Mean and interquartile range (IQR)] to items related to satisfaction

Click here to view
Table 3: Students' perceptions [Median and inter quartile range (IQR)] regarding skills gained after MSP


Click here to view


Faculty response rate was 40%. All faculty who responded in the questionnaire had more than 7 years of teaching and research experiences, and had published their research work in scientific journals. [Table 4] depicts faculty responses to closed ended questions asked in the questionnaire.
Table 4: Faculty perspectives [Mean and inter quartile range (IQR)] regarding MSP

Click here to view


Majority of students felt that mentors' support, team work and guidance from MSP coordinators facilitated conduct of MSP. Lack of adequate time was the major hindering factor [Table 5].
Table 5: Students' responses to open-ended questions regarding the conduct of MSP

Click here to view


To this date, 14 MSPs resulted in publications in indexed journals and 15 resulted in conference presentations. Around 5 MSPs lead to inter-departmental and inter-institutional collaborations.


 :: Discussion Top


It is essential to impart knowledge and skills and inculcate a positive attitude regarding research among students from the beginning of their medical career through programs like MSP. The study initiated an evaluation of MSP program by measuring its short-term outcomes. Through this study, we developed valid instruments for assessing students' knowledge, attitude and skills regarding research, and satisfaction regarding the program. Furthermore, the study also led to the development of criteria for the assessment of posters and project reports. The instruments developed detected a positive change in students' attitude and knowledge gain while experiencing research. The present study, in addition, provided evidences for effectiveness and defined the scope for further refinement of the MSP program.

Retrospective pre-test [Table 1] demonstrated that the MSP changed students' attitude positively. The attitude test also showed that, after the MSP, students understood the importance of research in their career in a better way. Although there is a statistical significance, the post-intervention total attitude score was just 1 point more than the pre-intervention score. In the MSP, students completed a research project within a short span of 6 months simultaneously learning other subjects of the second year of MBBS course, including the preparation of project report and presentation of poster. It was unlikely that students had an opportunity to have an experience of doing a major project. Hence though the short-term research program like MSP inculcates the knowledge regarding research methodology, more intense and long-term research programs may be required see the major attitudinal change. The study also revealed that students perceived the MSP as stressful [Table 1] and [Table 2]. The MSPs were implemented in the curriculum without making any changes in the already existing one. Students were expected to work on projects while simultaneously learning other subjects of the MBBS course, so it was an extra burden on the part of students. Therefore, it is noteworthy to mention that any curriculum innovation has to be made along with pruning of some other material from the curriculum. However, it has to be remembered that removing any material may come at the expense of student learning. [19]

As the majority of project reports and posters were of A category, it could be assumed that participating in MSPs instilled research skills in students. The newly-developed project report and poster evaluation instruments could differentiate different grades of posters and project reports. There was a weak correlation between scores of project reports and posters. This could be due to inability of student to choose appropriate information from project report to prepare poster or students were more careful in designing project reports as they were aware that the poster would judged by external judges and the best posters would be identified and awarded.

This study demonstrated only a small change in students' knowledge after doing research. Our aim was not to quantify the students' knowledge on research but to test our hypothesis that MSPs improve students' knowledge in research and our instrument was able to detect that change.

Faculty responses regarding MSP reveal that they are ready to accept assignments of guiding students in MSPs, though they had not perceived any benefits from MSPs [Table 4]. Probably it was a very early phase of the experience of conducting MSP, and faculty might understand the benefits of it to students and themselves as they gain more experience from it. The mentors who responded in the questionnaire had >7 years of teaching and research experiences. Hence they may not have gained any additional benefits from guiding MSP.

Though this was a quasi-experimental study, the causative conclusion regarding the extent to which the MSPs influenced the knowledge, attitude change in students could not be made as there could be many factors that influenced this positive change. Moreover, the research skills gained was assessed through poster, project reports and students' perceptions, which does not tell actual skills gained.

Despite these limitations, this study acts as a road map to action plan for remedial measures like the introduction of MSPs without increasing students' overall workload and stress. It will be worthwhile to study the influence of MSPs on students' learning approaches. The long-term impact of MSP on students' career interest, scholarly work, and patients can also be explored.

Our model of undergraduate research project was found to be fostering positive attitude and a knowledge base about scientific research in students. As the majority of project reports and posters were of A category, it can be assumed that participating in MSPs instilled research skills in students. This model is practical and feasible to be implemented in undergraduate medical education worldwide. Moreover, researchers may use various instruments developed in this study across the globe for evaluating undergraduate research programs at their institute.


 :: Acknowledgements Top


We thank students who participated in this study. We are grateful to Faculty of Melaka Manipal Medical College, Manipal Campus, Manipal University and faculty and fellows of GSMC-FAIMER Regional Institute-Mumbai, India for providing helpful comments on aspects of this project.

 
 :: References Top

1.
Personne J, Gruppuso PA, Kanter SL, Boninger M. Required vs. elective research and in-depth scholarship programs in the medical student curriculum. Acad Med 2010:85:405-8.  Back to cited text no. 1
    
2.
Harasym PH, Mandin H, Sokol PA, Lorscheider FL. Development of a research elective program for first- and second-year medical students. Teach Learn Med 1992;4:173-9.  Back to cited text no. 2
    
3.
Ogunyemi D, Bazargan M, Norris K, Jones-Quaidoo S, Wolf K, Edelstein R, et al. Edelstein R. The development of a mandatory medical thesis in an urban medical school. Teach Learn Med 2005;17:363-9.  Back to cited text no. 3
    
4.
Shapiro J, Coggan P, Rubel A, Morohasi D, Fitzpatrick C, Danque F. The process of faculty-mentored student research in family medicine: Motives and lessons. Fam Med 1994;26:283-9.  Back to cited text no. 4
    
5.
Ebbert A Jr. A retrospective evaluation of research in the medical curriculum. J Med Educ 1960;35:637-43.  Back to cited text no. 5
[PUBMED]    
6.
Dehaven MJ, Chen L. Teaching medical students research while reaching the underserved. Fam Med 2005;37:315-7.  Back to cited text no. 6
    
7.
Bradford WD, Anthony D, Chu CT, Pizzo SV. Career characteristics of graduates of a medical scientist training program, 1970-1990. Acad Med 1996;71:484-7.  Back to cited text no. 7
    
8.
Boyer MH. A decade's experience at Tufts with a four-year combined curriculum in medicine and public health. Acad Med 1997;72:269-75.  Back to cited text no. 8
    
9.
Chongsiriwatana KM, Phelan ST, Skipper BJ, Rhyne RL, Rayburn WF. Required research by medical students and their choice of a women's health care residency. Am J Obstet Gynecol 2005;192:1478-80.  Back to cited text no. 9
    
10.
Watt CD, Greeley SW, Shea JA, Ahn J. Educational views and attitudes, and career goals of MD-PhD students at the University of Pennsylvania School of Medicine. Acad Med 2005;80:193-8.  Back to cited text no. 10
    
11.
Zorzi A, Rourke J, Kennard M, Peterson M, Miller K. Combined research and clinical learning make rural summer studentship program a successful model. Rural Remote Health 2005;5:401.  Back to cited text no. 11
    
12.
Harris R, Kinsinger LS, Tolleson-Rinehart S, Viera AJ, Dent G. The MD-MPH program at the University of North Carolina at Chapel Hill. Acad Med 2008;83:371-7.  Back to cited text no. 12
    
13.
Houlden RL, Raja JB, Collier CP, Clark AF, Waugh JM. Medical students' perceptions of an undergraduate research elective. Med Teach 2004;26:659-61.  Back to cited text no. 13
    
14.
Smith FG, Harasym PH, Mandin H, Lorscheider FL. Development and evaluation of a research project program for medical students at the University of Calgary Faculty of Medicine. Acad Med 2001;76:189-94.  Back to cited text no. 14
    
15.
Abelmann WH, Nave BD, Wilkerson L. Generation of physician-scientists manpower: A follow-up study of the first 294 graduates of the Harvard-MIT program of health sciences and technology. J Investig Med 1997;45:272-5.  Back to cited text no. 15
    
16.
Segal S, Lloyd T, Houts PS, Stillman PL, Jungas RL, Greer RB 3 rd . The association between students' research involvement in medical school and their postgraduate medical activities. Acad Med 1990;65:530-3.  Back to cited text no. 16
    
17.
Reinders JJ, Kropmans TJ, Cohen-Schotanus J. Extracurricular research experience of medical students and their scientific output after graduation. Med Educ 2005;39:237.  Back to cited text no. 17
[PUBMED]    
18.
Zier K, Friedman E, Smith L. Supportive programs increase medical students' research interest and productivity. J Investig Med 2006;54:201-7.  Back to cited text no. 18
    
19.
Bierer SB, Chen HC. How to measure success: The impact of scholarly concentration on students-a literature review. Acad Med 2010:85:438-52.  Back to cited text no. 19
    
20.
McLaughlin JA, Jordan GB. Logic models: A tool for telling your program's performance story. Eval Program Plann 1999;22:65-72.  Back to cited text no. 20
    
21.
Kumar R. Research Methodology: A Step-by-Step Guide for Beginners. 2 nd ed. SAGE publications ltd: Dorling Kindersley (India) Pvt. Ltd; 2005.   Back to cited text no. 21
    
22.
Kothari CR. Research Methodology: Methods and Techniques. 2 nd ed. New Age International: New Age International; 2008.  Back to cited text no. 22
    
23.
Glassick CE, Huber MT, Maeroff GI. Scholarship Assessed: Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass; 1997.  Back to cited text no. 23
    
24.
Bordage G, Caelleigh AS, Steinecke A, Bland CJ, Crandall SJ, McGaghie WC, et al., Review criteria for research manuscripts. Acad Med 2001;76:897-978.  Back to cited text no. 24
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4], [Table 5]



 

Top
Print this article  Email this article
 
Online since 12th February '04
2004 - Journal of Postgraduate Medicine
Official Publication of the Staff Society of the Seth GS Medical College and KEM Hospital, Mumbai, India
Published by Wolters Kluwer - Medknow