 |
|
||||
|
|||||
| About | Latest Articles | Back-Issues | Articles
|
Search | Instructions | Online Submission | Subscribe | Etcetera | Contact |
| |||||||||||||||||||||
|
Short-term outcomes of a program developed to inculcate research essentials in undergraduate medical students V Devi1, K Ramnarayan2, RR Abraham3, V Pallath4, A Kamath5, S Kodidela61 Department of Pharmacology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India 2 Department of Pathology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India 3 Department of Physiology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India 4 Department of Microbiology, Melaka Manipal Medical College, Manipal Camps, Manipal University, Manipal, Karnataka, India 5 Department of Community Medicine, Kasturba Medical College, Manipal, Karnataka, India 6 Department of Pharmacology, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0022-3859.159315
Background: Participation in research during undergraduate studies may increase students' interest in research and inculcate research essentials in them. Aims: The purpose of this study was to evaluate the effectiveness of the mentored student project (MSP) program. Settings and Design: In the MSP program, students in groups (n = 3 to 5) undertook a research project, wrote a scholarly report, and presented the work as a poster presentation with the help of a faculty mentor. To begin with, the logic model of the program was developed to identify short-term outcomes of the program on students, mentors, and the institution. A quasi-experimental design was used to measure the outcomes. Materials and Methods: A mixed method evaluation was done using a newly-developed questionnaire to assess the impact of the MSP on students' attitude, a multiple-choice question (MCQs) test to find out the impact on students' knowledge and grading of students' project reports and posters along with a survey to check the impact on skills. Students' satisfaction regarding the program and mentors' perceptions were collected using questionnaires. Evidence for validity was collected for all the instruments used for the evaluation. Statistical Analysis: Non-parametric tests were used to analyze data. Based on the scores, project reports and posters were graded into A (>70% marks), B (60-69% marks), and C (<59% marks) categories. The number of MSPs that resulted in publications, conference presentation and departmental collaborations were taken as impact on the institution. Results: Students' response rate was 91.5%. The students' attitudes regarding research changed positively (P = 0.036) and score in the MCQ test improved (P < 0.001) after undertaking MSP. Majority of project reports and posters were of grade A category. The majority of the items related to skills gained and satisfaction had a median score of 4. The MSPs resulted in inter-departmental and inter-institutional collaborations, 14 publications and 15 conference presentations. An area for improvement noted was to have the MSP implemented in the curriculum without increasing students' overall workload and stress. Conclusion: The study identified strengths and weaknesses of the MSP program. Our model of undergraduate research project may be incorporated in undergraduate medical programs to foster positive attitude and knowledge base about scientific research and to instil research skills among students. Keywords: Attitude, knowledge, mentored, outcome, research, skills, student
The evolution of the medical field requires that medical educators constantly and systematically look at better ways to prepare future physicians. [1] Despite the absence of definite data, it is believed that student participation in research during undergraduate studies during medical school leads to better clinicians or physician scientists who will be able to create and apply new knowledge related to biomedicine. [1] Studies have shown that research experience during medical school training increased students' interest in research, [2],[3],[4] improved their confidence in conducting research, [4] influenced students' choice of clinical specialty, [5],[6],[7],[8],[9],[10] improved knowledge regarding research principles, [2],[3],[6],[11] fostered abilities to critically evaluate literature [7],[12],[13] and skills to write scientific papers. [14] The knowledge and skills gained while undertaking research work may further increase students' participation in research thereby increasing students' publications and presentations. [15],[16],[17] Despite the above evidences on educational benefits of research projects in medical school, whether such programs should be required as a part of the medical school curriculum is still debatable. [1] Studies have shown that these activities caused stress on students. [2],[4],[18] Their improper implementation in the curriculum may have several negative effects on students' learning. [1] When educational interventions become an extra burden on students, it may adversely affect their performance in examinations by affecting their learning and distracting them from studying for examinations. Bierer and Chen (2010) in their systematic review on the impact of scholarly concentration on medical students, recommend the requirement of more rigorous evaluation designs to show the positive and negative effects of such programs. [19] The aim of this study was to initiate program evaluation of an introduced curriculum change, mentored student project (MSP) at the institution. The primary objective was to measure the short-term outcomes of MSPs on students' knowledge, attitude, and skills regarding research. This study also describes outcomes of MSP on faculty and the institution. During the study we also developed instruments to measure the same, as systematic instrument design and use of reliable instruments are vital if the results of education testing are to be considered valid.
Educational context and subjects The MSP program was implemented in the curriculum in 2007 with the intention of developing research skills essential for the career development of medical students. In MSPs, medical students (n = 141) were required to undertake a research project involving in-depth study of an area of their interest, write a scholarly report, and present the work as a poster presentation in 6 month duration during the second year of the course. The MSP coordination committee with two faculty members as coordinators and Dean as the chairman functioned to monitor the smooth conduct of the MSP program. Faculty coordinators oriented students regarding the MSPs; research methodology; preparation of project report; and the poster preparation and presentation. The coordinators supervised student projects done under the guidance of faculty mentors and also ensured that the time line was followed. Keeping in mind the cultural background and specific needs of the international students, our medical school took the initiative in developing a mentorship scheme, in which the mentors (faculty members who are basic scientists) were required to supervise the academic progress of those students who were assigned to them. Mentors must review and approve the project proposal and final report. Mentors guide the students during all phases of the project from research topic selection, ethical consideration, writing the proposal, data collection, analysis and interpretation, writing up of final report and designing poster for poster presentation. Feedback taken from faculty and students of the institution in 2010 indicated the need for evaluation of the MSP program. To begin with the logic model [20] of the MSP program was developed to identify short-term outcomes of the program [Figure 1].
The ethics committee of the institution approved present quasi-experimental study. The convenience sampling method was used. Written informed consent was obtained from the students and faculty before responding in instruments. Assessing outcomes All the instruments required for assessing the outcomes of the MSP were developed using standard protocols and involving four experts with the Foundation for Advancement of Medical Education and Research (FAIMER) fellowship degree. To develop questionnaires, experts checked content validity individually. The questionnaire was further refined using a Delphi process in a series of meetings with a panel having above experts. All closed-ended questions in questionnaires were in a 5-point Likert scale (1 = disagree and 5 = strongly agree). Students' knowledge Fifteen multiple-choice questions (MCQs) were developed by coordinators based on standard literature on research methodology [21],[22] and standard processes to establish validity and reliability. To test students' knowledge, the finalized 15 MCQ administered to students before (n = 153) and soon after (n = 140) completing MSP with poster presentation being the final requirement for the completion. Students' attitude To test the impact of the MSPs on students' attitude, a questionnaire was developed after a literature review. [19] To assess test retest reliability, students were asked to respond in the questionnaire twice with a week interval in between. Following this, items with low a Cronbach's α (CA) value were deleted. The final questionnaire with 13 items was used to assess students' (n = 140) attitude by doing a retrospective pretest. Students' responded in the questionnaire in a 5-point Likert scale. Students' skills Quality of students' project reports and posters along with students' perception on skills gained gives evidence for the impact of MSP on research skills of students. Students' project report and poster evaluation instruments (with 35 and 20 criteria respectively) were developed based on Glassick's six standards for assessing scholarships [23] and review criteria for scholarly manuscript. [24] The questionnaire developed to assess students' perceptions regarding skills gained had 15 items with a Likert scale. After the completion of the MSP, two evaluators independently evaluated students' posters (n = 41) and project reports (n = 39). The percentage marks scored was used for grading the reports and posters as A (>70% marks), B (60-69% marks), and C (<59% marks) categories. Impact on faculty A questionnaire with 13 Likert-type statements and 4 open ended questions was developed. Faculty responded in this questionnaire after the completion of the MSP. Impact on the institution The number of research projects resulted in publications, conference presentations, and research collaborations were noted. A questionnaire with 17 Likert type statements was developed to measure students' satisfaction regarding the program. Perceptions regarding the factors that facilitated and hindered the conduct of the MSPs and measures taken to overcome those obstacles were collected using open-ended questions. Analysis Data analysis of attitude test was done using SPSS version 16. Change in students' total attitude score was done using Wilcoxon Signed Ranks test. The item wise responses were cross-tabulated and subjected to McNemar's test. The item response analysis for difficulty index and discrimination index of MCQs was done using Excel worksheets. Comparison of MCQs scores in pre- and post-test was done using the Independent Sample T test. Concurrent validity between the project report and poster evaluating criteria was assessed by Pearson's product-moment correlation coefficient (PPMCC). The students' and faculty's perceptions collected using other questionnaires were expressed as the median and the interquartile range (IQR). The students' and faculty responses to open-ended questions were tabulated in the decreasing order of frequency of their appearance. Later, two authors independently grouped the responses into three categories. Finally, the categories and responses were finalized by consensus among authors.
The response rate was 91.5% (140/153) during post-test of knowledge and retrospective pre-test of attitude. The CA value for test retest reliability of attitude questionnaire was 0.879. The average total median score in attitude scale was 37 (32,40), which was significantly increased to 38 (32,44) after undertaking MSP (P = 0.036). After undertaking MSPs, statistically significant and positive changes in attitude were observed regarding most of the items except items number 5 and 10, where the change was negative [Table 1].
The majority (87%) of MCQs was within the difficulty index of 20% to 80%, whereas 73% of MCQs had the discriminating index of ≥0.2. The students' scores in MCQ test showed statistically significant improvements (P < 0.001, 95% CI -1.62 to -0.71) following the MSPs (pre-test: 6.95 ± 1.95; post-test: 8.04 ± 2.15). The majority of posters (68%) and project reports (62%) were of A grade. However, 20% and 18% of project reports were of B and C grade, respectively, whereas 25 % and 7% of posters were of B and C grade respectively. The PPMCC between scores obtained in posters and project reports was 0.418. The CA values for questionnaires related to satisfaction and skills gained were 0.747 and 0.738 respectively. Median scores for most of the items denoting students' satisfaction and perceptions regarding skills gained were ≥3 [Table 2] and [Table 3].
Faculty response rate was 40%. All faculty who responded in the questionnaire had more than 7 years of teaching and research experiences, and had published their research work in scientific journals. [Table 4] depicts faculty responses to closed ended questions asked in the questionnaire.
Majority of students felt that mentors' support, team work and guidance from MSP coordinators facilitated conduct of MSP. Lack of adequate time was the major hindering factor [Table 5].
To this date, 14 MSPs resulted in publications in indexed journals and 15 resulted in conference presentations. Around 5 MSPs lead to inter-departmental and inter-institutional collaborations.
It is essential to impart knowledge and skills and inculcate a positive attitude regarding research among students from the beginning of their medical career through programs like MSP. The study initiated an evaluation of MSP program by measuring its short-term outcomes. Through this study, we developed valid instruments for assessing students' knowledge, attitude and skills regarding research, and satisfaction regarding the program. Furthermore, the study also led to the development of criteria for the assessment of posters and project reports. The instruments developed detected a positive change in students' attitude and knowledge gain while experiencing research. The present study, in addition, provided evidences for effectiveness and defined the scope for further refinement of the MSP program. Retrospective pre-test [Table 1] demonstrated that the MSP changed students' attitude positively. The attitude test also showed that, after the MSP, students understood the importance of research in their career in a better way. Although there is a statistical significance, the post-intervention total attitude score was just 1 point more than the pre-intervention score. In the MSP, students completed a research project within a short span of 6 months simultaneously learning other subjects of the second year of MBBS course, including the preparation of project report and presentation of poster. It was unlikely that students had an opportunity to have an experience of doing a major project. Hence though the short-term research program like MSP inculcates the knowledge regarding research methodology, more intense and long-term research programs may be required see the major attitudinal change. The study also revealed that students perceived the MSP as stressful [Table 1] and [Table 2]. The MSPs were implemented in the curriculum without making any changes in the already existing one. Students were expected to work on projects while simultaneously learning other subjects of the MBBS course, so it was an extra burden on the part of students. Therefore, it is noteworthy to mention that any curriculum innovation has to be made along with pruning of some other material from the curriculum. However, it has to be remembered that removing any material may come at the expense of student learning. [19] As the majority of project reports and posters were of A category, it could be assumed that participating in MSPs instilled research skills in students. The newly-developed project report and poster evaluation instruments could differentiate different grades of posters and project reports. There was a weak correlation between scores of project reports and posters. This could be due to inability of student to choose appropriate information from project report to prepare poster or students were more careful in designing project reports as they were aware that the poster would judged by external judges and the best posters would be identified and awarded. This study demonstrated only a small change in students' knowledge after doing research. Our aim was not to quantify the students' knowledge on research but to test our hypothesis that MSPs improve students' knowledge in research and our instrument was able to detect that change. Faculty responses regarding MSP reveal that they are ready to accept assignments of guiding students in MSPs, though they had not perceived any benefits from MSPs [Table 4]. Probably it was a very early phase of the experience of conducting MSP, and faculty might understand the benefits of it to students and themselves as they gain more experience from it. The mentors who responded in the questionnaire had >7 years of teaching and research experiences. Hence they may not have gained any additional benefits from guiding MSP. Though this was a quasi-experimental study, the causative conclusion regarding the extent to which the MSPs influenced the knowledge, attitude change in students could not be made as there could be many factors that influenced this positive change. Moreover, the research skills gained was assessed through poster, project reports and students' perceptions, which does not tell actual skills gained. Despite these limitations, this study acts as a road map to action plan for remedial measures like the introduction of MSPs without increasing students' overall workload and stress. It will be worthwhile to study the influence of MSPs on students' learning approaches. The long-term impact of MSP on students' career interest, scholarly work, and patients can also be explored. Our model of undergraduate research project was found to be fostering positive attitude and a knowledge base about scientific research in students. As the majority of project reports and posters were of A category, it can be assumed that participating in MSPs instilled research skills in students. This model is practical and feasible to be implemented in undergraduate medical education worldwide. Moreover, researchers may use various instruments developed in this study across the globe for evaluating undergraduate research programs at their institute.
We thank students who participated in this study. We are grateful to Faculty of Melaka Manipal Medical College, Manipal Campus, Manipal University and faculty and fellows of GSMC-FAIMER Regional Institute-Mumbai, India for providing helpful comments on aspects of this project.
[Figure 1]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||
