Journal of Postgraduate Medicine
 Open access journal indexed with Index Medicus & ISI's SCI  
Users online: 4051  
Home | Subscribe | Feedback | Login 
About Latest Articles Back-Issues Article Submission Resources Sections Etcetera Contact
 
  NAVIGATE Here 
  Search
 
  
 RESOURCE Links
 ::  Similar in PUBMED
 ::  Search Pubmed for
 ::  Search in Google Scholar for
 ::  Article in PDF (237 KB)
 ::  Citation Manager
 ::  Access Statistics
 ::  Reader Comments
 ::  Email Alert *
 ::  Add to My List *
* Registration required (free) 

  IN THIS Article
 ::  References
 ::  Article Tables

 Article Access Statistics
    Viewed2394    
    Printed62    
    Emailed0    
    PDF Downloaded24    
    Comments [Add]    
    Cited by others 2    

Recommend this journal


 


 
  Table of Contents     
LETTER
Year : 2011  |  Volume : 57  |  Issue : 2  |  Page : 176-177

Negative studies published in medical journals of India do not give sufficient information regarding power/sample size calculation and confidence interval


1 Department of Pharmacology, Govt. Medical College, Surat, India
2 Department of Community Medicine, Govt. Medical College, Surat, India

Date of Web Publication4-Jun-2011

Correspondence Address:
Jaykaran
Department of Pharmacology, Govt. Medical College, Surat
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0022-3859.81861

Rights and Permissions




How to cite this article:
Jaykaran, Saxena D, Yadav P, Kantharia N D. Negative studies published in medical journals of India do not give sufficient information regarding power/sample size calculation and confidence interval. J Postgrad Med 2011;57:176-7

How to cite this URL:
Jaykaran, Saxena D, Yadav P, Kantharia N D. Negative studies published in medical journals of India do not give sufficient information regarding power/sample size calculation and confidence interval. J Postgrad Med [serial online] 2011 [cited 2019 Nov 14];57:176-7. Available from: http://www.jpgmonline.com/text.asp?2011/57/2/176/81861


Sir,

Many studies published in medical journals are considered as negative studies as they fail to disprove the null hypothesis. These kinds of studies should be interpreted cautiously as chances of false interpretation are large. A reader can consider an intervention not statistically significant as compared to comparator (placebo or drug) on the basis of negative results but this difference may be because of either actual no difference or because of less sample size hence less power or by chance (Type 2 error). So a study should have an adequate sample size/ power so that negative results because of less sample size can be ruled out. One most important reason for these underpowered studies is: no calculation of sample size during early phase of the study. [1] Sample size/power calculation is very important during the design phase of the study and its method should be reported in sufficient detail so that readers can asses the validity of the study. [1] Sufficient information about the methodological parameters should be given in the published manuscript of negative studies so that readers can asses the validity of the study independently as a poorly conducted negative study may not be valid but may discourage the reader to do work in the same area. This is very important in the light of findings which show that many negative studies are underpowered to see moderate and large differences in outcome and many of them have incorrect statistics. [2],[3] In the case of negative studies, reporting of one more component is important and that is the confidence interval. Confidence interval gives a range in which the actual population value may lie and also gives an idea about the clinical significance. [4] Studies done for Western journals show that information about power/sample size and confidence interval is not reported adequately in published studies. [5],[6],[7],[8] But information is lacking about the studies published in Indian medical journals and specially negative studies, so this study was designed with the aim of surveying negative studies published in Indian medical journals to see whether or not adequate information is given regarding the sample size/power calculation and confidence interval.

PubMed indexed journals subscribed by the central library of our institute were taken into consideration. These journals were Annals of Indian Academy of Neurology (AIAN), Indian Journal of Orthopedics (IJOrtho), Indian Journal of Critical Care Medicine (IJCCM), Indian Journal of Dermatology, Venereology and Leprology (IJDVL), Indian Journal of Nephrology (IJN), Indian Journal of Dermatology (IJD), Indian Journal of Ophthalmology (IJO), Indian Journal of Urology (IJU), Indian Journal of Anesthesia (IJA), Indian Journal of Psychiatry (IJPsy), Indian Pediatrics (IP), Indian Journal of Medical Research (IJMR), Indian Journal of Medical Science (IJMS) and Indian Journal of Community Medicine (IJCM). All the original articles published in these journals in 2009 were evaluated by first author (JK) to sort out the negative studies from them. Studies were considered negative studies on the basis of these criteria- one primary outcome is not statistically significant, second most important outcome is not statistically significant (if primary outcome is not reported). Most important outcome was considered to be that outcome for which either sample size calculation was done or which came first in the abstract. [9] All negative studies were evaluated by first author (JK) for reporting of power, sample size and confidence interval. Second author (DS) reassessed all the negative studies (k = 0.85) and any discrepancies were resolved by consensus. Descriptive statistics in the form of frequency and percentage were used. Reporting of sample size in clinical trials and other studies were compared with Fisher's exact test.

Out of 276 articles published in these 14 journals in 2009, 52 (18.8%, 95% CI 14.6-23.8%) were negative studies [Table 1]. Out of 52 negative studies 39 (76.4%) were prospective studies, nine (17.6%) were clinical trials and four (7.6%) were retrospective studies. Out of 52 negative studies sample size calculation was reported in only 10 (19.2%, 95% CI 10.8- 1.9%). Out of these 10 articles all components of sample size calculation (Type 1 error, Type 2 error, power, delta) was reported in seven articles, only power was reported in the remaining three articles. Out of the seven articles reporting effect size (delta), rational for selection of particular effect size was mentioned in only 3 articles . Out of 52 negative studies confidence interval was reported in only three (5.7%, 95% CI 1.9-15.6%) studies. Average sample size was 54.3. Most of the studies had a sample size less than 40 per group [Table 2]. It was observed that reporting of sample size was more in clinical trials as compared to other studies (5/9 Vs. 5/43, P = 0.010) [Table 3]. Out of these 52 negative studies lower sample size/power was mentioned as a limitation in 14 (26.9%, 95% CI 16.7-40.2%) articles.
Table 1: Negative studies published in Indian medical journals

Click here to view
Table 2: Sample size of negative studies published in Indian medical journals (n = 52)

Click here to view
Table 3: Reporting of sample size in negative studies published in Indian medical journals

Click here to view


On the basis of this study it can be concluded that negative studies published in Indian Medical Journals are grossly deficient in the reporting of power/sample size calculation and confidence interval hence it is difficult to check the validity of these studies. These findings are similar to studies done for Western journals. [3],[4],[5],[6] In a study done by Moher et al., it was observed that sample size was calculated in only 32% negative studies which are more than observed in this study. [7]

It is observed that many of the medical journals are biased towards publication of positive studies. Almost 90% of published articles deal with positive results. A properly conducted negative study should be published so that a researcher planning to do similar work may get some idea about the probable results and hence can save resources. Most of the meta-analyses and systematic reviews are based on published literature so the conclusions reached in these reviews may get biased as most of the published studies are positive studies. [10],[11]

 
 :: References Top

1.Jaykaran, Yadav P, Chavda N, Kantharia ND. Some issue related to the reporting of statistics in clinical trials published in Indian medical journals: A Survey. Int J Pharmacol 2010;6:354-9.  Back to cited text no. 1
    
2.Jaykaran, Saxena D, Yadav P, Kantharia ND. Solanky P. Quality of reporting of descriptive and inferential statistics in negative studies published in Indian Medical Journals. J Pharm Negat Results 2011 [In Press].  Back to cited text no. 2
    
3.Jaykaran, Saxena D, Yadav P, Kantharia ND. Most of the inconclusive studies published in Indian Medical Journals are underpowered. Indian Pediatr 2011 [In Press]  Back to cited text no. 3
    
4.Gardner MJ, Altman DG. Confidence intervals rather than p-values: Estimation rather than hypothesis testing. BMJ (Clin Res Ed) 1986;292:746-50.  Back to cited text no. 4
    
5.Hebert RS, Wright SM, Dittus RS, Elasy TA. Prominent medical journals often provide insufficient information to assess the validity of studies with negative results. J Negat Results Biomed 2002;1:1.  Back to cited text no. 5
[PUBMED]  [FULLTEXT]  
6.Dimick JB, Diener-West M, Lipsett PA. Negative results of randomized clinical trials published in the surgical literature: Equivalency or error? Arch Surg 2001;136:796-800.  Back to cited text no. 6
[PUBMED]  [FULLTEXT]  
7.Williams HC, Seed P. Inadequate size of 'negative' clinical trials in dermatology. Br J Dermatol 1993;128:317-26.  Back to cited text no. 7
[PUBMED]    
8.Keen HI, Pile K, Hill CL. The prevalence of under-powered randomized clinical trials in rheumatology. J Rheumatol 2005;32:2083-8.  Back to cited text no. 8
[PUBMED]  [FULLTEXT]  
9.Moher D, Dulberg CS, Wells GA. Statistical power, sample size, and their reporting in randomized controlled trials. JAMA 1994;272:122-4.  Back to cited text no. 9
[PUBMED]    
10.Ali J. Negative results as part of quality research: An insight. J Pharmacol Negat Results 2010;1:4-5.  Back to cited text no. 10
    
11.Crowther M, Lim W, Crowther MA. Systematic review and meta-analysis methodology. Blood 2010;116: 3140-6.  Back to cited text no. 11
[PUBMED]  [FULLTEXT]  



 
 
    Tables

  [Table 1], [Table 2], [Table 3]

This article has been cited by
1 Comparison of methodological quality of positive versus negative comparative studies published in Indian medical journals: a systematic review
Jaykaran Charan,Mayur Chaudhari,Ryan Jackson,Rahul Mhaskar,Tea Reljic,Ambuj Kumar
BMJ Open. 2015; 5(6): e007853
[Pubmed] | [DOI]
2 How to calculate sample size for different study designs in medical research?
Charan, J. and Biswas, T.
Indian Journal of Psychological Medicine. 2013; 35(2): 121-126
[Pubmed]



 

Top
Print this article  Email this article
 
Online since 12th February '04
2004 - Journal of Postgraduate Medicine
Official Publication of the Staff Society of the Seth GS Medical College and KEM Hospital, Mumbai, India
Published by Wolters Kluwer - Medknow