|
Reporting quality of multivariable logistic regression in selected Indian medical journals
R Kumar1, A Indrayan1, P Chhabra2,
1 Department of Biostatistics and Medical Informatics, University College of Medical Sciences, Delhi, India
2 Department of Community Medicine, University College of Medical Sciences, Delhi, India
Correspondence Address:
R Kumar
Department of Biostatistics and Medical Informatics, University College of Medical Sciences, Delhi
India
Abstract
Background: Use of multivariable logistic regression (MLR) modeling has steeply increased in the medical literature over the past few years. Testing of model assumptions and adequate reporting of MLR allow the reader to interpret results more accurately. Aims: To review the fulfillment of assumptions and reporting quality of MLR in selected Indian medical journals using established criteria. Setting and Design: Analysis of published literature. Materials and Methods: Medknow.com publishes 68 Indian medical journals with open access. Eight of these journals had at least five articles using MLR between the years 1994 to 2008. Articles from each of these journals were evaluated according to the previously established 10-point quality criteria for reporting and to test the MLR model assumptions. Statistical Analysis: SPSS 17 software and non-parametric test (Kruskal-Wallis H, Mann Whitney U, Spearman Correlation). Results: One hundred and nine articles were finally found using MLR for analyzing the data in the selected eight journals. The number of such articles gradually increased after year 2003, but quality score remained almost similar over time. P value, odds ratio, and 95% confidence interval for coefficients in MLR was reported in 75.2% and sufficient cases (>10) per covariate of limiting sample size were reported in the 58.7% of the articles. No article reported the test for conformity of linear gradient for continuous covariates. Total score was not significantly different across the journals. However, involvement of statistician or epidemiologist as a co-author improved the average quality score significantly (P=0.014). Conclusions: Reporting of MLR in many Indian journals is incomplete. Only one article managed to score 8 out of 10 among 109 articles under review. All others scored less. Appropriate guidelines in instructions to authors, and pre-publication review of articles using MLR by a qualified statistician may improve quality of reporting.
How to cite this article:
Kumar R, Indrayan A, Chhabra P. Reporting quality of multivariable logistic regression in selected Indian medical journals.J Postgrad Med 2012;58:123-126
|
How to cite this URL:
Kumar R, Indrayan A, Chhabra P. Reporting quality of multivariable logistic regression in selected Indian medical journals. J Postgrad Med [serial online] 2012 [cited 2023 Jun 9 ];58:123-126
Available from: https://www.jpgmonline.com/text.asp?2012/58/2/123/97174 |
Full Text
Introduction
Commonly used multivariable methods in medical literature are multiple linear regression, logistic regression and Cox-proportional hazards model. [1] Multivariable logistic regression (MLR) is used when response is binary and explanatories are two or more, which could be continuous, categorical, or ranked. MLR is commonly applied for two purposes: (i) to determine the effect of a specific covariate after adjustment for the potential confounders; and (ii) to estimate the likelihood of a response for an individual from the set of predictors.
All models are always associated with certain assumptions and need proper reporting that allow readers to interpret the study results more accurately. If the underlying assumptions are violated or not tested or merely assumed, this may impact the accuracy of the estimate of the parameters as well as their statistical significance.
Few studies have been conducted on reporting quality and testing of basic assumptions of statistical methods applied in the articles published in Indian medical journals. None is published on the multivariable methods. A recent study conducted on 196 original articles published in two Indian pharmacology journals between 2002 and 2010 found that only 22% reported appropriate descriptive statistics. Only two out of 196 articles mentioned about the assumptions related to the statistical method applied, and 69.3% articles used appropriate statistical methods. [1] Another study carried out on 45 original articles published in Indian pediatrics journal in 2007 and 2008 to evaluate appropriateness of statistical techniques used and type of statistical errors revealed that 92% of the articles had applied adequate statistical tests, while only 24% described sample size details, and confidence interval was calculated in 13%. [2] No study seems to have been done on the appropriateness of MLR in Indian medical journals.
Several papers assessed MLR assumptions and reporting quality in different fields of medicine in international journals [3],[4],[5],[6],[7],[8] and stressed the need to improve the reporting quality and to test assumptions. We replicate the study for the Indian medical journals because it is presently unknown whether articles published in Indian journals have similar deficiencies or not. We also studied the effect of some other parameters on MLR assumptions and reporting quality such as sample size, number of authors, journal- PubMed indexed or not, and involvement of statistician or epidemiologist as a co-author, which were not studied earlier.
Materials and Methods
The Medknow.com [9] website contains both PubMed-indexed and non-indexed medical journals and provides open access. It published a total of 79 medical journals as of December 2008 - out of these 68 are Indian and 11 are foreign journals. We scanned all these 68 Indian journals for articles on MLR. Journals with at least five articles using MLR for data analysis from 1994 to 2008 were included in the study to make a valid comparison across the journals.
To locate the eligible journals for this study, we typed keyword "logistic regression" in search option of Medknow. This search found 230 Indian articles from 68 Indian journals. Only eight journals had at least five valid MLR articles when letters, case reports, validation articles, etc. were excluded.
Bagley et al., [10] criteria were used to assess the quality regarding fulfillment of assumptions and reporting of MLR. These 10 criteria are (i) sufficient cases (>10) per covariate, (ii) conformity of linear gradient for continuous variables, (iii) testing of interactions, (iv) testing of collinearity, (v) validation of model, (vi) P value, odds ratio, and 95% confidence interval of odds ratio, (vii) selection of potential covariates, (viii) coding of covariates, (ix) goodness-of-fit or classification summary, and (x) model fitting procedure. Each criterion was assigned score 0 if not fulfilled, and 1 if fulfilled. Not applicable condition could be only in case of linear gradient where no continuous covariate was in the model or where all continuous covariates were converted into categorical. The score was standardized for this inapplicable criterion so that all had common base and were comparable.
The first author (RM) assessed the accuracy of MLR criterion. To determine the reliability of the first author, 30 articles on MLR were randomly selected by the third author (PC), and assessed independently and tested by Cohen's Kappa statistic where value ranged from 0.92 to 1.00 for these articles. Second author (AI) was consulted where scoring between first and third author were different and decision of second author was considered as final. We also studied the trend of number of articles and quality score over time by dividing years into four blocks ≤1999, 2000-2002, 2003-2005, and 2006-2008.
Statistical analysis
Since the total score was computed by adding the scores on 10 criteria considering equal weightage to each of criterion, the total score has a feature of a quasi-interval scale and it can be treated as interval-scale. Because of this feature normality of standardized score was tested by the Kolmogorov-Siminrov and Sharpiro-Wilk tests and both tests were significant with P=0.0001 and P=0.005, respectively. Our subsequent analysis revealed that this was due to the flat peakedness (Platykurtic) of the scores' distribution rather than for lack of symmetry. Thus, non-parametric Kruskal-Wallis test was applied for comparison of scores among the journals and among the four blocks of years. Mann-Whitney with Bonferroni adjustment was used for pair-wise comparisons only if quality score across the journals or among four block years were found to be significant. Spearman correlation was used to measure the association of quality score with the number of authors, and sample size. Mann-Whitney test was also used to compare the scores between other groupings. Exact method based on binomial distribution was applied to find 95% confidence interval when small number of articles fulfilled the criterion. All analysis was done by statistical software SPSS Version 17 (SPSS, Chicago, IL) and P value < 0.05 was considered as significant.
Results
The distribution of MLR articles according to journal, mean±SD, median (inter-quartile range), minimum, and maximum of standardized score are summarized in [Table 1].The median score was a meager 3.33 out of possible 10. The highest was 4.44 for Journal of Postgraduate Medicine and Indian Journal of Community Medicine (IJCM). The number of authors for these articles varied from 2 to 11 with median five authors. Sample size in these articles ranged from 29 to 11786, with median of 289 subjects.{Table 1}
The results according to the criteria are summarized in [Table 2]. It seems 41.3% articles might be over-fitted. The criterion of conformity of linear gradients was applicable to only 44 articles, but none mentioned or tested this conformity. P value was reported in 94.4% (103) articles.{Table 2}
Name of statistical software was specified in 88 (80.7%) of the 109 articles and the most common software was SPSS (SPSS; Chicago, IL) [72 articles, 66.1%].The standardized score among the journals was not found to be significantly different at 5% level (P=0.08).Trend of number of articles using MLR showed a steep increase in the 2003-2005 block, specifically year 2004 onwards [Figure 1]. However, there is no significant difference in quality score among the four block years (P=0.951).{Figure 1}
The involvement of a statistician or epidemiologist improved the score significantly (P=0.014 - one-tailed). No significant difference was found in quality score between PubMed-indexed and non-indexed journals (P=0.540). There was positive significant relationship between the sample size and quality score (Spearman ρ= 0.346, P=0.0035) whereas there was inverse but not significant relation between the number of authors and quality score (Spearman ρ=-0.083, P=0.394).
Discussion
The number of articles using MLR as a statistical tool increased in Indian medical journals over time but the trend of quality score did not show any significant improvement. Only three criteria were fulfilled in at least 50% of the reviewed articles. Two criteria - coding of independent variables and model fitting procedure were fulfilled by 41.3% and 45%, respectively, of these articles. The remaining five criteria were rarely met.
Inaccurate reporting of MLR models makes it difficult for the reader to understand the study results, for example, apparent effect of a covariate depends on the corresponding unit size of continuous covariates or coding of other covariates. [11]
Our findings are different from results obtained for articles published in international journals. For example, Mikolajczyk et al., [3] found 83% using coding of potential covariates in 104 randomly selected articles published in obstetrics and gynecology journals from 2005 and 2006 whereas this was 41.3% in our study for Indian journals. They found that assessment of possible interactions was reported in 18%, and this was only 3.67% in our study. These percentages in the study conducted by Kalil and colleagues [4] in six major journals on organ transplant from January 1, 2005 to January 1, 2006 were 9% for coding and 19% for possible interactions. Similarly, coding and possible interaction percentage in the study conducted by Ottenbacher and colleagues [8] in two epidemiology journals in the years 2000 and 2001 were 10% for coding and 39% for possible interactions, respectively. Reporting of model fitting procedure, collinearity and conformity of linear gradient of rank and continuous covariates was 49%, 4.7%, and 25% respectively in the Kalil study [4] and 65%, 17% and 19%, respectively in the Ottenbacher study, [6] whereas in our study these percentages are 45%, 1.83% and 0%. Thus, Indian journals are lagging behind in these respects. Results on other criteria like number of events per independent variable, validation of model, and significance (P value, odds ratio, and 95% of odds ratio) are consistent with findings for international journals. [3],[8]
The conformity of linear gradient was generally not tested by the researchers around the world. This may be due lack of awareness or non-availability of automatic option in the statistical software for this test. Another reason may be that it requires technical expertise.
Our study has some limitations such as inclusion of selected journals, use of specific criteria and incomplete information in some articles. Critics may argue that lack of reporting does not mean that the researchers did not test the assumptions. In addition, sometimes researchers are unable to describe fully due to space constraints. Nevertheless, the 10-point scoring we adopted is well-established.
Results of MLR can be confidently implemented only when all the quality criteria are adopted and clearly reported. The detailed description on multivariable analysis and how to report multivariable analysis in a scientific article is available in the literature. [12],[13]
Despite increased use of logistic regression in articles published in Indian journals, severe deficiencies in reporting of MLR were noted. In our opinion, the main reasons behind these deficiencies are: mostly researchers do only what their software allows to do. When the techniques become available in a standard software package, only then they will be used; lack of adequate training in statistical methods of medical researchers or medical professionals; shortage of competent biostatistics in the country; lack of statistical series or perspective/reviews on statistical methods in Indian medical journals; sometimes researchers are unwilling to contact the biostatistician.
The reporting quality of multivariable statistical methods can be improved by the joint efforts of authors, editors, and peer reviewers. When multivariable methods are applied by the author he or she should consult the biostatistician or gather the information related to the multivariable method from previous statistical reviews on the respective statistical method. Journals should incorporate a checklist of commonly used multivariable methods such as multivariable logistic regression in their instructions to the author and relax the word limit of the statistical analysis section, so that the author can explain the complete information of the statistical method used. Editors should generate more awareness by publishing statistical series or perspective/reviews articles, like [14] on commonly used univariable or multivariable statistical methods.
References
| 1 | Karan J, Goyal JP, Bhardwaj P, Yadav P. Statistical reporting in Indian pediatrics. Indian Pediatr 2009;46:811-2. |
| 2 | Karan J, Preet Y. Quality of reporting statistics in two pharmacology journals. J PharmacolPharmacother 2011;2:85-9. |
| 3 | Mikolajczyk RT, DiSilvestro A, Zhang J. Evaluation of logistic regression reporting in current obstetrics and gynecology literature. ObstetGynecol 2008;111:413-9. |
| 4 | Kalil AC, Mattei J, Florsescu DF, Sun J, Kalil RS. Recommendations for the assessment and reporting of multivariable logistic regression in transplantation literature. Am J Transplant 2010;10:1686-94. |
| 5 | Khan KS, Chien PF, Dwarakanath LS. Logistic regression models in obstetrics and gynecology literature. Obstet Gynecol 1999;93:1014-20. |
| 6 | Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable logistic regression models in the pulmonary and critical care literature. Chest 2003;123:923-8. |
| 7 | Mullner M, Mathews H, Altman DG. Reporting on statistical methods to adjust for confounding: A cross-sectional survey. Ann Intern Med 2002;136:122-6. |
| 8 | Ottenbacher KJ, Ottenbacher HR, Tooth L, Ostir GV. A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions. J ClinEpidemiol 2004;57:1147-57. |
| 9 | MedKnow.com. Mumbai:Publish high quality peer-reviewed scholarly journal. Available from:http://www.medknow.com/journals.asp[Last cited 2008 Dec 31]. |
| 10 | Bagley SC, White H, Golomb BA. Logistic regression in the medical literature: Standards for use and reporting, with particular attention to one medical domain. J ClinEpidemiol 2001;54:979-85. |
| 11 | Hosmer DW, Lemeshow S. Applied logistic regression. New York: Wiley; 1989. |
| 12 | Katz MH. Multivariable analysis: A primer for readers of medical research. Ann Intern Med 2003:138:644-50. |
| 13 | Lang T. Documenting research in scientific articles: Guidelines for authors: 3. Reporting multivariable analysis. Chest 2007;131:628-32. |
| 14 | Kumar R, Indrayan A. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr 2011;48:277-86. |
|
