Original Article

Statistical Errors in Medical Residency Theses

10.14235/bas.galenos.2021.5145

  • Ayşegül YABACI TAK
  • Fatma Ezgi CAN
  • Fisun KASKIR KESİN
  • Robab AHMADIAN
  • İlker ERCAN

Received Date: 14.07.2020 Accepted Date: 14.02.2021 Bezmialem Science 2022;10(2):157-162

Objective:

The aim of this study is to evaluate the theses of residency in medicine in terms of statistical errors made and thus to contribute to the production of quality scientific publications by ensuring that scientific publishers in the field of medicine are sensitive and careful about statistics when doing their work.

Methods:

In this study, we investigated 321 thesis theses which are defended from 6 different universities are obtained from the database of the Turkish Higher Education Council. The investigation is was conducted in terms of “Errors Related to p-values”, “Errors Related to Tests”, “Mathematical Notation Errors”, “Statistical Symbol Errors”, “Inappropriate Interpretation”, “Presentation of The the Statistical Method Analysis and Results in The the Incorrect Section of The the Manuscript”, “Errors in Summarizing Data”,“Incomprehensible Statistical Terms” and “Errors in Statistical Terminology”.

Results:

There was at least one statistical error in all 321 medicine residency theses examined. The most common error was “errors in summarizing data” with a ratio of 70.1% (n=225), while the least common error was “incomprehensible statistical expressions” with a ratio of 14.3% (n=46).

Conclusion:

As a result, both researchers and consultants who undertake scientific studies have a responsibility to minimize these errors. To prevent statistical errors, students who are doing residency in medicine are required to receive the necessary training in statistical literacy, to have basic statistical knowledge, and to receive consultancy from a biostatistics expert for statistical evaluations. Students who residency in medicine in preventing statistical errors are required to receive the necessary training in statistical literacy, to have basic statistical knowledge, and to receive consultancy from a biostatistics expert for statistical evaluations.

Keywords: Statistical errors, residency in medicine thesis, statistical review

Introduction

Statistical science is used in all stages of scientific research, from the preparation stage of a studyies through to the reporting stage. It is known that the correct use of statistics and statistical procedures is extremely important in scientific studies today. Medical authorities also stress the importance of statistics and note that physicians should be a good reader of statistics, both in scientific studiesy and in the conduct of specialized theses. Researchers who lack adequate statistical knowledge may make error in using statistical science at any stage of the research, including data planning, design, execution, analysis and presentation. When the medical literature was examined, it was observed that the majority of studies had errors in terms of statistical procedures. Some of these errors have had a major effect on the results by directly affecting the results, while others are were presentation errors in terminology and dido not have a major effect on the outcome (1-3).

Statistical errors can be seen in the articles as well as in the theses of residency in medicine. Although the authors are responsible for the errors made in the thesis work as majors, the thesis advisors are also responsible for the academic prestige of the work.

Subsequent publication of thesis work bywith statistical errors would also result in a loss of academic confidence of the thesis authors and their thesis advisor. Therefore, consultants in medical residency training are required to guide their students for their statistical knowledge proficiency and to direct them to receive biostatistical counseling in their studies. Many researchers seeking to draw attention to errors and deficiencies in statistics and methodology have emphasized since the 1960s the importance of accurate use of statistics in scientific publications in medical journals since the 1960s (4,5). When studies in literature studies were examined, it was found that some subject matter experts looked at the research design used, the design flaws and mistakes, and the inappropriate use of design in medical studies. Some subject matter experts have examined the statistical tests used, the use of unsuitable statistical methods, and the failure of statistical tests used in medical studies to be specified in the study (6,7). Similarly, some subject matter experts have worked on errors in summarizing data and errors in statistical terminology (8-11). Some subject matter experts have emphasized the knowledge of statistics and statistical training for clinicians in their work (11-14). It is observed that statistical errors were made in medical studies due to the lack of statistical knowledge of researchers lack of statistical knowledge and their failure to consult a biostatistics expert. For this reason, the first step of the medical residency theses should be checking statistical errors in the requirement assessment. For this reason, the first step of the medical residency theses of the scientific study of physicians occurs in terms of statistical errors in the requirement assessment. It is important to receive advice from a biostatistics expert in the planning phase, analysis phase, reporting phase, as well as consulting for statistical review and evaluation of statistical quality prior to publication of articles (5,12,15-17).

The aim of this study is to statistically evaluate the theses of medical residency students. In this way, the statistical competencies of the specialized students will be evaluated.


Methods

When literature research on statistical errors were analyzed, it was found that in most of the researches at least one statistical error was present. In the study of statistical errors in psychiatry journals, it was reported that 40% of 164 studies had statistical errors (18). The error rate was found to be 50% in the study conducted by Glantz (15). In 52% of 62 scientific papers reviewed by Gore et al. (8), at least one statistical error has beenwas found. Lukić and Marušić (19) reported that statistics were not reliable in 63% of 144 articles published in medical journals 63% of the 144 articles published by Lukić and Marušić (19) in medical journals reported that the statistics were not reliable. Šimundić and Nikolac (20) reported at least one error in 87% of 55 articles submitted to the medical journal. Ercan et al. (21) reported statistical errors in 96% of 181 studies submitted to the Medical Sciences journal Medical Sciences reported statistical errors. Statistical errors were found in 93% of 157 articles reviewed by Günel Karadeniz et al. (2) in radiology journals. When the studies mentioned above are were taken as reference, the median ratio determined according to the studies is was 0.63 (0.40-0.96). In the light of this knowledge, it was decided to review 321 medical residency theses in terms of statistical errors for a=0.05 significance level and d=0.052 margin of error.

In our study, by scanning the database of the Turkish Higher Education Council (THEC), 321 medical residency theses belonging to different departments of 6 different medical faculties with at least 40 years of experience were randomly selected. The selected medical residency theses were examined in terms of statistical errors by five researchers (A. Yabacı Tak, F.E. Can, F. Kaskır Kesin, R. Ahmadian ve I. Ercan) who were experts in Biostatistics. The medical residency theses were first evaluated individually by subject matter experts, and then evaluated together by five researchers. The classification of the statistical errors found in the residency theses was done in accordance with the statistical error groupings specified in the studies of Ercan and Demirtas (1) and Ercan et al. (7). Accordingly, statistical errors were classified as follows;

Errors relating to the p-value;

• p-values given in closed form (p<0.01, p<0.05, p>0.05,etc.),

• Non-reported p-values,

• Incorrect p-values,

• Incorrect demonstration of p-values (p=0.000, p<0.0005, etc.).

Errors relating to the statistical tests;

• Statistical technique defined but not used,

• Incorrect name for the statistical test,

• Undefined statistical test,

• Use of incorrect test,

• Statistical analysis required but not performed.

Other errors;

• Mathematical demonstration errors,

• Statistical symbol errors,

• Incomprehensible statistical terms,

• Inappropriate interpretation,

• Errors in (statistical) terminology,

• Errors in summarizing data,

• Presentation of statistical method-analysis and results in the incorrect section of the manuscript.

Statistical Analysis

Statistical errors identified by each researcher were confirmed by the study team. In this way, a complete harmony between the researchers was achieved. Therefore, the inter- rater reliability criterion was not calculated. Statistical errors were presented as frequency and percentage, taking into account the number of medical residency theses studied. IBM SPSS Statistics 22.0 was used to analyze the data.


Results

There were at least one statistical error in all 321 medical residency theses. The most common error was “errors in summarizing data” with a ratio of 70.1% (n=225), while the least common error was “incomprehensible statistical expressions” with a ratio of 14.3% (n=46). The details of the distribution of statistical errors made in the medical residency theses examined are given in Table 1.


Discussion

In this study, statistical errors made in medical residency theses were examined. In our literature research, while there were articles on statistical errors in the publications, there was no study on statistical errors in in which studies on residency theses in medicine were conducted.

When a residency thesis it is turned into a scientific publication after the residency theses, it is used as a reference by scientists and offered to human service. Therefore, the accuracy and reliability of residency thesis studies is very important. Statistics is one of the most important factors for the accuracy and reliability of a scientific study, from the planning stage of the study to the reporting stage. Studies of medicine residency thesis studies is are the first step of academic studies for clinicians. For this reason, in this study, it was investigated whether there are were statistical errors in residency theses in medicine and what kinds of statistical errors, if any. Statistical errors in studies are generally classified as errors related to p-value, errors related to statistical tests and other errors. When 321 medical residency theses obtained randomly from THEC database were examined, errors related to p-value were found to be relatively high. It is known that this error is an important error considering the importance of p-value. In the medicine residency theses examined, the most common mistake about p-value was the wrong display of p-value with a rate of 47.4%, followed by “p-value with a closed form” of with a rate of 46.4%. When similar studies in the literature were examined, this ratio was found to be 18.43% and 37.25% respectively in the studies of Ercan et al. (21) in 2015 and 2017, respectively. In the study conducted by Günel Karadeniz et al. (2), this rate is was 42.04%. Although it is not considered an error by some readers to give the p-value in closed form, it prevents the actual information obtained from statistical testing from being reached. For example, it is error to state p>0.05, instead of p=0.058. At the same time, actual values of the p-value are needed in studies such as meta-analysis (22). One of the most common errors encountered when examining p-value errors is was that the p-values are were not correct with a ratio of 34.6%. When analyzed in other studies in the literature; this rate was 13.36% in the study of, Ercan et al. (21) in (2015 and it was 8.82% in the study of Ercan et al. (7) in2017.) this rate is 13.36%, 8.82% and Ercan et al. (21) reported this rate as 17.83%. The authors emphasized that this rate is was obtained only from articles where p-value can could be controlled, so this ratio may might actually be higher.

When the errors for statistical tests are examined, the most common error is the use of an undefined statistical test. This rate was obtained as 44.2% in our study. The least mistake made in this classification is was not to apply the statistical test defined in the method section with a rate of 18.7% in our study. This rate was found to be 11.52% in the study by Ercan et al. (21) In the studies in which Hanif and Ajmal (23) evaluated 80 research articles published in local journals indexed and recognized in Pakistan, the rate was 26.25%. In our study, we found that the name of the statistical test was incorrectly given in 25.2% of medicine residency theses. While this rate was 3.23% in medical journals in the study of Ercan et al. (21), This this rate was 12.50% in the study of Hanif and Ajmal (23) and this rate was 12.66% in the studies of Günel Karadeniz et al. (2). Another important error related to statistical tests is the use of the incorrect statistical test. While this rate was 28.3% in medicine residency theses, theseis rates wereas 7.83% and 10.78%, respectively, in the studies of Ercan et al. (7,21) in 2015 and-2017. In the study of Hanif and Ajmal (23) study, this rate was 28.75% while in the study of Günel Karadeniz et al. (2), it was 6.33%. In some cases, clinicians evaluate the situation subjectively in their studies and make comments without statistical testing. However, a scientific result cannot be achieved without statistical testing. While the rate of this kind of error rate was 21.8% in medicine residency theses, when Ercan et al. (7,21) reported this rate as 17.51% in their studyies in 2015 and 1.96% in their studyies in 2017, and Günel Karadeniz et al.(2) reported this rate as 8.86%.

In the 321 medicine residency thesis theses examined for the errors classified as “Other Errors”, it was seen that the most common error was the “errors made in summarizing the data” with a rate of 70.1%. For example, regardless of the distribution of numerical data, it was observed that mean, standard deviation or median, minimum and maximum values were given in theses. The rate of “Errors in summarizing data” wasere 26.73% in their study of medicine articles inin the study of Ercan et al. (7) and it was 57.84% in their studythe study of Ercan et al. (21). In the study of Hanif and Ajmal (23), it was 16.25%, while in the study of Günel Karadeniz et al. (2), it was 64.56%.

Another error addressed is notation errors. Notation errors were classified as mathematical notation errors and statistical symbol errors. In medicine residency theses, these rates were 51.4% and 19.6%, respectively. In similar studies in the literature, Ercan et al. (7) reported 6.91% and 3.23%, respectively, while Ercan et al. (21) reported 2.94% and 3.43%, respectively, and Günel Karadeniz et al. (2)reported 16.46% and 3.80%, respectively. Errors in notation are known to mislead the reader in scientific studies and lead to a misunderstanding of the results.

In some studies, it was observed that the researchers used statistical expressions that were not understood in a manner contrary to statistical language and that there were errors in statistical terminology. In medicine residency theses, these rates was 14.3% and 28.7%, respectively. Accordingly, it was observed that comments that were not in accordance with statistical language were contradictory in the interpretation of results in scientific research. In our study, this rate was 23.1%. In the study of Ercan et al. (7) study, the rate of errors of incomprehensible statistical terms was 4.15%, while in their 2017 study, this rate was 0.49%. Likewise, the rate of inappropriate interpretation errors was reported as 8.76% in the study of Ercan et al. (21) in 2015 and 14.71% in the study of Ercan et al. (7) in 2017. In the study of Günel Karadeniz et al. (2), the rate of errors of incomprehensible statistical terms errors was 22.78%, while the rate of errors of inappropriate interpretation errors was of the statistical analysis in the discussion section. The rate of this error was 42.1% in our study. When similar studies were examined, this rate was 6.91% and 10.13% (7,21).

Study Limitations

According to the results obtained from our study, they we showed that the researchers were insufficient in statistics or did not show the necessary care in this regard. Statistical errors can affect the results either directly or indirectly. Considering the examined theses, it is was concluded that the rate of errors both directly and not directly affecting the result is was not very low. The reasons for these errors can be considered as, not consulting a biostatistics expert on the subject, assuming that they are sufficient in statistics, but not having sufficient knowledge, and carelessness (1,21). In the evaluation process of medicine residency theses, it would be more useful to have statistical evaluation earlier than other expert evaluations. Because, an error found as a result of a statistical evaluation that needs to be corrected can affect the results and therefore the discussion of the study. This will lead to a waste of time and extend the research process.


Conclusion

As a result, both researchers and consultants who undertake scientific studies have a responsibility to minimize these errors. To prevent statistical errors, students who are doing residency in medicine are required to receive the necessary training in statistical literacy, to have basic statistical knowledge, and to receive consultancy from a biostatistics expert for statistical evaluations.Students who residency in medicine in preventing statistical errors are required to receive the necessary training in statistical literacy, to have basic statistical knowledge, and to receive consultancy from a biostatistics expert for statistical evaluations. The thesis advisor should be more sensitive about statistics during the thesis evaluation process and ensure that medical students receive consultancy from a biostatistics expert during the execution of their thesis.

Ethics

Ethics Committee Approval: The theses in our article have been examined by scanning the Thesis Center of the Council of Higher Education (YÖK), and an Ethics Committee approval is not required as there is no interventional or clinical situation.

Informed Consent: Patient consent information is also not required.

Peer-review: Externally peer reviewed.

Authorship Contributions 

Data Collection or Processing: A.Y.T., F.E.C., F.K.K., R.A., İ.E., Analysis or Interpretation: A.Y.T., F.E.C., F.K.K., R.A., İ.E., Literature Search: A.Y.T., F.E.C., F.K.K., R.A., İ.E., Writing: A.Y.T.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study received no financial support.


Images

  1. Ercan I, Demirtas H. Statistical errors in medical publication. Biom Biostat Int J 2015;2:19-21.
  2. Günel Karadeniz P, Uzabacı E, Atış Kuyuk S, Kaskır Kesin F, Can FE, Seçil M, et al. Statistical errors in articles published in radiology journals. Diagn Interv Radiol 2019;25:102-8.
  3. Ercan I, Yazici B, Yang Y, Ozkaya G, Cangur S, Ediz B, et al. Misusage of statistics in medical research. Eur J Gen Med 2007;4:128-34.
  4. Schor S, Karten I. Statistical evaluation of medical journal manuscripts. JAMA 1966;195:1123-8.
  5. Altman DG. Statistical reviewing for medical journals. Stat Med 1998;17:2661-74.
  6. Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal. JAMA 1990;263:1355-7.
  7. Ercan I, Kaya MO, Uzabaci E, Mankir S, Can FE, Albishir MB. Examination of published articles with respect to statistical errors in veterinary sciences. Acta Veterinaria-Beograd 2017;67:33-42.
  8. Gore SM, Jones IG, Rytter EC. Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976. Br Med J 1977;1:85-7.
  9. MacArthur  RD,  Jackson  GG.  An  evaluation  of  the  use  of  statistical methodology in  the Journal  of  Infectious  Diseases.  J  Infect  Dis 1984;149:349-54.
  10. Ercan I, Ocakoglu G, Sigirli D, Özkaya G. Assessment of Submitted Manuscripts in Medical Sciences According to Statistical Errors.Tıp Bilimlerinde Gönderilen Çalismalarin Istatistiksel Hatalara Göre Degerlendirilmesi. Türkiye Klinikleri J Med Sci 2012;32:1381-7.
  11. Feinstein AR. Clinical biostatistics. XXV. A survey of the statistical procedures in general medical journals. Clin Pharmacol Ther 1974;15:97-107.
  12. Welch GE 2nd, Gabbe SG. Review of statistics usage in the American Journal of Obstetrics and Gynecology. Am J Obstet Gynecol 1996;175:1138-41.
  13. Gardenier J, Resnik DB. The misuse of statistics: concepts, tools, and a research agenda. Account Res 2002;9:65-74.
  14. Altman DG. Statistics and ethics in medical research. Misuse of statistics is unethical. Br Med J 1980;281:1182-4.
  15. Glantz SA. Biostatistics: how to detect, correct and prevent errors in the medical literature. Circulation 1980;61:1-7.
  16. Bhattacharyya T, Bhattacharjee A, Balasubramanian S. Bridging the gap between biostatisticians and oncologists: Need of the hour in comprehensive cancer research. Indian J Cancer 2015;52:561-2.
  17. Nieminen P, Virtanen JI, Vähänikkilä H. An instrument to assess the statistical intensity of medical research papers. PloS One 2017;12:e0186882.
  18. McGuigan SM. The use of statistics in the British Journal of Psychiatry. Br J Psychiatry 1995;167:683-8.
  19. Lukić IK, Marusić M. Appointment of statistical editor and quality of statistics in a small medical journal. Croat Med J 2001;42:500-3.
  20. Šimundić A-M, Nikolac N. Statistical errors in manuscripts submitted to Biochemia Medica journal. Biochemia medica: Biochemia Medica 2009;19:294-300.
  21. Ercan I, Karadeniz PG, Cangur S, Ozkaya G, Demirtas H. Examining of published articles with respect to statistical errors in medical sciences. International Journal of Hematology and Oncology 2015;25:130-8.
  22. Ercan I. p-değeri açık mı yoksa kapalı mı yazılmalı? J Pediatr Inf 2010;4:47.
  23. Hanif A, Ajmal T. Statistical er rs in medical journals (A critical appraisal). Annals of King Edward Medical University 2011;17:178.