Validity of Qualis database as a predictor of evidence hierarchy and risk of bias in randomized controlled trials: a case study in dentistry

Abstract

OBJECTIVE: To evaluate the validity of the Qualis database in identifying the levels of scientific evidence and the quality of randomized controlled trials indexed in the Lilacs database. METHODS: We selected 40 open-access journals and performed a page-by-page hand search, to identify published articles according to the type of study during a period of six years. Classification of studies was performed by independent reviewers assessed for their reliability. Randomized controlled trials were identified for separate evaluation of risk of bias using four dimensions: generation of allocation sequence, allocation concealment, blinding, and incomplete outcome data. The Qualis classification was considered to be the outcome variable. The statistical tests used included Kappa, Spearman's correlation, Kendall-tau and ordinal regressions. RESULTS: Studies with low levels of scientific evidence received similar Qualis classifications when compared to studies with high levels of evidence. In addition, randomized controlled trials with a high risk of bias for the generation of allocation sequences and allocation concealment were more likely to be published in journals with higher Qualis levels. DISCUSSION: The hierarchy level of the scientific evidence as classified by type of research design, as well as by the validity of studies according to the bias control level, was not correlated or associated with Qualis stratification. CONCLUSION: Qualis classifications for journals are not an approximate or indirect predictor of the validity of randomized controlled trials published in these journals and are therefore not a legitimate or appropriate indicator of the validity of randomized controlled trials.