TY - JOUR
T1 - NutriGrade: A scoring system to assess and judge the meta-evidence of randomized controlled trials and cohort studies in nutrition research
AU - Schwingshackl, Lukas
AU - Knüppel, Sven
AU - Schwedhelm, Carolina
AU - Hoffmann, Georg
AU - Missbach, Benjamin
AU - Stelmach-Mardas, Marta
AU - Dietrich, Stefan
AU - Eichelmann, Fabian
AU - Kontopantelis, Evangelos
AU - Iqbal, Khalid
AU - Aleksandrova, Krasimira
AU - Lorkowski, Stefan
AU - Leitzmann, Michael F.
AU - Kroke, Anja
AU - Boeing, Heiner
PY - 2016/11/1
Y1 - 2016/11/1
N2 - The objective of this study was to develop a scoring system (NutriGrade) to evaluate the quality of evidence of randomized controlled trial (RCT) and cohort study meta-analyses in nutrition research, building upon previous tools and expert recommendations. NutriGrade aims to assess the meta-evidence of an association or effect between different nutrition factors and outcomes, taking into account nutrition research-specific requirements not considered by other tools. In a pretest study, 6 randomly selected meta-analyses investigating diet-disease relations were evaluated with NutriGrade by 5 independent raters. After revision, NutriGrade was applied by the same raters to 30 randomly selected meta-analyses in the same thematic area. The reliability of ratings of NutriGrade items was calculated with the use of a multirater k, and reliability of the total (summed scores) was calculated with the use of intraclass correlation coefficients (ICCs). The following categories for meta-evidence evaluation were established: high (8-10), moderate (6-7.99), low (4-5.99), and very low (0-3.99). The NutriGrade scoring system (maximum of 10 points) comprises the following items: 1) risk of bias, study quality, and study limitations, 2) precision, 3) heterogeneity, 4) directness, 5) publication bias, 6) funding bias, 7) study design, 8) effect size, and 9) dose-response. The NutriGrade score varied between 2.9 (very lowmeta-evidence) and 8.8 (high meta-evidence) for meta-analyses of RCTs, and it ranged between 3.1 and 8.8 for meta-analyses of cohort studies. The k value of the ratings for each scoring item varied from 0.32 (95% CI: 0.22, 0.42) for risk of bias for cohort studies and 0.95 (95% CI: 0.91, 0.99) for study design, with a mean k of 0.66 (95% CI: 0.53, 0.79). The ICC of the total score was 0.81 (95% CI: 0.69, 0.90). The NutriGrade scoring system showed good agreement and reliability. The initial findings regarding the performance of this newly established scoring system need further evaluation in independent analyses.
AB - The objective of this study was to develop a scoring system (NutriGrade) to evaluate the quality of evidence of randomized controlled trial (RCT) and cohort study meta-analyses in nutrition research, building upon previous tools and expert recommendations. NutriGrade aims to assess the meta-evidence of an association or effect between different nutrition factors and outcomes, taking into account nutrition research-specific requirements not considered by other tools. In a pretest study, 6 randomly selected meta-analyses investigating diet-disease relations were evaluated with NutriGrade by 5 independent raters. After revision, NutriGrade was applied by the same raters to 30 randomly selected meta-analyses in the same thematic area. The reliability of ratings of NutriGrade items was calculated with the use of a multirater k, and reliability of the total (summed scores) was calculated with the use of intraclass correlation coefficients (ICCs). The following categories for meta-evidence evaluation were established: high (8-10), moderate (6-7.99), low (4-5.99), and very low (0-3.99). The NutriGrade scoring system (maximum of 10 points) comprises the following items: 1) risk of bias, study quality, and study limitations, 2) precision, 3) heterogeneity, 4) directness, 5) publication bias, 6) funding bias, 7) study design, 8) effect size, and 9) dose-response. The NutriGrade score varied between 2.9 (very lowmeta-evidence) and 8.8 (high meta-evidence) for meta-analyses of RCTs, and it ranged between 3.1 and 8.8 for meta-analyses of cohort studies. The k value of the ratings for each scoring item varied from 0.32 (95% CI: 0.22, 0.42) for risk of bias for cohort studies and 0.95 (95% CI: 0.91, 0.99) for study design, with a mean k of 0.66 (95% CI: 0.53, 0.79). The ICC of the total score was 0.81 (95% CI: 0.69, 0.90). The NutriGrade scoring system showed good agreement and reliability. The initial findings regarding the performance of this newly established scoring system need further evaluation in independent analyses.
KW - Meta-analysis
KW - Meta-evidence
KW - Nutrition
KW - Reliability
KW - Scoring
UR - http://www.scopus.com/inward/record.url?scp=84997142690&partnerID=8YFLogxK
U2 - 10.3945/an.116.013052
DO - 10.3945/an.116.013052
M3 - Article
AN - SCOPUS:84997142690
SN - 2161-8313
VL - 7
SP - 994
EP - 1004
JO - Advances in Nutrition
JF - Advances in Nutrition
IS - 6
ER -