Skip to main content

Table 1 Reproducibility-related characteristics. Variable numbers (N) are dependent upon study design. Full detailed protocol pertaining to our measured variables is available online (https://osf.io/x24n3/)

From: Reproducible and transparent research practices in published neurology research

Indicators of reproducibility included in present studySignificance of measure variable for transparency and reproducibility
Publications
 All (N = 400)Publication accessibility (Is the publication open access to the general public or accessible through a paywall?)The general public’s ability to access scientific research may increase transparency of results and improve the ability for others to critically assess studies, potentially resulting in more replication studies
Funding
 Included studies (N = 389)Funding statement (Does the publication state their funding sources?)Explicitly providing source of funding may help mitigate bias and potential conflicts of interest
Conflict of interest
 Included studies (N = 271)Conflict of interest statement (Does the publication state whether or not the authors had a conflict of interest?)Explicitly providing conflicts of interest may allow for full disclosure of factors that may promote bias in the study design or outcomes
Publication citations
 Empirical studiesa (N = 271)Citations by a systematic review/meta-analysis (Has the publication been cited by any type of data synthesis publication, and if so, was it explicitly excluded?)Systematic reviews and meta-analyses evaluate and compare existing literature to assess for patterns, strengths, and weaknesses of studies regarding a particular field or topic
Analysis scripts
 Empirical studiesb (N = 271)Availability statement (Does the publication state whether or not the analysis scripts are available?)Providing access to the analysis script helps improve credibility by providing the replicators the opportunity to analyze raw data with the same analysis procedure
Method of availability (Ex: Are the analysis scripts available upon request or in a supplement?)
Accessibility (Can you view, download, or otherwise access the analysis scripts?)
Materials
 Empirical studiesc (N = 255)Availability statement (Does the publication state whether or not the materials are available?)Providing the materials list allows replicators to reproduce study using the same materials, promoting
Method of availability (Ex: Are the materials available upon request or in a supplement?)
Accessibility (Can you view, download, or otherwise access the materials?)
Pre-registration
 Empirical studiesb (N = 271)Availability statement (Does the publication state whether or not it was pre-registered?)Pre-registering studies may help mitigate potential bias and increase the overall validity and reliability of a study
Method of availability (Where was the publication pre-registered?)
Accessibility (Can you view or otherwise access the registration?)
Components (What components of the publication were pre-registered?)
Protocols
 Empirical studiesb (N = 271)Availability statement (Does the publication state whether or not a protocol is available?)Providing replicators access to protocols allows for a more accurate replication of the study, promoting credibility
Components (What components are available in the protocol?)
Raw data
 Empirical studiesb (N = 271)Availability statement (Does the publication state whether or not the raw data are available?)Providing replicators with access to raw data can help reduce potential bias and increase validity and reliability
Method of availability (Ex: Are the raw data available upon request or in a supplement?)
Accessibility (Can you view, download, or otherwise access the raw data?)
Components (Are all the necessary raw data to reproduce the study available?)
Clarity (Are the raw data documented clearly?)
  1. a“Empirical studies” are publications that include empirical data such as clinical trial, cohort, case series, case reports, case-control, secondary analysis, chart review, commentaries (with data analysis), laboratory, surveys, and cross-sectional designs
  2. bEmpirical studies determined to be case reports or case series were excluded in regard to reproducibility related questions (materials, data, protocol, and registration were excluded) as recommended by Wallach et al.
  3. cEmpirical studies determined to be either case reports, case series, commentaries with analysis, meta-analysis, or systematic review were excluded as they did not provide materials to fit the category