Table 1 Types and examples of spin evaluated in Patient Centered Outcomes Research Institute (PCORI) Draft Final Research Reports and publications
Category of spin | Strategy Used/Explanation, abstracted from Lazarus et al. and Chiu et al. [7, 8] | Examples of reviewers/editors’ comments |
|---|---|---|
1. Reporting bias | ccNot reporting adverse events or lack of focus on harms (e.g., no warning on important safety issues), • Selective reporting of outcomes favoring the beneficial effect of the experimental treatment (e.g., statistically significant results for efficacy outcomes or statistically non-significant results for harm outcomes), • Misleading reporting of study design • Use of linguistic spin or “hype” (i.e., rhetorical manipulations to convince the readers of the beneficial effect of the treatment such as “excellent” results, “encouraging” outcomes, “a trend toward significance”), • No consideration of limitations | • Your report, particularly the Discussion, emphasizes “positive” results. • To report there was no main effect on a particular measure or to imply changes between different time points on a particular measure is misleading given the analyses conducted. • the authors should recalibrate the tone of the report, especially the Discussion and Conclusions sections, so that they are more balanced. • However, this reviewer still questions your terminology describing this outcome as “robust.” |
2. Inappropriate interpretation | • Claiming a beneficial effect of the intervention despite statistically non-significant results, • Claiming an equivalent effect of the interventions for statistically non-significant results despite wide confidence interval • Claiming that the treatment is safe for statistically non-significant safety outcomes despite lack of power • Concluding a beneficial effect despite no comparison test performed • Interpretation of the results according to statistical significance (p-value) instead of clinical relevance | • The main concern here is whether the...score is statistically significant, and if it is, whether this small change is clinically meaningful • The effect, even when statistically significant, was quite modest clinically. • The results section would benefit from limiting statements to those that can be supported by the statistics conducted, avoiding speculation or general statements of effects, and minimizing the reporting of marginally significant findings. • Given the lack of a comparison group, evaluating effectiveness may not be possible. • There is too much self-selection bias to draw any meaningful conclusions from the results |
3. Attribution of causality | • Claiming a causal effect between the intervention being assessed and the outcome of interest despite a non-randomized design | • The study design does not include a control group, which should be discussed as a limitation. • The discussion implies that the...result may be a reflection of confounding. If you believe this to be the case, I suggest mentioning it in the abstract so as not to impress on the reader that this is a causal finding. |
4. Inappropriate extrapolation | • Extrapolation from the population, interventions or outcome actually assessed in the study to a larger population, different interventions or outcomes • Inadequate implications for clinical practice. | • More text on how the results, taken from a sample of convenience to academic medical centers in [elided], may not apply to other populations that face problems … • Furthermore, are these results, derived in a reasonably homogeneous population, generalizable to other underserved populations? Given the issues outlined in these comments, it would seem advisable to qualify some of your conclusions and to emphasize even more strongly that further data in a larger study are needed to say anything more conclusive. • The very low representation of people of color is troubling and should be discussed more at length in the results and discussion. |