Differences in the reporting of conflicts of interest and sponsorships in systematic reviews with meta-analyses in dentistry: an examination of factors associated with their reporting

Abstract

Background

Reporting conflicts of interest (COI) and sources of sponsorship are of paramount importance in adequately interpreting the results of systematic reviews. Some evidence suggests that there is an influence of COI and sponsorship on the study results.

The objectives of this meta-research study were twofold: (a) to assess the reporting of COI and sponsorship statements in systematic reviews published in dentistry in three sources (abstract, journal’s website and article’s full text) and (b) to assess the associations between the characteristics of the systematic reviews and reporting of COI.

Methods

We searched the PubMed database for dental systematic reviews published from database inception to June 2023. We assessed how COI and sponsorship statements were reported in the three sources. We performed a logistic regression analysis to assess the associations between the characteristics of the systematic reviews and the reporting of COI.

Results

We assessed 924 abstracts published in PubMed and on the corresponding journals´ websites. Similarly, full texts associated with the 924 abstracts were also assessed. A total of 639 (69%) and 795 (88%) studies had no statement of COI in the abstracts on PubMed and the journal’s website, respectively. In contrast, a COI statement was reported in 801 (87%) full texts. Sponsorship statements were not reported in 911 (99%) and 847 (93%) abstracts published in PubMed and a journal´s website, respectively. Nearly two-thirds of the full-text articles (N = 607) included sponsorship statements. Journal access was significantly associated with COI statement reporting in all three sources. Open-access journals have significantly higher odds to report COI in PubMed and full-texts, while have significantly lower odds to report COI in the websites, compared with subscription or hybrid journals. Abstract type was significantly associated with COI statement reporting on the journal’s website and in the full text. Review registration based on the full text and the number of authors were significantly associated with COI statement reporting in PubMed and in the full texts. Several other variables were found to be significantly associated with COI statement reporting in one of the three sources.

Conclusions

COI and sponsorship statements seem to be underreported in the abstracts and homepage of the journals, compared to the full-texts. These results were particularly more pronounced in abstracts published in both the PubMed database and the journals’ websites. Several characteristics of systematic reviews were associated with COI statement reporting.

Introduction

Systematic reviews are an important source of information to answer clinical questions [1], inform the prevalence of a specific disease or condition [2] and also serve as a basis for the development of clinical guidelines [3].

An important part of an systematic review is the abstract that summarises the most important information reported in the full text. Some published data suggest that many clinicians may access only the abstract of a scientific article, mainly due to a lack of time for reading due to their various duties [4]. Hence, an abstract should contain all the information that could be important for the reader to interpret potential biases that could interfere with the study results. In the biomedical field, abstracts are usually published in major databases, such as PubMed, and on the websites of the journals publishing the article.

Potential financial conflicts of interest (COI) may be a source of bias in studies. For example, financial COI was reported to be associated with more positive results in articles published in two major medical journals [5]; in the review, after controlling for sample size, study design and country of primary authors, studies with COIs had 2.35 (95% confidence interval [CI] 1.08–5.09) times higher odds of reporting positive results than those without COIs among all treatment studies from The New English Journal of Medicine and The Journal of the America Medical Association. Evidence also suggests that there is a positive association between COI and the report of positive results in dental randomised clinical trials (RCTs) [6]; in the review, the RCTs with COIs had 2.40–9.19 times higher odds of reporting positive results depending on the definition of COI. There has also been a reported association between financial and non-financial COIs and favourable recommendations in other types of scientific publications, such as clinical guidelines, opinion articles and narrative reviews [7].

Sponsorship has also been associated with more favourable efficacy results and conclusions in trials of devices and drugs that are sponsored by their manufacturers [8]; in this systematic review of 25 papers, studies that were sponsored by industry had a 1.27 times higher risk of reporting favourable efficacy results. Reporting sponsorship should also be a requirement for abstracts in order to allow an adequate interpretation of findings. There is a substantial amount of literature assessing the impact of COIs/sponsorship on systematic reviews [9,10,11,12,13,14].

Therefore, the objectives of this study were twofold: (a) to assess how COI and sponsorship statements are reported in systematic reviews in three sources (the abstract, journal’s website and article full text) and (b) to assess the associations between the characteristics of the systematic reviews and reporting of COI.

Materials and methods

Eligibility criteria

In the current research, we included systematic reviews with meta-analyses in the dental field that were published in English and included in PubMed. All types of systematic reviews with meta-analyses were included, and there was no restriction on species. We included only reviews with meta-analyses because we hypothesised that estimates from meta-analyses might be more relevant in the decision-making process. Therefore, one can argue that these meta-analyses are more sensitive to COIs and sponsorship. Systematic reviews without meta-analyses, published in languages other than English or outside of the dental field were excluded.

Search strategy, data selection and rationale

On 16 June 2023, we searched for dental systematic reviews published in the PubMed database using a pre-defined search strategy (see supplementary file). The search included articles published from database inception to June 2023. Duplicates were removed, and the articles that remained had their abstracts assessed for the reporting of COI.

The abstracts available in PubMed were then assessed to identify the reporting of statements on COI. We chose PubMed to check the information on COIs for two main reasons: (1) the abstracts in PubMed are widely and freely available to anyone, and (2) PubMed is the most well-known biomedical database; many clinicians and interested readers likely use it as a reference to assess abstracts of indexed articles. After checking the abstracts reported in PubMed and the journals’ websites, we scrutinised the full text associated with each abstract for information on included statements reporting potential COIs. The research procedures in the use of PubMed, the journals’ websites and full text articles were conducted independently and in duplicate by two authors (JH, CMF) with a sample of 10% of the included abstracts/articles until an agreement of at least 80% was reached among the assessors. One assessor (JH) then conducted the procedures with the remaining sample (90%).

Data extraction

The following data were directly extracted into a standardised Excel form: (1) reporting of COI statement (YES/NO); (2) reporting of sponsorship (YES/NO); (3) type of COI reported (financial or non-financial); (4) type of systematic review (interventional or non-interventional); (5) type of primary studies in the systematic review (in-vitro or animals, humans or both); (6) journal type (dentistry or other); (7) systematic review registration (YES/NO); (8) continent of origin (North America, South America, Europe, Asia, Africa or Oceania); (9) country of origin (developing and developed); (10) abstract type (structured or non-structured); (11) journal impact factor (IF) (JCR 2022); (12) number of citations in Google Scholar; (13) journal access (subscription/hybrid or open access); (14) journal reporting a COI policy in the instructions to authors (YES/NO); (15) journal reporting the type of COI in the instructions to authors (financial, non-financial, both, only say that authors need to report COI, or no information); and (16) number of authors. Two assessors (JH, CMF) independently extracted a sample of articles until they reached at least 80% agreement [15]. One assessor (JH) then extracted the remaining data.

Statistical analysis

Descriptive statistics were used to summarise the characteristics of the included systematic reviews. Proportions were used for categorical variables, and median and interquartile range (IQR) were used for continuous variables because all the three continuous variables (i.e. journal impact factor, number of citations, and number of authors) were not normally distributed based on Kolmogorov–Smirnov tests. To compare the prevalence of the reporting of COI and sponsorship among the three sources (PubMed, website and full text), Cochran’s Q tests were used. To assess the associations between the characteristics of the systematic reviews (independent variables) and the reporting of COI on the three sources separately, binary logistic regression analysis was performed. First, univariate binary logistic regression analysis was performed to assess the association of each independent variable with the reporting of COI separately. Second, the multicollinearity of the independent variables that were significant in the univariate analyses (P < 0.05) were tested using the variance inflation factor (VIF) before they were included in the subsequent multivariate binary logistic regression analysis. When a VIF value of a variable was > 5, collinearity was considered present, and the variable was excluded from the following analysis [16]. Third, a multivariate binary logistic regression analysis with backward selection was performed to further assess the association between the independent variables and the reporting of COIs.

Results

Selection of abstracts/full-text

A total of 969 abstracts were initially retrieved from the PubMed database. After the title and abstract assessment, 42 abstracts (4%) were excluded because they did not meet the eligibility criteria. After the full-text analysis, three other articles were excluded. Therefore, 924 abstracts with their respective full texts were finally included. Figure 1 reports the selection of the abstracts/articles.

Fig. 1

Flow of the selection process

Descriptive results

The most prevalent country of affiliation of the first author was Brazil (n = 185, 20%), followed by China (n = 88, 10%) and the USA (n = 86, 9%). Most of the studies (n = 524, 57% of the whole sample) were published between 2020 and 2023. A large range of dentistry-associated journals, for example, Clinical Oral Investigations (n = 44, 5%), BMC Oral Health (n = 22, 2%), Journal of Evidence-Based Dental Practice (n = 10, 1%) and others (in total: n = 762; 82%), were identified. About two-thirds (n = 622, 67%) of the journals were either accessible via subscription or had a hybrid system. Articles accessible via open access comprised 33% (n = 302) of the present sample. In addition, 758 of the 924 (82%) reviews analysed were based on interventional studies, and 816 (88%) reviews included studies on human subjects. The most observed dental discipline was implant dentistry, with a total of 352 (38%) articles. The journal impact factor ranged from 0.2 to 18.6 (median = 3.4) and the median of number of citations of the included articles was 22.0. A total of 849 (92%) reviews have been published in a journal with an impact factor.

The characteristics of the assessed abstracts/articles are reported in detail in Table 1.

Table 1 Characteristics of the systematic reviews included

Reporting of COI statement and sponsorship

A high percentage of the abstracts (PubMed = 69%, journals’ websites = 88%) did not include a statement about COIs. When a COI statement was reported, “no COI” was the most frequently mentioned COI type in all three examined sources (PubMed: n = 283, 99%; journals’ websites: n = 109, 97%; full text: n = 780, 97%). In the full texts (n = 13, 2%), the second most declared COI type after “no COI” mentioned by at least one of the authors of the respective review was a financial COI. In contrast, a statement regarding COI was frequently observed in the full texts (n = 801, 87%). Overall, the prevalence of COI statement reporting differed significantly among the three sources (P < 0.01) (Table 2). Similarities were observed regarding sponsorship statements in the abstracts, where an even higher proportion of no sponsorship statement was observed (PubMed = 99%, journals’ websites = 93%). In comparison, nearly two-thirds of the full-text articles included a sponsorship statement (66%). The prevalence of sponsorship statement reporting in the full texts was significantly higher than that in the PubMed abstracts (P < 0.01) and journal website abstracts (P < 0.01). However, no significant difference was found in sponsorship statement reporting between the abstracts on PubMed and the journals’ websites (P = 0.05) (Table 3).

Table 2 Characteristics of the independent variables based on COI reporting at different sources

Table 3 Comparison of COI reporting between different sources

Regression analyses

Table 4 shows the association between the different characteristics of the reviews and the reporting of COI in the PubMed abstracts. Based on the univariate analysis, the COI reporting in PubMed was significantly associated with the type of primary studies based on the full text (P < 0.01), review registration based on full text (P < 0.01), continents of origin of the reviews (P < 0.01), abstract type (P < 0.01), review access based on website (P < 0.01), review access based on internet (P < 0.01), journal type (P < 0.01), journal access (P < 0.01), journal reporting a COI policy in the instructions to authors (P = 0.02), number of citations (P < 0.01) and number of authors (P < 0.01). The VIF values of all the significant variables were < 5, and therefore, those variables were all included in the subsequent multivariate analysis. The direction of the association of the significant variables in the univariate analyses are reported in Supplementary files.

Table 4 Univariate and multivariate binary logistic regression analysis for the reporting COI at PubMed

Based on the multivariate analysis with backward selection, the type of primary studies based on the full text (for humans, OR: 0.32; 95%CI: 0.16, 0.64; P < 0.01), review registration based on the full text (OR: 0.44; 95%CI: 0.29, 0.67; P < 0.01), review access based on website (OR: 0.33; 95%CI: 0.18, 0.62; P < 0.01), journal type (OR: 2.46; 95%CI: 1.48, 4.08; P < 0.01), journal access (OR: 13.87; 95%CI: 8.75, 21.97; P < 0.01) and number of authors (OR: 1.09; 95%CI: 1.02, 1.16; P = 0.02) remained statistically significant. The reviews with the primary studies in humans, no registration based on full-text, and no full access based on homepage have significantly less odds to report the COI at PubMed level than the reviews with the primary studies in-vitro and in animals, presence of registration based on full-text, and presence of full access based on homepage. In addition, the reviews published in non-dental journals, open access journals, and with bigger number of authors have significantly higher odds to report the COI at PubMed level than the reviews in dental journals, subscribed or hybrid journals, and with smaller number of authors.

Table 5 shows the associations between different characteristics of the reviews and the reporting of COI on the journals’ websites. Based on the univariate analysis, the COI reporting in the journals’ websites was significantly associated with abstract type (P < 0.01), review access based on website (P < 0.01) and journal access (P < 0.01). The VIF values of all the significant variables were < 5, and therefore, those variables were all included in the subsequent multivariate analysis. The direction of the association of the significant variables in the univariate analyses are reported in Supplementary files.

Table 5 Univariate and multivariate binary logistic regression analysis for the reporting COI at website

Based on the multivariate analysis with backward selection, abstract type (OR: 0.48; 95%CI: 0.28, 0.83; P < 0.01) and journal access (OR: 0.26; 95%CI: 0.14, 0.47; P < 0.01) remained statistically significant. Reviews with non-structured abstracts and published in open access journals have significantly less odds to report COI at home page level than the reviews with structured abstracts and published in subscribed or hybrid journals.

Table 6 shows the association between the different characteristics of the reviews and the reporting of COI in the full texts. Based on the univariate analysis, the COI reporting in the full texts was significantly associated with review registration based on full text (P < 0.01), continents of origin of the reviews (P = 0.01), abstract type (P = 0.01), review access based on website (P < 0.01), journal type (P = 0.02), journal access (P < 0.01), number of citations (P < 0.01), and number of authors (P < 0.01). The VIF values of all the significant variables were < 5, and therefore, those variables were all included in the subsequent multivariate analysis. The direction of the association of the significant variables in the univariate analyses are reported in Supplementary files.

Table 6 Univariate and multivariate binary logistic regression analysis for the reporting COI at full-text

Based on the multivariate analysis with backward selection, review registration was based on the full text (OR: 0.56; 95%CI: 0.36, 0.86, P < 0.01), continents of origin of the reviews (for South America, OR: 0.40; 95%CI: 0.18, 0.86; P = 0.02), abstract type (OR: 2.00; 95%CI: 1.19, 3.35; P < 0.01), journal access (OR: 3.10; 95%CI: 1.76, 5.45; P < 0.01) and number of authors (OR: 1.27; 95%CI: 1.12, 1.43; P < 0.01) remained statistically significant. Reviews with no registration based on full-text and performed in South America have significantly less odds to report the COI at full-text level than the reviews with presence of registration based on full-text and performed in North America. In addition, the reviews with non-structured abstracts, published in open access journals, and with bigger number of authors have significantly higher odds to report the COI at full-text level than the reviews with structured abstracts, published in subscribed or hybrid journals, and with smaller number of authors.

Discussion

Main findings

The present study found consistent disagreement regarding the reporting of COI in abstracts of dental systematic reviews between different sources (i.e. PubMed, website, and full-text). Furthermore, the information on COI was reported in more detail in the full text of the articles than in the abstracts published on PubMed and on the journals’ websites. More than one-third of the selected articles did not report any statement on sponsorship in their full text. Regression analyses demonstrated that the reporting of COI at PubMed is significantly associated with type of primary studies, review registration, review access, journal type, journal access, and number of authors. The reporting of COI at website is significantly associated with abstract type and journal access. The reporting of COI at full-text is significantly associated with review registration, continents of origin of the reviews, abstract type, journal access, and number of authors. Journal access was significantly associated with COI statement reporting in all three sources. Abstract type, review registration based on the full text and number of authors were significantly associated with COI statement reporting in two sources.

Interpretation of the findings

It appears that there was underreporting of statements on COI and sponsorship in both sources reporting abstracts: PubMed and the journal websites (Table 3). Furthermore, for systematic reviews that reported COI statements, the overwhelming majority (more than 96%) did not report any potential financial or non-financial COI (Table 3). The present findings are in agreement with a previous study by Faggion et al. [17] that assessed the reporting of COI and sponsorship in 1,000 articles published in dental journals. In a sample of 95 systematic reviews included in the sample of 1,000 articles, 4% reported a COI. However, the present findings are in disagreement with a study by Bou-Karroum et al. that assessed financial and non-financial COI in systematic reviews in health policy and systems research [18]. In that study, 15% of the systematic reviews had at least one author reporting at least one type of COI. However, our study reported more COI statements than it did (87% versus 80%, respectively) (Table 3). Another study by Kee et al. [19], published in the field of psoriasis, concluded that 82% of the systematic reviews assessed had at least one author with a COI. One possibility is that authors of systematic reviews published in dentistry may not report financial and non-financial COIs in detail.

There were also statistically significant differences in sponsorship reporting between the full texts and PubMed/journal websites, with the latter being underreported (Table 3). Research by Lundh et al. suggests that primary research studies on drugs and devices sponsored by their manufacturing companies have more positive results and conclusions than those studies supported by other sources [20]. Similarly, a study by Veroniki et al. suggested that industry-sponsored secondary research in the form of network meta-analyses seems to report more favourable conclusions than non–industry-sponsored network meta-analyses [14]. Nevertheless, even if readers were able to access the full text of the articles in this sample, they would not obtain any information about sponsorship in more than one-third of the articles. Therefore, improvements in reporting information on sponsorship are needed in the three different sources to allow an adequate interpretation of the findings by interested readers.

In the regression analysis, journal access was significantly associated with the COI reporting statement for all three sources. Reviews published in open-access journals were more likely to report COI statements in full texts and PubMed than reviews in subscription or hybrid journals (Tables 4 and 6). However, we also found that the reviews in open-access journals were less likely to report COI on the journals’ websites than reviews in subscription or hybrid journals (Table 5). We do not have any plausible explanations for these findings. In addition, we found that unregistered reviews were less likely to report COI in both the full text and PubMed than registered reviews (Tables 4 and 6). An article published by Stewart et al. suggests that prospective registration of the protocols can ensure transparency, robustness and accountability in the research process [21]. Systematic reviews are often committed to adhering to certain standards and guidelines set by the registration platform, and a statement of the reporting of COI is always required in many registration platforms (e.g. PROSPERO). Guidelines on the reporting of COI in registered protocols may also help/remind authors to include COI information in the abstracts and full texts of the review when they are written. This may be why registered reviews were more likely to report a COI statement. In addition, reviews with a larger number of authors were more likely to report a COI statement in both the full text and the PubMed abstract (Tables 4 and 6). A study by Wiehn et al. suggests that the number of authors was positively associated with the reporting quality of the publications [22]. This may be because a larger number of authors can lead to a more rigorous informal peer-review process prior to a manuscript’s submission, which may help enhance the overall quality of the reporting of a publication, including the reporting of a COI statement. We also found that the reviews with non-structured abstracts were less likely to report a COI statement on journals’ websites (Table 5) but more likely to report a COI statement in full texts (Table 6), compared with reviews with structured abstracts. However, the opposite conclusions are difficult to explain.

Limitations and strengths

Although the study included a large sample of systematic reviews published in dentistry, we cannot rule out some bias in the selection of systematic reviews [23]. We included only systematic reviews with meta-analyses; therefore, evidence from systematic reviews without meta-analyses was not included. Furthermore, we included only articles indexed in PubMed and some sort of publication bias should be considered. A strength of our study is the innovative approach to investigating COI reporting in different sources.

Conclusion

There was a lack of reporting of information about COI and sponsorship statements in three different sources (i.e. PubMed, full-texts and the journal’s website). Furthermore, there were statistically significant differences in COI and sponsorship reporting between the various sources. Because readers may not have access to all three sources, it is suggested that all sources report equivalent information on potential COIs and sponsorships in detail.