Skip to main content
Download PDF

Publication and non-publication of clinical trials in PTSD: an overview

Research Integrity and Peer Review volume 4, Article number: 15 (2019) Cite this article

Abstract

Background

Although a large number of clinical trials on interventions demonstrating efficacy (or lack thereof) are conducted annually, much of this evidence is not accessible to scientists and clinicians.

Objectives

We aimed to determine the publication rate of posttraumatic stress disorder (PTSD) trials that have been registered in clinical trial registries, and the factors associated with publication.

Methods

Trials, completed on January 15, 2015, were identified via the US National Institutes of Health clinical trials registry, the European Union Clinical Trials Register and the WHO International Clinical Trials Registry Platform. A systematic search for publications (published by the end of March 2018) related to each of the registered trials were then performed.

Results

Four hundred and thirty-eight of 1982 potentially eligible trials were included. Only 34% of interventional trials were registered prior to initiation, 9% were registered within 2 months of starting and 20% after trial completion. Of the 438 included trials, 72% had generated peer-reviewed publications, while an additional 7% had disseminated results in some other form (such as on the trial database), 26 months after trial completion. Randomisation of a trial was the only factor individually associated with publication, in logistic regression analysis (p < 0.001). Intervention type, university as sponsor and study registration prior to completion were factors that influenced the time to publication, using Cox regression (p < 0.001).

Conclusions

This study underscores the importance of timely and accurate publication and dissemination of trial results, in order to avoid the potential waste of resources and to ensure research integrity and patient safety. We suggest that authors and journal editors adhere to conditions set out by the International Committee of Medical Journal Editors and that more diligent data sharing is encouraged through prospective trial registration and trial reporting websites.

Peer Review reports

Background

Although over US$100 billion is invested in medical research, each year reports suggest that less than 50% of funded research is ever published in full [1,2,3,4,5]. There are a number of ethical arguments proposed for why the results of all studies, particularly those involving human volunteers, should be published or reported [6,7,8,9]. Turner and colleagues (2013) note that these include the need to increase the returns on public investment (i.e. by the sharing of knowledge and practical advances), increase transparency, prevent research duplication (and wastage of time and resources), allow the public (including participants who have given of their time to the study) access to the research process and the results and inform patient and public decision-making (i.e. through sharing new knowledge and practical advances) [9].

Publication in a peer-reviewed journal is an important form of research dissemination. It offers a record of a study’s scientific contribution, provides evidence to clinicians to inform practice and supports policy makers in decision-making. The scientific community, however, has a poor track record of dissemination of clinical trial results [7, 10]. Poor publication rates may be attributed to a plethora of factors, with some authors struggling to publish data that may not meet the standards of journal editors and readers [7]. Publication bias, leading to lower publication rates for trials with negative or inconclusive results, and delayed publication times are other reasons why research is not published [11,12,13].

In order to avoid the potential damage that can result from incomplete knowledge and ensure the integrity of research, many organisations have established trial registries as another means to share data. ClinicalTrials.gov was launched on February 29, 2000, initially, with the main aim of establishing a registry of clinical trials investigating drugs for patients with serious or life-threatening diseases, regulated by the Food and Drug Administration (FDA). In 2004, the International Committee of Medical Journal Editors (ICMJE) released a statement that, as of July 2005, journals would require registration with a public trial registry, prior to enrolment, as a condition of publication [14]. In 2006, the World Health Organization (WHO) established standards and rules for the registration of studies in registries, and in 2007, launched the International Clinical Trials Registry Platform (ICTRP), which brings together data from all registries meeting their standards into a single searchable portal [15]. Researchers are currently required to register interventional trials while the registration of observational studies is optional [16]. The Declaration of Helsinki, since 2008, re-stipulates that research studies involving human subjects must be registered in a publicly accessible database before recruitment of the first subject and further states that researchers, authors, sponsors, editors and publishers all have ethical obligations with regard to the publication and dissemination of the results of their research [17]. Clinicaltrials.gov and the European Clinical Trials Database (EudraCT) have each included a section that permits investigators to submit their results, since 2008 and 2013, respectively.

Recent advances in reporting requirements include the Department of Health and Human Services (HHS) final rule, which clarifies and expands the requirements for reporting results information [18]. This was made publicly available on September 16, 2016. The NIH, concurrently, issued a complementary policy under which all NIH-funded clinical trials are expected to submit registration and results’ information, whether or not the trials are covered by the FDA Amendments Act requirements. This NIH Final Rule (42 CFR Part 11) came into effect on January 18, 2017, with compliance expected on April 18, 2017 [19]. Further, the American Psychological Association’s (APA’s) revised reporting standards for writing journal articles were released. This calls for registration of a study prior to its implementation and a summary of results on completion [20].

We examined the publication of all registered and completed posttraumatic stress disorder (PTSD)-related intervention trials in the ICTRP and in two major trial registries (ClinicalTrials.gov and EudraCT), in order to determine the rate of and time to publication and factors associated with publication. In order to be as inclusive as possible, we included trials where individuals with PTSD were included or where any PTSD-related outcomes (e.g. diagnosis, severity, symptomatology) were assessed. PTSD is a highly comorbid disorder and many of the drug and psychological interventions investigated in trials have trans-diagnostic application and utility.

In this paper, we address the following questions:

  1. (i)

    What proportion of PTSD-related intervention trials registered in ClinicalTrials.gov, EudraCT and ICTRP go on to be published and what are the characteristics of these studies?

  2. (ii)

    What is the time to publication and what factors pertinent to registration are associated with time to publication?

We hypothesised the following:

  1. (i)

    Proportion of and time to publication of PTSD-related studies would be similar to other conditions/disciplines and that individual factors related to study quality and characteristics would influence publication rates and times (we did not have pre-specified hypotheses as to what these individual factors would be).

  2. (ii)

    Publication rates of studies with positive (statistically significant) outcomes would be higher than those with null (not statistically significant) outcomes.

Outcomes were as follows:

  1. (i)

    Proportion of completed intervention trials that go on to be published.

  2. (ii)

    Length of time between study completion and publication.

  3. (iii)

    Factors that contributed to publication.

Methods

This was a cross-sectional overview of registered and subsequently published trials. Considering that this was an overview that entailed pooled analysis of secondary data, ethical approval was not sought. Two independent researchers (SS1 and LvdH) selected studies through each phase of selection. The responsible researchers agreed on the search terms to use and on study selection and exclusion criteria, as well as on data abstraction (selection of data items extracted from each contributing article) and data extraction. The search terms included all applicable synonyms for PTSD, were compiled in accordance with the instructions for each database and were tested prior to implementation. The exact search terms used were “PTSD OR posttraumatic stress disorder OR post-traumatic stress disorder OR posttraumatic stress disorder.” In each of the registries and search platform, we searched all fields and placed no limits on the searches. Our search terms were broad as we aimed to identify all possible PTSD-related studies. Studies were identified via Clinicaltrials.gov (www.clinicaltrials.gov), EudraCT (https://www.clinicaltrialsregister.eu/) and ICTRP (http://apps.who.int/trialsearch/Default.aspx). We extracted data on all studies and manually identified studies that were closed or completed. The ICTRP is a search platform containing trial registration data obtained from primary registries and partner registries internationally that meet the WHO clinical registry standards. We selected these three registries/platforms because they are the largest. Trials registered since the inception of each registry were downloaded on January 15, 2015; therefore, the date range for the NIH registry was on February 29, 2000, until January 15, 2015, for the EU register on May 1, 2004, until January 15, 2015, and for the WHO platform on May 1, 2007, until January 15, 2015. Inclusion criteria were predefined. Any PTSD-related study, with original quantitative data, was included. Qualitative studies, reviews, case reports and letters to the editor were excluded (although we did not expect to find these other study types registered on the trial registries as they do not clearly fall within the purpose of trial registries, we did). After ongoing or terminated trials, duplicates and studies not related to PTSD were removed, and there were 438 records to review. (Terminated trials were excluded as they are less likely to publish results, and our main aim was to evaluate the publication of completed studies.) See Fig. 1. We obtained independent peer review and approval of the search strategy and documentation by an information specialist who utilises the PRESS methodology.

Fig. 1
figure 1

Flow diagram of the selection process

Full size image

Information of interest pertaining to each trial was systematically extracted from the databases by two researchers (SS1 and LvdH) and reconciled. This included the following:

  1. 1)

    Database registration (which database/platform, when registered)

  2. 2)

    Study status (completed, terminated/withdrawn, active)

  3. 3)

    For completed studies: type of study (interventional, observational, study design); intervention type (i.e. behavioural/ psychological, such as cognitive behavioural therapy, narrative exposure therapy, motivational interviewing, mindfulness-based therapy; drug, device/procedure, such as transcranial direct current stimulation, photobiomodulation, biofeedback), study duration, number of arms, control conditions, location, numbers planned vs enrolled; participants (types, numbers in each arm); time between end of trial and publication; study sponsors as defined in the ICTRPFootnote 1 (none, pharmaceutical/industry, government, non-commercial, hospital, university); date and phase of study registered; study outcomeFootnote 2 (primary and secondary outcomes); changes to trial information, contact details of responsible party

  4. 4)

    Publication: in a journal or disseminated in another form

Five investigators then searched for peer-reviewed publications or any results disseminated for each individual study that was registered (LvdH and SS1 conducted the initial search and data extraction, which were verified by AS; IE and SW updated the results which were checked by SS1). For each registered trial, we searched via PubMed, Scopus, Google Scholar and Google using keywords listed (when available), trial titles, investigator names and trial registration codes to identify matching publications and/or results. Any report of a trial that had included results for PTSD as a primary or secondary outcome or as a covariate or had sampled individuals with PTSD was included. We considered a trial published if it appeared in a peer-reviewed journal (whether or not we could obtain the full article). We considered trial results disseminated if they appeared in a peer-reviewed journal, in a trial registry, in a report (e.g. media article, organisation website) or thesis. If publications were not found, investigators of these completed trials were e-mailed to ask for study-related publications and results (71 e-mail addresses were obtained via the registries and 96 via online searches; we were unable to obtain e-mail addresses for 27 investigators. Of the 167 investigators who were e-mailed, 70 responded). If the completion date of a trial was missing (5% of trials), we used the publication date as a proxy when this was available. This search was completed on March 21, 2018, which yielded a minimum of 26 months between trial completion (according to the completion date listed in the trial registries) and publication. We describe all studies in the results, but given the small number of observational studies listed and the fact that registration for these studies is voluntary (they are not required to be registered), we excluded them from further analysis.

Publications and trial data were independently reviewed by two investigators to determine whether (i) manuscripts or reported results were in fact related to the trial listed in the registry (as not all manuscripts/reports noted the trial registration number in the publication, it was not always clear, so a second reviewer double-checked this), (ii) PTSD-related findings were reported (e.g. some reports/papers reported on other study findings, but omitted to report findings related to PTSD—PTSD might have been a secondary outcome in some of these), (iii) PTSD-related outcomes were the primary or secondary outcomes measured, both or other (e.g. a condition or covariate) and (iv) whether primary and secondary findings were positive (statistically significant, p < 0.05), null (no statistically significant finding, p ≥ 0.05) or mixed (both statistically significant and no statistically significant findings in primary and/or secondary outcomes). Where results were reported in both a publication and another disseminated source and these differed, the publication results were used.

Statistical analysis

We present descriptive statistics for all studies split according to interventional and observational trials. As our main outcomes of interest were related to interventional trials, we examined univariate factors associated with the publication of interventional trials and entered significant variables into a logistic regression. The Kaplan-Meier method [21] was used to estimate the time between intervention trial completion and a paper related to that trial being published in a journal while the Breslow statistic [22] was used to determine significance. Significant covariates were then entered into a Cox Regression [23] to determine whether the time to publication was predicted by significant covariates. Analyses were conducted in SPSS, with two-tailed tests. p values of ≤ 0.05 were considered significant.

Results

Descriptive results of trials from the registry

All 438 trials were found on the ICTRP search platform, although 53 (12%) were not initially identified via ICTRP with the search terms used. Of these, 52 (12%) were identified in ClinicaTrials.gov only and 1 (0.2%) on EudraCT only. Furthermore, 322 (74%) trials were identified in both ICTRP and ClinicalTrials.gov, 2 (0.5%) in ICTRP and EudraCT and 61 (14%) in ICTRP only. In terms of registries of primary origin, 381 (87%) were registered in ClinicalTrial.gov; 21 (5%) on BioMed Central’s International Standard Registered Clinical/soCial sTudy Number (ISRCTN); 9 (2%) in the Australian New Zealand Clinical Trials Registry (ANZCTR); 8 (2%) in the Iranian Registry of Clinical Trials (IRCT), with 5 or less from Japan Primary Registries Network (JPRN) (1%); Chinese Clinical Trials Registry (ChiCTR) (< 1%); EudraCT (< 1%); German Clinical Trials register (< 1%); Netherlands Trial register (< 1%); and Clinical Trials Registry - India (CTRI) (< 1%).

The majority of included trials, 380 (87%) were interventional and 58 (13%) were observational in design. [See Table 1]. According to the data downloaded from the trial registries, PTSD was the primary condition in 91% of studies, main outcomes were PTSD related in 69% of trials, and secondary outcomes were PTSD related in 34% of trials. Around two thirds (66%) of studies were conducted in North America, followed by Europe (15%), Asia (13%), Africa (2%), Australia (2%) and South America (1%). Only 4.2% of studies were registered in low- and middle-income countries (based on the Word Bank categories).

Table 1 Descriptive data for interventional and observational trials listed in registries
Full size table

Blinding status varied significantly (p < 0.001) according to intervention type with drug trials more likely to be double-blind (73%) than open-label (19%). Psychological intervention trials were more likely to be open-label (43%) followed by single-blind (37%), and combined drug and psychological intervention trials were more likely to be double-blind (60%) than single-blind (28%).

Pharmaceutical companies and other industry accounted for the smallest sponsorship contribution (8%). Industry/pharmaceutical companies were more likely (p < 0.001) to be a sponsor of drug trials (79%), followed by sponsorship of trials of psychological interventions (17%) and combined psychological and drug interventions (3%). Universities were more likely (p = 0.001) to be sponsors of psychological intervention trials (74%), followed by drug (19%) and combined psychological and drug trials (6%). Hospitals were more likely (p = 0.024) to be a sponsor of psychological intervention trials (56%), followed by drug trials (24%), trials of procedures/devices (12%) and combined psychological and drug intervention trials (8%). Governmental organisations were also more likely (p = 0.034) to sponsor psychological intervention trials (71%), followed by drug (17%), combined drug and psychological intervention trials (7%) and studies of procedures/devices (5%). There was no significant relationship between non-commercial sponsorship and the type of intervention.

Entries into the database were changed over time, with evidence that information entered was updated or changed at least once in 82% of the studies. Registration rates increased after registration was made a requirement by the ICJME—37% of trials were registered prior to initiation in 2006 or later versus 20% in the preceding years (p = 0.04).

Published and disseminated result outcomes

Of the trials that were registered, 317 (74%) yielded traceable peer-reviewed publications, while another 32 (8%) studies had disseminated results in some other form (e.g. on the trial registry, in a dissertation or published report) (Table 2). Findings were obtained for 308 (81%) interventional trials. We were unable to obtain the remaining papers that had been disseminated, and thus, unable to extract the results.

Table 2 Publication data for (i) all studies and (ii) interventional trials
Full size table

Factors associated with publication of interventional trials

Whether a trial was randomised (p = 0.002), the type of intervention studied (p = 0.010), whether a pharmaceutical company/industry was involved in sponsoring the trial (p = 0.005) and whether a contact email address was provided in the registry (p = 0.021), were factors that were significantly associated with a trial being published in a journal (Table 3). These factors were subsequently entered into a logistic regression.

Table 3 Factors associated with publication of interventional trials
Full size table

The overall model was significant (χ2 (6) = 22.26, p < 0.001), with a Nagelkerke R2 of 0.085, and predicted 75% of publications correctly. The only variable that was individually associated with publication status was randomisation (p = 0.007). Having pharmaceutical/industry sponsorship (p = 0.073), provision of a contact email (p = 0.088) and intervention type were not significant (Table 4).

Table 4 Logistic regression of factors influencing publication
Full size table

Factors associated with time to publication of interventional trials

The Kaplan-Meier method [21] was used to estimate the time between trial completion and a trial-related paper being published (Table 5). The median time to journal publication was 27.0 months (95% CI 24.0; 30.0). Factors that influenced time to publication of interventional trials were whether a control condition was used (p = 0.021), whether the trial was randomised (p = 0.013), the intervention type (p = 0.004), whether a university was involved as a sponsor (p = 0.026), the number of treatment arms (p = 0.034) and whether the study was registered in a trial registry before it was completed (p = 0.040).

Table 5 Time to publication for interventional trials
Full size table

These significant factors were entered into a Cox regression (Table 6). The overall model was significant (χ2 (9) = 32.69, p < 0.001) with a 2 log-likelihood of 2271. Variables influencing publication time were intervention type—drug-only studies took significantly longer to publish than any other intervention study type; university sponsorship was associated with faster publication time; and trials that were registered in a database before being completed were also associated with faster publication times. Randomisation, number of treatment arms and having a control group were no longer statistically significant.

Table 6 Cox regression of factors involved in publication time
Full size table

Discussion

We conducted a review of trials related to PTSD in the ICTRP platform and two of the largest trial registry databases with a view to determining factors that might influence publication. Of the 380 registered interventional trials included, we found that at least 26 months after trial completion, the peer-reviewed journal publication rate was 74%. This increased to 82% when we included trials that had published results in some other form (e.g. in a trial registry or in a published report). This is better than or equivalent to the rate reported for other disease areas and is likely to further increase with time since trial completion. In one study on vaccine trials, the publication rate after 12 months was 12% and increased to 73% at 48 months post-study completion [24]. Another study of randomised clinical trials registered with ClinicalTrials.gov reported a publication rate of 71% at 60 months after study completion [25]. Time from the close of a study until publication was approximately 2 years and 3 months. This is in line with findings from other studies [26, 27], but may be longer given that when the completion date of a trial was missing, we used the publication date as a proxy.

It is concerning that even though registration prior to enrolment is required by the ICMJE, and has been included in the Declaration of Helsinki, only one third of interventional trials were registered prior to initiation, a further 9% within 2 months of starting and 20% after trial completion. Furthermore, despite the rate of registration prior to trial initiation having increased over time, the rate since 2006 is much lower than expected (over 60% were registered after trial initiation), when considering that the ICMJE instituted its requirement in 2005. A recent review of articles published in psychiatry journals which require prospective trial registration found that 34% were retrospectively registered and that only 33% of studies were correctly prospectively registered [28]. Another study found that only 54% of recently published antidepressant and cognitive behavioural therapy trials were registered and only 25% were properly registered [29]. Only 15% of RCTs published in clinical psychology journals were registered prospectively and even fewer (1%) were both prospectively and completely registered [30]. Similarly, the large number of trials with no documented difference in the number of participants planned for enrolment and the number actually enrolled is likely due to studies that were registered retrospectively. As clinical trial registries are used by researchers, clinicians and the public, it is important that they are accurate and up to date. Our findings also highlight the dearth of trials registered in low- and middle-income countries. Studies from these countries may be registered in trial databases not included in this review, but as the ICTRP only includes registries fulfilling their standards, these databases may not meet current requirements.

Trial randomisation was the only factor associated with peer-reviewed publication, when other factors were accounted for. Although not all prior studies have found that more rigorous trial designs (i.e. randomisation) are associated with publication [13], our results indicate that they could be. Although only six interventional trials had primary pharmaceutical/industry sponsorship and 29 had some industry involvement, these trials trended towards a lower likelihood of publication. Given that the industry provided sponsorship for 79% of drug-only trials, this finding suggests that sponsorship may influence publication with journals/industry setting stricter criteria for drug trials. Provision of a contact e-mail and intervention type was not significant when these other variables were accounted for. Whether results were positive or null was also not significantly associated with publication status. This finding, however, does need to be interpreted with caution as it is likely influenced by the number of unpublished trials for which we were unable to obtain any data.

With regard to the time to publication of interventional trials, trials of drug interventions alone were likely to be published later than those of drug and psychological interventions, psychological interventions alone and those assessing a procedure or device. Again, this may be because journals and/or authors have stricter publication requirements for drug trials. Nonetheless, this is concerning given the potentially serious and far-reaching implications of a lack of up-to-date evidence on the provision of effective pharmacotherapies to inform treatment guidelines. Trials with a university involved as a sponsor were likely to be published sooner than those without. This may be due to universities providing funding to their own academics, who are often rewarded based on their publication outputs. Trials registered in a trial database before trial completion were also likely to be published sooner than those registered after completion. This may indicate that authors are likely to consider and prepare their data for publication prior to trial completion and that those who register their trials before trial closeout intend to publish their data. Trial randomisation was no longer statistically significant when these other factors were accounted for.

Some of the authors we contacted provided reasons for not having published their work. These included results not worth publishing, small sample size, null findings and publication rejected by journals submitted to. Other reports have noted similar motivations for non-publication. Publication bias and lower publication rates for studies with negative or inconclusive results include the high standards of journal editors and the desire for positive results [7, 9, 11]. It has, however, been pointed out that the bias towards positive findings may come from the authors themselves, rather than journal editors and reviewers.

The strengths of this study are that it considered a large number of trials that encompassed a variety of research methodologies and were registered across a number of databases. To our knowledge, this is the first to do this in the field of psychiatry and PTSD. Despite our focus on PTSD-related trials, our results support the reports of publication bias in other fields [24, 29]. It is, however, important to recognise that there have been major developments in trial reporting regulations and guidelines in recent years (42 CFR Part 11, The Final Rule, APA’s revised JARS guidelines). Thus, trials conducted prior to these may be unrepresentative of current registration and reporting practices. Nonetheless, our findings support these new registration and reporting requirements and indicate the need for mechanisms (i.e. policies to support registration and reporting with penalties for investigators who do not follow standards for clinical research [31]) that ensure these regulations are adhered to, as we have demonstrated that since the ICMJE regulations many trials were still only registered after completion.

The following limitations should also be noted: (i) incorrect, inconsistent and incomplete information listed in the trial registries. For example, we found that information was not updated regularly, information in one section of the registry did not always align with others and trials that appeared on the list of “complete” studies were at times still listed as “active.” This may also imply that we may have missed trials that were in fact completed, but were not updated as such on the system. This has been a drawback since the inception of the registries [32]. There may also be some misclassification of hospital and university sponsors. We based this on the name of the sponsor as entered in the trial registration database. In some instances, university hospitals may be the primary sponsor but the university may, in fact, be listed as the sponsor. We performed a post hoc analysis to assess whether the effect of university sponsorship on time to publication was independent of hospital sponsorship by adding hospital sponsorship to the model and university sponsorship remained significantly associated with time to publication, whereas hospital sponsorship was not; (ii) although we made every effort to locate published reports—peer-reviewed and other, we may not have traced all, as publications were not always linked to trial registration numbers and not all study contacts listed responded; (iii) we included various study types, not only intervention trials in our initial analyses. Given that only intervention trials are required to be listed on these registries, we opted to exclude the observational studies from further analysis. This, however, constituted only a small percentage of studies; (iv) similarly, there were only a small number of trials sponsored by industry, which warrants care in the interpretation of the related findings; (v) we attempted to retrieve publications after a minimum of 26 months post-completion. It is possible that our publication rate would have increased had we used a longer post-completion timeframe; (vi) although the search strategies were the same across databases/search portals, not all the studies were identified in ICTRP. This may be related to differences in search algorithms; (vii) these results may not be generalizable to all trials. For example, previous studies suggest that the publication rate for registered trials is higher than for other trials [33]. Similarly, registered trials might differ from trials initially published as abstracts (e.g. Scherer et al., 2018) or trials identified through Internal Review Boards (e.g. Chan et al., 2004) [5, 34]. Furthermore, registered trials may differ systematically from unregistered trials, confounding interpretation of factors associated with journal publication. Drugs and devices, for instance, are FDA-regulated products and, as such, these trials are much more likely to be registered and also more likely to be reported in biomedical journals (which are often ICMJE members and have registration requirements). Trials of non-regulated products, on the other hand, are more likely to be reported in psychology and allied health journals (which are not ICMJE members and do not have registration requirements). This is illustrated by a study that reviewed the registration of clinical trials in clinical psychology journals, which found that most clinical trials in psychology journals (with the exception of registered trials in clinical psychology) were not registered [30].

In sum, 74% of trials were published in peer-reviewed journals and a further 8% had published results in some other form 26 months after trial completion; publication time from trial completion was approximately 2 years and 3 months; despite ICMJE guidelines, a little over one third of interventional trials were registered prior to initiation; and few trials were registered in low- and middle-income countries. Randomised trials were more likely to be published and those with a contact e-mail and no pharmaceutical sponsorship were not significant when other variables were accounted for; null findings were not associated with publication status. Trials sponsored by a university and those registered in a trial database before trial completion were likely to be published sooner than those registered after completion, while drug-only trials took longer to publish.

Conclusion

By not sharing knowledge gained from these trials, researchers waste valuable research funds, slow down advancement in the field, put patients and study participants at risk of potentially ineffective or even harmful interventions and distort the evidence base [11, 13]. In order to safeguard research integrity, we therefore support the call to encourage and make mandatory other methods of data sharing, for example via prospective trial registration and reporting websites such as those that we examined. Furthermore, given that the utility of information on registries is dependent on the quality of the information entered by investigators, the large amount of inconsistent or missing information we found on the databases indicates that greater vigilance is required to improve the reliability and accuracy of reporting. Recent initiatives in trial reporting regulations are a step towards this.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. Sponsor refers to the individual, organization, group or other legal entity which takes responsibility for initiating, managing and/or financing a study. The Primary Sponsor may or may not be the main funder.

  2. Outcome or outcome measure is defined as measurements observed or collected from those human subjects who are enrolled in a research trial.

Abbreviations

%:

Percentage

ANZCTR:

Australian New Zealand Clinical Trials Registry

APA:

American Psychiatric Association

ChiCTR:

Chinese Clinical Trials Registry

CTRI:

Clinical Trials Registry - India

EU:

European Union

EudraCT:

European Clinical Trials Database

FDA:

Food and Drug Administration

HHS:

Department of Health and Human Services

ICMJE:

International Committee of Medical Journal Editors

ICTRP:

International Clinical Trials Registry Platform

IQR:

Interquartile Range

IRCT:

Iranian Registry of Clinical Trials

ISRCTN:

International Standard Registered Clinical/soCial sTudy Number

JPRN:

Japan Primary Registries Network

Mdn:

Median

n :

Number

NGO:

Non-governmental Organisation

NIH:

National Institutes of Health

PTSD:

Posttraumatic stress disorder

WHO:

World Health Organization

References

  1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Obstet Gynecol [Internet]. 2009 Dec [cited 2017 May 19];114(6):1341–1345. Available from: http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00006250-200912000-00028.

  2. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

    Article  Google Scholar 

  3. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet [Internet]. 2014 [cited 2017 May 19];383(9913):267–276. Available from: http://www.sciencedirect.com/science/article/pii/S014067361362228X.

    Article  Google Scholar 

  4. Ross JS, Mocanu M, Lampropulos JF, Tse T, Krumholz HM. Time to publication among completed clinical trials. JAMA Intern Med; 2013 [cited 2017 May 19];173(9):825. Available from: http://archinte.jamanetwork.com/article.aspx?doi=10.1001/jamainternmed.2013.136.

    Article  Google Scholar 

  5. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. In: Scherer RW, editor. Cochrane Database of Systematic Reviews [Internet]. Chichester, UK: John Wiley & Sons, Ltd; 2007 [cited 2017 May 19]. Available from: http://doi.wiley.com/10.1002/14651858.MR000005.pub3.

  6. Chalmers I, Gray M, Sheldon T. Handling scientific fraud. Prospective registration of health care research would help. BMJ [Internet]. BMJ Publishing Group; 1995 [cited 2017 May 19];311(6999):262. Available from: http://www.ncbi.nlm.nih.gov/pubmed/7627070.

    Article  Google Scholar 

  7. Hudson KL, Collins FS. Sharing and reporting the results of clinical trials. JAMA; 2015 [cited 2017 May 19];313(4):355. Available from: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2014.10716.

  8. Savulescu J, Chalmers I, Blunt J. Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability. BMJ [Internet]. 1996 [cited 2017 May 19];313(7069). Available from: http://www.bmj.com/content/313/7069/1390?variant=full.

  9. Turner S, Wright D, Maeso R, Cook A, Milne R. Publication rate for funded studies from a major UK health research funder: a cohort study. BMJ Open [Internet]. British Medical Journal Publishing Group; 2013 May 2 [cited 2017 May 19];3(5):e002521. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23645914.

    Article  Google Scholar 

  10. Prayle A, Hurley M, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ Open. 2012;3444:d7373.

    Google Scholar 

  11. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. In: Hopewell S, editor. Cochrane Database of Systematic Reviews [Internet]. Chichester, UK: John Wiley & Sons, Ltd; 2009 [cited 2017 May 19]. Available from: http://doi.wiley.com/10.1002/14651858.MR000006.pub3.

  12. Song SY, Koo DH, Jung SY, Kang W, Kim EY. The significance of the trial outcome was associated with publication rate and time to publication. J Clin Epidemiol [Internet]. Elsevier Inc; 2017; Available from: https://doi.org/10.1016/j.jclinepi.2017.02.009.

    Article  Google Scholar 

  13. Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ [Internet]. 1997 [cited 2017 May 19];315(7109). Available from: http://www.bmj.com/content/315/7109/640.short.

  14. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med [Internet]. Massachusetts Medical Society; 2004 [cited 2017 May 19];351(12):1250–1251. Available from: http://www.nejm.org/doi/abs/10.1056/NEJMe048225.

    Article  Google Scholar 

  15. World Health Organization. International Clinical Trials Registry Platform (ICTRP). http://www.who.int/ictrp/en/ [Internet]. 2010 [cited 2017 May 19]; Available from: http://ci.nii.ac.jp/naid/10027448411/.

  16. World Health Organization. International Standards for Clinical Trial Registries WHO Library Cataloguing-in-Publication Data. WHO Libr Cat Data [Internet]. 2012 [cited 2017 May 19]; Available from: http://www.who.int/about/licensing/copyright_form/en/index.html.

  17. World Medical Association (WMA). Declaration of Helsinki. Ethical principles for medical research involving human subjects. Jahrb für Wiss und Ethik [Internet]. 2009 [cited 2017 May 19];14(1). Available from: https://www.degruyter.com/view/j/jfwe.2009.14.issue-1/9783110208856.233/9783110208856.233.xml.

  18. National Institutes of Health. NIH policy on dissemination of NIH-funded clinical trial information. Fed Regist. 2016;81:64922–8.

    Google Scholar 

  19. Department of Health and Human Services. Final rule—clinical trials registration and results information submission. Fed Regist. 2016;81:64981–5157.

    Google Scholar 

  20. Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao S, et al. Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. Am Psychol. 2018;73(1):3–25.

    Article  Google Scholar 

  21. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc [Internet]. 1958 [cited 2017 May 19];53(282):457–481. Available from: http://www.tandfonline.com/doi/abs/10.1080/01621459.1958.10501452.

    Article  Google Scholar 

  22. Breslow N. Biometrika Trust. A generalized Kruskal-Wallis test for comparing K samples subject to unequal patterns of censorship. Source: Biometrika [Internet]. 1970 [cited 2017 Jun 15];57(3):579–94. Available from: http://www.jstor.org/stable/pdf/2334776.pdf?refreqid=excelsior%3A3741da0818802adef36a227ffc0d79bd.

  23. Cox DR. Regression models and life-tables. J R Stat Soc Ser B [Internet]. 1972 [cited 2017 Jul 28];34(2):187–220. Available from: http://links.jstor.org/sici?sici=0035-9246%281972%2934%3A2%3C187%3ARMAL%3E2.0.CO%3B2-6.

  24. Manzoli L, Flacco ME, D’Addario M, Capasso L, De Vito C, Marzuillo C, et al. Non-publication and delayed publication of randomized trials on vaccines: survey. BMJ [Internet]. 2014 [cited 2017 May 19];348. Available from: http://www.bmj.com/content/348/bmj.g3058.full.

  25. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. Bmj [Internet]. 2013;347(oct28 9):f6104 Available from: http://www.bmj.com/cgi/doi/10.1136/bmj.f6104.

    Article  Google Scholar 

  26. Chen R, Desai NR, Ross JS, Zhang W, Chau KH, Wayda B, et al. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ [Internet]. 2016 [cited 2017 May 19];352. Available from: http://www.bmj.com/content/352/bmj.i637.short.

  27. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ [Internet]. 2012 [cited 2017 May 19];344. Available from: http://www.bmj.com/content/344/bmj.d7292.

  28. Scott A, Rucklidge JJ, Mulder RT, Strech D, Mann H, Berlin J. Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. Wicherts JM, editor. PLoS One [Internet]. Public Library of Science; 2015 [cited 2017 May 19];10(8):e0133718. Available from: http://dx.plos.org/10.1371/journal.pone.0133718.

  29. Shinohara K, Tajika A, Imai H, Takeshima N, Hayasaka Y, Furukawa TA. Protocol registration and selective outcome reporting in recent psychiatry trials: new antidepressants and cognitive behavioural therapies. Acta Psychiatr Scand [Internet]. 2015 [cited 2017 May 19];132(6):489–498. Available from: http://doi.wiley.com/10.1111/acps.12502.

    Article  Google Scholar 

  30. Cybulski L, Mayo-Wilson E, Grant S. Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J Consult Clin Psychol. 2016;84(9):753–67.

    Article  Google Scholar 

  31. Mayo-Wilson E, Heyward J, Keyes A, Reynolds J, White S, Atri N, et al. Clinical trial registration and reporting: a survey of academic organizations in the United States. BMC Med [Internet]. 2018;16(60). Available from: https://doi.org/10.1186/s12916-018-1042-6.

  32. Viergever RF, Ghersi D, Frizelle F, Haug C, Hoey J. The quality of registration of clinical trials. Scherer RW, editor. PLoS One [Internet]. Public Library of Science; 2011 [cited 2017 May 19];6(2):e14701. Available from: http://dx.plos.org/10.1371/journal.pone.0014701.

  33. Chan A-W, Pello A, Kitchen J, Axentiev A, Virtanen JI, Liu A, et al. Association of trial registration with reporting of primary outcomes in protocols and publications trial registration and reporting of primary outcomes in publications letters. JAMA [Internet]. 2017;318(17):1709–1711. Available from: https://doi.org/10.1001/jama.2017.13001.

    Article  Google Scholar 

  34. Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomised trials comparison of protocols to published articles. JAMA [Internet]. 2004;291(20):2457–2465. Available from: https://doi.org/10.1001/jama.291.20.2457.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Isabel Eggers (IE) and Simonne Wright (SW) for their assistance with publication searches and data extraction and verification.

Funding

The research reported in this publication was supported by the South African Medical Research Council for the “Shared Roots” Flagship Project, Grant no. MRC-RFA-IFSP-01-2013/SHARED ROOTS through the funding received from the South African National Treasury under its Economic Competitiveness and Support Package; the South African Research Chair in PTSD, hosted by Stellenbosch University, funded by the DST and administered by the NRF [S Seedat & S Suliman], the Claude Leon Foundation [S Suliman] and the SAMRC Clinician Researcher Programme [LvdH]. The funders played no role in the design of the study, collection, analysis and interpretation of data or in writing the manuscript. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the South African Medical Research Council, the National Research Foundation or the Claude Leon Foundation.

Author information

Authors and Affiliations

  1. Department of Psychiatry, Faculty of Medicine and Health Sciences, Stellenbosch University, PO Box 19063, Cape Town, Tygerberg, 7505, South Africa

    Sharain Suliman, Leigh van den Heuvel, Alexandra Suryapranata & Soraya Seedat

  2. Division of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Cardiff, UK

    Jonathan I. Bisson

Authors
  1. Sharain Suliman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leigh van den Heuvel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexandra Suryapranata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan I. Bisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Soraya Seedat
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SS2 conceptualised the review. SS1 and LvdH conducted the search and data extraction. AS assisted with the data extraction and verification. SS1, LvdH and SS2 contributed to the protocol development. All authors contributed to analysis and interpretation of data, intellectual content and read and approved the submitted manuscript.

Corresponding author

Correspondence to Sharain Suliman.

Ethics declarations

Ethics approval and consent to participate

As an overview that entailed pooled analysis of secondary data, ethical approval was not required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Suliman, S., van den Heuvel, L., Suryapranata, A. et al. Publication and non-publication of clinical trials in PTSD: an overview. Res Integr Peer Rev 4, 15 (2019). https://doi.org/10.1186/s41073-019-0074-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41073-019-0074-6

Keywords

Research Integrity and Peer Review

ISSN: 2058-8615

Contact us