Table 1 Overview of applications we included in the analysis
Application | Description |
|---|---|
In Authorea, authors can create executable papers collaboratively. They can attach code and data to figures to make them reproducible. Authors can also directly submit to a journal and, at the same time, publish a preprint. | |
Binder creates a containerized executable environment based on a repository (e.g., on GitHub/Lab, Zenodo) including a Jupyter Notebook [24]. Readers can launch the analysis and inspect the workflow in a browser. | |
Code Ocean creates “capsules” containing code, data, and the computational environment. While reading, users can execute and inspect the analysis in a separate window below the article or on Code Ocean’s website [25]. | |
RDS originates from the life sciences. Authors can publish executable documents based on Stencila (https://stenci.la/), an open-source editor for articles. The executable document, which contains the whole narrative and executable code snippets, is not only a supplement but the actual scientific article. | |
Galaxy [26] provides features tailored to use cases in the life sciences. It is a web app for developing comput. Analyses without programming expertise. Scientists can upload and analyze data using Jupyter Notebooks [27]. | |
Gigantum packages code, data, the computational environment, and the work history into a Git repository. Gigantum is composed of a client app for creating as well as executing analyses locally and a cloud-based infrastructure for sharing computations and collaborating with peers. | |
Manuscripts is an online tool for writing executable documents collaboratively based on the concept of literate programming, but featuring a “What you see is what you get” user interface. The runtime environment of the author is, however, not considered. | |
o2r [22] originates from the geosciences and addresses publishers who want to extend their infrastructure via a reproducibility service during the process of paper submission [28]. Authors can create interactive figures, allowing readers to change model parameters using a slider [29]. | |
REANA [4, 30] originates from particle physics and provides a specification for capturing data, code, and the comput. Environment. Based on this structure and manually created configuration files, REANA provides command line interface (CLI) commands to run large analyses on a remote REANA cloud. | |
ReproZip [31, 32] provides a set of CLI commands for encapsulating data, code, and the computational environment. Users can execute the resulting bundle on a server provided by ReproZip [33] or locally on different systems. | |
With Whole Tale [34], authors can create so-called “Tales” that combine narrative, data, code, and the computational environment. Readers can inspect the materials and execute the analysis in the original environment. |