Reproducibility in Journal of Structural Engineering Papers

Reproducibility of research results is central to advancing academic research but is also challenging due to increased effort in preparing the paper documentation, technology constraints such as data storage and computing requirements, and proprietary issues (Rosenberg et al. 2020). In 2020, the ASCE Journal Water Resources Planning and Management adopted and published its policy regarding reproducibility (Rosenberg et al. 2021). Several incentives were introduced for papers that meet its standards in order to promote reproducible work within that journal. This initiative is a timely and necessary step, and in the right direction, to make research more impactful and advance it quickly through follow-on work.

There is a need to develop a similar approach toward reproducibility of research results for both the Journal of Structural Engineering and the Journal of Bridge Engineering, and this editorial was cowritten by the editors in chief of both journals. It is also important to take into account the nature and culture of research within the subdiscipline. Reproducibility cannot be answered with a simple yes or no but, rather, resides on a continuum from data availability to full replicability, with results reproducibility somewhere in between (Rosenberg et al. 2020). A policy around reproducibility in structural engineering must recognize that significant portions of the work involve physical testing of laboratory or field specimens for which full reproducibility of the results may not be practical. Additionally, even broader models that focus on full systems and systems of systems add another layer of complexity, which often requires inherent assumptions resulting from a dearth of data. Nonetheless, results produced through models that can be reproduced should be encouraged in all journals.

A new classification within research articles is being implemented in the Journal of Structural Engineering that underscores the importance of reproducibility in published results. Recognizing the nature of research in this field, papers that publish in these journals are encouraged to make data available and to include enough metadata that the data can be used by others around the world who were not involved in the original effort. It is clearly recognized that while data availability does not mean that the results are fully reproducible, this is an important first step in the continuum of reproducibility. While the policy is limited to data availability including robust metadata, authors are highly encouraged to go beyond availability to make models and other materials available in repositories for the results to be reproducible.

To incentivize research and make data available in these two journals, a visible recognition badge with an inscription “Open Data/Models/Codes” prominently displayed on the paper will be provided. Such papers will also be promoted within the journal(s) and may be considered for a special recognition upon the editors’ recommendation such as an annual Best Data-Verified Paper award.

At the time of submission, the authors should indicate whether their paper should be considered as a candidate for a data-verified paper (DVP). If yes, authors of the DVP will be asked to include a “Data Availability Description” section describing the data that are made available (e.g., which figures, sections), repository location, and details (e.g., hardware and software, version) pertaining to the shared data. All the data that the authors claim to be available should be made available at the time of review in a public repository. Guidelines on making such data available in a repository are summarized in Rosenberg et al. (2020) and include standardized formats, detailed instructions, including pertinent metadata, version control for code, and synthetic data with results where proprietary issues make it challenging to share original data.

The review process for DVPs will be as follows. Three associate editors with the joint title data verification editor will be appointed to utilize their expertise to determine if the data are representative of what is stated by the paper authors, and the data are available to anyone worldwide without necessary communication with the authors. This will be done in parallel with the normal technical review process and will not delay the duration of the paper review cycles.