Strengthen the Interoperability and Reusability of Research Outputs (SIRRO)

Rigorous design, transparent reporting, and reproducible workflows are major factors strengthening the interoperability and reusability of research data, and are hence crucial to increase the value of research data and, more broadly, the value of research outputs. The Swiss Reproducibility Network (SwissRN) is a peer-led consortium aiming to promote and ensure exactly such rigorous research practices in Switzerland, UZH is a local node of SwissRN through the Center for Reproducible Science. It combines the experience of experts from across a wide range of disciplines and aims to benefit from interdisciplinary synergies.

The aim of the current project is to

1. strengthen SwissRN as an existing community engaging with ORD practices that have the goal of strengthening interoperability and reusability, and
2. intensify the efforts of SwissRN towards a systematic assessment of the impact and obstacles in the implementation of ORD practices.

The project aims to focus on the ORD practices of preregistration (the deposition of a detailed research plan ahead of data collection on a repository) and data management planning as measures to avoid bias and to increase quality. The project contains four main parts:

SIRRO addresses the following challenges in its four parts.

Researchers’ Understanding and Perception of ORD Practices

Researchers and their careers will be directly affected by ORD related implementation measures. But a wide-spread, representative assessment of researchers’ awareness and acceptance of ORD implementation measures in Switzerland including their perceived impact on careers does not yet exist. To facilitate the uptake of ORD practices in Switzerland a representative survey is needed. The planned survey concerns ORD-related practices in general, as well as more specific topics such as preregistration and research data management, etc. Researchers will be able to indicate importance in three different areas: academic career, scientific progress in general, and personal satisfaction. We intend to reach a large, representative sample of researchers from diverse disciplines and at different career stages.

Measuring Research Output

There is a large body of literature on assessing researcher productivity using traditional metrics, i.e., number of publications etc. Alternative measures to assess a researcher’s impact or productivity (altmetrics, relative citation ratio, etc.) emerged, but their focus is still on traditional research output. More innovative measures like the S-index were proposed but are not yet implemented. In general, there is a lack of knowledge regarding the production of alternative research outputs different from scientific publications, as for example preprints, preregistrations, data, or software; especially with respect to the degree by which differences in disciplines, institutions, and other research(er) demographics can be observed. To assess the production of alternative research outputs, an exploratory data analysis will be performed on publicly available data from the SNSF.

Perceived Obstacles and Incentives

Preregistration has demonstrated substantial value in a variety of research contexts. Nevertheless, there are reasonable concerns about relevance, applicability, feasibility, implementation, and unintended impacts of preregistration particularly in more exploratory research that need to be addressed. Data management planning should be done at the same, early stage of the research cycle and is equally valuable and sometimes contested as preregistration. The uptake of both practices may profit from each other’s popularity in certain fields helping to mutually motivate researchers to benefit from their advantages. But to indeed increase the uptake of any new practice in a field all actors need to be involved and communities consulted such that obstacles can be identified and eliminated.
To better understand the perceived obstacles and factors facilitating adoption of preregistration (including data management planning), we focus on a specific field of research: animal studies. A feasibility study will be performed to assess the possibility of implementing mandatory preregistration and reporting of animal experiments in Switzerland, using the regulations for human research as a starting point. To engage the community from the beginning, a targeted survey and semi-structured interviews will explore the current use of preregistration, and factors promoting and inhibiting its use. This part of SIRRO is part of a project mandated by the Federal Food Safety and Veterinary Office and will be aligned with the project of an international network with similar aims but across all scientific disciplines.

Training Activities

Methodological best practices are under constant revision and improvement; preregistration and data management planning are examples of that. Contrary to ubiquitous specialized training, cross-disciplinary courses focusing on best practices to enhance the quality of research are lacking at universities and are urgently needed to ensure a common understanding of rigorous experimental design, research standards and objective evaluation of data.

The outputs of SIRRO in its four parts are

1. The results of the survey will uncover researchers' knowledge, awareness, and attitudes with respect to various ORD-related research practices in three different areas: academic career, scientific progress in general, and personal satisfaction. [link to survey results]
2. The database assessment will result in a graphical visualization of the data as yearly counts (of publications, data sets, public communications, etc), stratified using different demographics of the researchers, their group and type of grant. [link to assessment]
3. The results of this part will provide an overview of the attitude of the researchers and other stakeholders towards preregistration, and identify obstacles and facilitators of its use. [link to feasibility]
4. The SIRRO training materials are openly available to the entire community. [training materials on OSF]

With this project SwissRN aims to intensify its efforts towards a systematic assessment of the impact and obstacles in the implementation of ORD practices by engaging with the entire research community in various ways.

More specifically, the four parts of SIRRO aim to advance the ORD field substantially by

1. identifying researchers’ knowledge, awareness, and attitudes towards ORD practices (including perceived obstacles) within Switzerland across all disciplines,
2. providing the data needed to initiate and support the effort of switching the focus from publications to other types of open research output,
3. uncovering attitudes, including perceived obstacles and facilitators, by including the voice of a research community before implementing new policy, and
4. offering training of essential skills for a multi-disciplinary audience.