Is reproducibility a word

From Wikipedia, the free encyclopedia

This article is about the reproducibility of scientific research results. For reproductive capacity of organisms, see fertility and fecundity.

Reproducibility, closely related to replicability and repeatability, is a major principle underpinning the scientific method. For the findings of a study to be reproducible means that results obtained by an experiment or an observational study or in a statistical analysis of a data set should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication^[1] but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.

With a narrower scope, reproducibility has been introduced in computational sciences: Any results should be documented by making all data and code available in such a way that the computations can be executed again with identical results.

In recent decades, there has been a rising concern that many published scientific results fail the test of reproducibility, evoking a reproducibility or replication crisis.

History[edit]

The first to stress the importance of reproducibility in science was the Irish chemist Robert Boyle, in England in the 17th century. Boyle’s air pump was designed to generate and study vacuum, which at the time was a very controversial concept. Indeed, distinguished philosophers such as René Descartes and Thomas Hobbes denied the very possibility of vacuum existence. Historians of science Steven Shapin and Simon Schaffer, in their 1985 book Leviathan and the Air-Pump, describe the debate between Boyle and Hobbes, ostensibly over the nature of vacuum, as fundamentally an argument about how useful knowledge should be gained. Boyle, a pioneer of the experimental method, maintained that the foundations of knowledge should be constituted by experimentally produced facts, which can be made believable to a scientific community by their reproducibility. By repeating the same experiment over and over again, Boyle argued, the certainty of fact will emerge.

The air pump, which in the 17th century was a complicated and expensive apparatus to build, also led to one of the first documented disputes over the reproducibility of a particular scientific phenomenon. In the 1660s, the Dutch scientist Christiaan Huygens built his own air pump in Amsterdam, the first one outside the direct management of Boyle and his assistant at the time Robert Hooke. Huygens reported an effect he termed «anomalous suspension», in which water appeared to levitate in a glass jar inside his air pump (in fact suspended over an air bubble), but Boyle and Hooke could not replicate this phenomenon in their own pumps. As Shapin and Schaffer describe, “it became clear that unless the phenomenon could be produced in England with one of the two pumps available, then no one in England would accept the claims Huygens had made, or his competence in working the pump”. Huygens was finally invited to England in 1663, and under his personal guidance Hooke was able to replicate anomalous suspension of water. Following this Huygens was elected a Foreign Member of the Royal Society. However, Shapin and Schaffer also note that “the accomplishment of replication was dependent on contingent acts of judgment. One cannot write down a formula saying when replication was or was not achieved”.^[2]

The philosopher of science Karl Popper noted briefly in his famous 1934 book The Logic of Scientific Discovery that “non-reproducible single occurrences are of no significance to science”.^[3] The statistician Ronald Fisher wrote in his 1935 book The Design of Experiments, which set the foundations for the modern scientific practice of hypothesis testing and statistical significance, that “we may say that a phenomenon is experimentally demonstrable when we know how to conduct an experiment which will rarely fail to give us statistically significant results”.^[4] Such assertions express a common dogma in modern science that reproducibility is a necessary condition (although not necessarily sufficient) for establishing a scientific fact, and in practice for establishing scientific authority in any field of knowledge. However, as noted above by Shapin and Schaffer, this dogma is not well-formulated quantitatively, such as statistical significance for instance, and therefore it is not explicitly established how many times must a fact be replicated to be considered reproducible.

Terminology[edit]

Replicability and repeatability are related terms broadly or loosely synonymous with reproducibility (for example, among the general public), but they are often usefully differentiated in more precise senses, as follows.

Two major steps are naturally distinguished in connection with reproducibility of experimental or observational studies:
When new data is obtained in the attempt to achieve it, the term replicability is often used, and the new study is a replication or replicate of the original one. Obtaining the same results when analyzing the data set of the original study again with the same procedures, many authors use the term reproducibility in a narrow, technical sense coming from its use in computational research.
Repeatability is related to the repetition of the experiment within the same study by the same researchers.
Reproducibility in the original, wide sense is only acknowledged if a replication performed by an independent researcher team is successful.

Unfortunately, the terms reproducibility and replicability sometimes appear even in the scientific literature with reversed meaning,^[5][6] when researchers fail to enforce the more precise usage.

Measures of reproducibility and repeatability[edit]

In chemistry, the terms reproducibility and repeatability are used with a specific quantitative meaning.^[7] In inter-laboratory experiments, a concentration or other quantity of a chemical substance is measured repeatedly in different laboratories to assess the variability of the measurements. Then, the standard deviation of the difference between two values obtained within the same laboratory is called repeatability. The standard deviation for the difference between two measurement from different laboratories is called reproducibility.^[8]
These measures are related to the more general concept of variance components in metrology.

Reproducible research[edit]

Reproducible research method[edit]

The term reproducible research refers to the idea that scientific results should be documented in such a way that their deduction is fully transparent. This requires a detailed description of the methods used to obtain the data^[9][10]
and making the full dataset and the code to calculate the results easily accessible.^{[11][12][13][14][15][16]}
This is the essential part of open science.

To make any research project computationally reproducible, general practice involves all data and files being clearly separated, labelled, and documented. All operations should be fully documented and automated as much as practicable, avoiding manual intervention where feasible. The workflow should be designed as a sequence of smaller steps that are combined so that the intermediate outputs from one step directly feed as inputs into the next step. Version control should be used as it lets the history of the project be easily reviewed and allows for the documenting and tracking of changes in a transparent manner.

A basic workflow for reproducible research involves data acquisition, data processing and data analysis. Data acquisition primarily consists of obtaining primary data from a primary source such as surveys, field observations, experimental research, or obtaining data from an existing source. Data processing involves the processing and review of the raw data collected in the first stage, and includes data entry, data manipulation and filtering and may be done using software. The data should be digitized and prepared for data analysis. Data may be analysed with the use of software to interpret or visualise statistics or data to produce the desired results of the research such as quantitative results including figures and tables. The use of software and automation enhances the reproducibility of research methods.^[17]

There are systems that facilitate such documentation, like the R Markdown language^[18]
or the Jupyter notebook.^[19][20][21]
The Open Science Framework provides a platform and useful tools to support reproducible research.

Reproducible research in practice[edit]

Psychology has seen a renewal of internal concerns about irreproducible results (see the entry on replicability crisis for empirical results on success rates of replications). Researchers showed in a 2006 study that, of 141 authors of a publication from the American Psychological Association (APA) empirical articles, 103 (73%) did not respond with their data over a six-month period.^[22] In a follow up study published in 2015, it was found that 246 out of 394 contacted authors of papers in APA journals did not share their data upon request (62%).^[23] In a 2012 paper, it was suggested that researchers should publish data along with their works, and a dataset was released alongside as a demonstration.^[24] In 2017, an article published in Scientific Data suggested that this may not be sufficient and that the whole analysis context should be disclosed.^[25]

In economics, concerns have been raised in relation to the credibility and reliability of published research. In other sciences, reproducibility is regarded as fundamental and is often a prerequisite to research being published, however in economic sciences it is not seen as a priority of the greatest importance. Most peer-reviewed economic journals do not take any substantive measures to ensure that published results are reproducible, however, the top economics journals have been moving to adopt mandatory data and code archives.^[26] There is low or no incentives for researchers to share their data, and authors would have to bear the costs of compiling data into reusable forms. Economic research is often not reproducible as only a portion of journals have adequate disclosure policies for datasets and program code, and even if they do, authors frequently do not comply with them or they are not enforced by the publisher. A Study of 599 articles published in 37 peer-reviewed journals revealed that while some journals have achieved significant compliance rates, significant portion have only partially complied, or not complied at all. On an article level, the average compliance rate was 47.5%; and on a journal level, the average compliance rate was 38%, ranging from 13% to 99%.^[27]

A 2018 study published in the journal PLOS ONE found that 14.4% of a sample of public health researchers had shared their data or code or both.^[28]

There have been initiatives to improve reporting and hence reproducibility in the medical literature for many years, beginning with the CONSORT initiative, which is now part of a wider initiative, the EQUATOR Network.
This group has recently turned its attention to how better reporting might reduce waste in research,^[29] especially biomedical research.

Reproducible research is key to new discoveries in pharmacology. A Phase I discovery will be followed by Phase II reproductions as a drug develops towards commercial production. In recent decades Phase II success has fallen from 28% to 18%. A 2011 study found that 65% of medical studies were inconsistent when re-tested, and only 6% were completely reproducible.^[30]

Noteworthy irreproducible results[edit]

Hideyo Noguchi became famous for correctly identifying the bacterial agent of syphilis, but also claimed that he could culture this agent in his laboratory. Nobody else has been able to produce this latter result.^[31]

In March 1989, University of Utah chemists Stanley Pons and Martin Fleischmann reported the production of excess heat that could only be explained by a nuclear process («cold fusion»). The report was astounding given the simplicity of the equipment: it was essentially an electrolysis cell containing heavy water and a palladium cathode which rapidly absorbed the deuterium produced during electrolysis. The news media reported on the experiments widely, and it was a front-page item on many newspapers around the world (see science by press conference). Over the next several months others tried to replicate the experiment, but were unsuccessful.^[32]

Nikola Tesla claimed as early as 1899 to have used a high frequency current to light gas-filled lamps from over 25 miles (40 km) away without using wires. In 1904 he built Wardenclyffe Tower on Long Island to demonstrate means to send and receive power without connecting wires. The facility was never fully operational and was not completed due to economic problems, so no attempt to reproduce his first result was ever carried out.^[33]

Other examples which contrary evidence has refuted the original claim:

• Stimulus-triggered acquisition of pluripotency, revealed to be the result of fraud
• GFAJ-1, a bacterium that could purportedly incorporate arsenic into its DNA in place of phosphorus
• MMR vaccine controversy – a study in The Lancet claiming the MMR vaccine caused autism was revealed to be fraudulent
• Schön scandal – semiconductor «breakthroughs» revealed to be fraudulent
• Power posing – a social psychology phenomenon that went viral after being the subject of a very popular TED talk, but was unable to be replicated in dozens of studies^[34]

See also[edit]

References[edit]

Further reading[edit]

• Timmer, John (October 2006). «Scientists on Science: Reproducibility». Ars Technica.
• Saey, Tina Hesman (January 2015). «Is redoing scientific research the best way to find truth? During replication attempts, too many studies fail to pass muster». Science News. «Science is not irrevocably broken, [epidemiologist John Ioannidis] asserts. It just needs some improvements. «Despite the fact that I’ve published papers with pretty depressive titles, I’m actually an optimist,” Ioannidis says. “I find no other investment of a society that is better placed than science.”»

External links[edit]

• Transparency and Openness Promotion Guidelines from the Center for Open Science
• Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results of the National Institute of Standards and Technology
• Reproducible papers with artifacts by the CTuning foundation
• ReproducibleResearch.net

Educalingo cookies are used to personalize ads and get web traffic statistics. We also share information about the use of the site with our social media, advertising and analytics partners.

Download the app
educalingo

PRONUNCIATION OF REPRODUCIBILITY

GRAMMATICAL CATEGORY OF REPRODUCIBILITY

Reproducibility is a noun.

WHAT DOES REPRODUCIBILITY MEAN IN ENGLISH?

Definition of reproducibility in the English dictionary

The definition of reproducibility in the dictionary is the quality or state of being capable of being produced or created again.

WORDS THAT RHYME WITH REPRODUCIBILITY

TRANSLATION OF REPRODUCIBILITY

Find out the translation of reproducibility to 25 languages with our English multilingual translator.

TENDENCIES OF USE OF THE TERM «REPRODUCIBILITY»

The term «reproducibility» is quite widely used and occupies the 36.377 position in our list of most widely used terms in the English dictionary.

Principal search tendencies and common uses of reproducibility

FREQUENCY OF USE OF THE TERM «REPRODUCIBILITY» OVER TIME

10 ENGLISH BOOKS RELATING TO «REPRODUCIBILITY»

Discover the use of reproducibility in the following bibliographical selection. Books relating to reproducibility and brief extracts from same to provide context of its use in English literature.

10 NEWS ITEMS WHICH INCLUDE THE TERM «REPRODUCIBILITY»

Find out what the national and international press are talking about and how the term reproducibility is used in the context of the following news items.

REFERENCE

« EDUCALINGO. Reproducibility [online]. Available <https://educalingo.com/en/dic-en/reproducibility>. Apr 2023 ».

Asked by: Emma Kassulke

Score: 4.5/5
(74 votes)

Reproducibility is a major principle underpinning the scientific method. For the findings of a study to be reproducible means that results obtained by an experiment or an observational study or in a …

What does reproducible mean in science?

B1: “Reproducibility” refers to instances in which the original researcher’s data and computer codes are used to regenerate the results, while “replicability” refers to instances in which a researcher collects new data to arrive at the same scientific findings as a previous study.

How do you define reproducible?

What is reproducibility of an experiment?

Reproducibility (Different team, same experimental setup). If an observation is reproducible, it should be able to be made by a different team repeating the experiment using the same experimental data and methods, under the same operating conditions, in the same or a different location, on multiple trials.

What is reproducibility in biology?

Reproducibility — defined here as the ability to produce similar results by independent replicate studies — is thus a cornerstone of the scientific method.

22 related questions found

воспроизводимость, воспроизводимость или сходимость результатов, повторяемость

существительное

- воспроизводимость или сходимость результатов

Словосочетания

results reproducibility — воспроизводимость результатов  
repeatability and reproducibility — повторяемость и воспроизводимость  
performance reproducibility — воспроизводимость характеристик  
reproducibility index — коэффициент воспроизводимости; коэффициент воспроизводства  
reproducibility limit — предел воспроизводимости  
reproducibility of measurements — воспроизводимость результатов измерений  
reproducibility of readings — воспроизводимость показаний  
reproducibility of results — воспроизводимость результатов измерений; воспроизводимость результатов  
reproducibility of test results — воспроизводимость результатов испытания  
reproducibility of tests — воспроизводимость результатов испытаний; параллельные испытания  

Примеры с переводом

Примеры, ожидающие перевода

Thank you. Listen to this article using the player above. ✖

In measuring the quality of experiments, repeatability and reproducibility are key. In this article, we explore the differences between the two terms, and why they are important in determining the worth of published research. 

What is repeatability?

Repeatability is a measure of the likelihood that, having produced one result from an experiment, you can try the same experiment, with the same setup, and produce that exact same result. It’s a way for researchers to verify that their own results are true and are not just chance artifacts.

To demonstrate a technique’s repeatability, the conditions of the experiment must be kept the same. These include: 

• Location
• Measuring tools
• Other apparatus used in the experiment
• Observer
• Hypothesis
• Time period (taking month-long breaks between repetitions isn’t good practice) 

Bland and Altman authored an extremely useful paper in 1986 which highlighted one benefit of assessing repeatability: it allows one to make comparisons between different methods of measurement.

Previous studies had used similarities in the correlation coefficient (r) between techniques as an indicator of agreement. Bland and Altman showed that r actually measured the strength of the relation between two techniques, not the extent to which they agree with each other. This means r is quite meaningless in this context; if two different techniques were both designed to measure heart rate, it would be bizarre if they weren’t related to each other!

What is reproducibility?

The reproducibility of data is a measure of whether results in a paper can be attained by a different research team, using the same methods. This shows that the results obtained are not artifacts of the unique setup in one research lab. It’s easy to see why reproducibility is desirable, as it reinforces findings and protects against rare cases of fraud, or less rare cases of human error, in the production of significant results.

Why are repeatability and reproducibility important?

Science is a method built on an approach of gradual advance backed up by independent verification and the ability to show that your findings are correct and transparent. Academic research findings are only useful to the wider scientific community if the knowledge can be repeated and shared among research groups. As such, irreproducible and unrepeatable studies are the source of much concern within science. 

What is the reproducibility crisis?

Over recent decades, science, in particular the social and life sciences, has seen increasing importance placed on the reproducibility of published studies. Large-scale efforts to assess the reproducibility of scientific publications have turned up worrying results. For example, a 2015 paper by a group of psychology researchers dubbed the “Open Science Collaboration” examined 100 experiments published in high-ranking, peer-reviewed journals. Of these 100 studies, just 68 reproductions provided statistically significant results that matched the original findings. These efforts are part of a growing field of “metascience” that aims to take on the reproducibility crisis.

How can we improve reproducibility?

A lot of thought is being put into improving experimental reproducibility. Below are just some of the ways you can improve reproducibility:

• Journal checklists – more and more journals are coming to understand the importance of including all relevant details in published studies, and are bringing in mandatory checklists for any published papers. The days of leaving out sample numbers and animal model descriptions in methods sections are over and blinding and randomization should be standard where possible.
• Strict on stats – power calculations, multiple comparisons tests and descriptive statistics are all essential to making sure that reported results are statistically sound. 
• Technology can lend a hand – automating processes and using high-throughput systems can improve accuracy in individual experiments, and enable more measurements to be taken in a given time, increasing sample numbers.