Thomas Pfeiffer

• October 05, 2021
• Speaker: Thomas Pfeiffer
  
• Topic: Replication Markets In The Social And Behavioural Sciences (joint work with Michael Gordon, Domenico Viganola, Anna Dreber, and Magnus Johannesson)
  

The last few years have seen a lively discussion about the reliability of published research. Concerns have been raised because of theoretical and conceptual considerations regarding false positives in the scientific literature, observations on the prevalence of questionable research practices, and empirical findings from large-scale replication projects. One approach to assess the credibility of published research is to elicit beliefs about its reproducibility. In four studies we have recently used surveys and prediction markets to estimate beliefs about replication in systematic large-scale replication projects.

The sample sizes in each study were small, which constrained the ability to test a number of hypotheses regarding the performance of prediction markets and surveys. However, the pooled data from these four studies cover 103 replications and allows to assess the performance of surveys and prediction markets in greater detail. In my presentation I will go over key findings from this dataset, and will discuss why forecasting replication outcomes, and experimental outcomes in general, can be of advantage for the practice of science.

Thomas Pfeiffer is a Professor in Computational Biology/Biochemistry at the New Zealand Institute for Advanced Study, Massey University. His research interests include game theory, decision-making, and meta science.