Gilad Feldman

• January 20, 2020
• Speaker: Gilad Feldman
  
• Topic: Mass Mobilizing For Collaborative “Credibility Revolution” Open-Science: Insights, Findings, And Opportunities From A Project Of 56 Pre-Registered Replications And Extensions In Judgement And Decision Making
  

Reproducibility and replicability are at heart of science, yet increasing evidence from recent years suggests that many of the findings in psychological science are irreproducible and non-replicable in what some termed as a “replication/reproducibility crisis” and a new “credibility revolution” calling for significant changes in the way we do science, to embrace open-science practices.
Even if we adopt these changes, it will take us decades to address the crisis. I will argue that to successfully address the situation we must mass mobilize students and leverage our courses to implement collaborative open science with real impact. I have been implementing such a change in my taught social-psychology courses at the University of Hong Kong (HKU). We have concluded the 3rd semester of running a mass replication effort of impactful Judgment and Decision Making (JDM) literature classics (summary).

Last year we completed 45 completed pre-registered replications (examples), and this year we pushed things further with 22 teams conducting replications of 11 new target studies, with 2 extensions for each article. We implemented a Registered Reports format (manuscript template, and supplementary template), with pre-registrations of code analyzing simulated datasets, advanced templates for all outputs, comprehensive updated guides, Slack open communications, and open peer-review with external reviewers from around the world, in collaboration with large-scale paid prediction markets.

This semester, we also went beyond replications and extensions, to try out new directions: 1) Students conducted open-science assessments of impactful Registered Replication Reports articles and our students’ projects from previous years, testing, coding, and learning from others’ pre-registrations, open-science code/data sharing, and reports, 2) Students collaboratively wrote a book detailing the development of the credibility revolution, 3) Students designed and ran in-class experiment activities on their class-mates to gain hands-on experience with adjustment and implementation of classics for engagement in real-life. I will briefly present the process, findings, main insights, and my own take-aways, and will review open resources we developed such as course materials, collaborative guides, and tools that can be used by all in their research labs and teaching.

Gilad is an assistant professor with the psychology department at the University of Hong Kong. Gilad’s work is in the areas of social-cognitive psychology and judgment and decision-making. Research focuses on the topics of biases and heuristics, especially regarding agency and action. In recent years, with the emergence of the credibility revolution (so-called “replication/reproducibility crisis”), Gilad has taken a special interest in the movement for advancing psychological science by giving workshops and seminars and guiding researchers and students about open-science, statistical tools and simulations using open-source software (R), meta-analyses, pre-registrations, replications, and reproducibility.

• Open workshop materials
• Open teaching materials
  

More details on Gilad can be found at http://giladfeldman.org