Jack Fitzgerald | Hypothetical incentives in experiments

Topic: Identifying the Impact of Hypothetical Incentives on Experimental Outcomes and Treatment Effects

Recent findings showing that outcome variables do not statistically significantly differ between incentivized and unincentivized conditions in experiments have spurred methodological challenges to experimental economics' disciplinary norm that experimental choices should be incentivized with real stakes. I show that the classical hypothetical bias measures estimated in these studies do not econometrically identify the kinds of hypothetical bias that matter in most modern experiments. Specifically, classical hypothetical bias measures are fully informative in 'elicitation experiments' where the researcher is uninterested in treatment effects (TEs). However, in 'intervention experiments' where TEs are of interest, classical measures are uninformative, and incentivization schemes matter if and only if TEs are heterogeneous between incentivization schemes. I demonstrate that classical hypothetical bias metrics can be misleading measures of hypothetical bias for intervention experiments, both econometrically and through several empirical applications. The fact that a given experimental outcome does not statistically significantly differ on average between incentivization conditions does not imply that all TEs on that outcome are unimpacted by incentivization. Therefore, the recent hypothetical bias literature does not justify leaving most modern experiments unincentivized. Norms in favor of completely or probabilistically incentivizing experimental choices remain useful for ensuring externally valid TEs in experimental economics.

Structure

The talk will be about 40 minutes with 15 minutes for questions and answers. Questions are welcome during the talk but will eat into question time at the end. The talk will not be recorded.

If you are interested in attending, please fill out the expression of interest form below.

EOI Form

Jack Fitzgerald is a PhD candidate in economics at Vrije Universiteit (VU) Amsterdam. He works on issues in applied econometrics, replication, and economics of science.

Equivalence testing is a major focus of his current research. He documents empirical evidence that null claims made in the absence of equivalence testing exhibit high error rates in top economics journals. To facilitate adoption, he offered practical guidelines and developed software commands for implementing equivalence testing. He also introduced equivalence testing improvements to standard methods (such as manipulation tests in regression discontinuity and hypothetical bias experiments) and introduced equivalence testing to different disciplines through empirical applications (for example, see his article in the Journal of Business Ethics).

Learn more about him on his personal website.