Sam Kirshner | Artificial Agents in OM Experiments

Artificial Agents in Operations Management Experiments

Large language models Artificial Agents in Operations Management Experiments (LLMs) like GPT hold promise for advancing behavioural research through their ability to simulate decision-making processes and test hypotheses in controlled experimental settings. This study examines GPT’s potential to replicate behavioural hypotheses in operations management (OM) experiments by conducting 9 studies from the Management Science Replication Project. Using a straightforward prompt instructing GPT to make decisions based on intuition, the model successfully replicated the core hypotheses in 8 of the 9 experiments. The one exception involved a hypothesis that was only partially supported in the original study. These results highlight GPT’s utility as a methodological tool for validating experimental findings. This research further explores different strategies for enhancing experimental replication, including the use of behavioural personas and simulated personality traits, while evaluating their relevance in piloting hypothesis tests. Beyond methodological contributions, the findings offer insights into how LLMs can be used in replication experiments and suggest future directions for leveraging LLMs in behavioural OM research.

Read the paper here, opens in a new window

If you are interested in attending, please fill out the expression of interest form below.

EOI Form, opens in a new window

Sam Kirshner is an Associate Professor in the School of Information Systems and Technology Management at the University of New South Wales. Sam completed his PhD in Management Science at Queen’s University, in Kingston, Canada. His primary research interests lie in analyzing behavioural decision-making in operations and technology management and studying how algorithms and artificial intelligence impact decision making. His research and commentaries have been published in prestigious academic journals including Management Science, Decision Sciences, Production and Operations Management, Manufacturing & Service Operations Management, the European Journal of Operational Research, Journal of Behavioral Decision Making, Tourism Management, and Science.

Learn more about him on his UNSW webpage, opens in a new window.