Replicating Experiments in Public Administration: Best Practice Recommendations for Achieving Generalizability

Research in public management and administration is increasingly relying on survey, laboratory, and field experiments. The high internal validity of these designs allows researchers to identify effects with greater confidence... [ view full abstract ]

Research in public management and administration is increasingly relying on survey, laboratory, and field experiments. The high internal validity of these designs allows researchers to identify effects with greater confidence than from observational studies. Yet, the generalizability of any single experiment is limited. Only by replicating a study, ideally varying the context, can broader lessons for theory and practice be drawn. Experiments are costly. So, whether researchers develop a new experiment or replicate an extend one that has already been conducted at least once, it is helpful to draw on existing experience to limit errors that could invalidate all the effort that went into the experiment. Drawing on our replications of several of the most-cited experiments in public administration, we provide an overview of our experiences throughout all stages of experimental replication. Using our replication of Brewer and Brewer’s (2011) publicness–work effort experiment as the running example, we distill our experiences into a clear set of best practices on how to design and implement replications of experiments in public administration. The first section of the paper reviews the rationales for replication in the social sciences in general and public administration in particular. We use Tsang and Kwan’s (1999) framework for conceptual clarity. In the second section, we identify the boundary conditions within which replications in public administration are likely to add value. In the third section, we cover each step involved in the planning of a replication. In the fourth section, we review our experiences in replicating Brewer and Brewer (2011) and identify practical challenges and how to address them. The final section offers a strategy for researchers interested in designing a replication to maximize their contribution and likelihood of scholarly impact.