Who do bureaucrats believe? A randomized controlled experiment testing perceptions of credibility of policy research

More than ever before, analysts in government have access to policy-relevant research and advocacy, which they consume, interpret, and apply in their role in the policy process. Academics have historically occupied a... [ view full abstract ]

More than ever before, analysts in government have access to policy-relevant research and advocacy, which they consume, interpret, and apply in their role in the policy process. Academics have historically occupied a privileged position of authority and legitimacy in the public domain as it relates to applied policy research, but some argue this is changing with the rapid growth of think tanks and research-based advocacy organizations. This paper is a replication study and expansion of Doberstein (2016), which presented a randomized controlled survey experiment using policy analysts from the British Columbia provincial government in Canada to systematically test the source effects of applied policy research. The experimental findings in Doberstein (2016) lent evidence to the hypothesis that academic research is perceived to be substantially more credible to government policy analysts than think tank or advocacy organization research, regardless of its content, and that sources perceived as more ideological were much less credible. This study seeks to reproduce these findings by replicating the survey experiment in three additional Canadian provincial governments to verify if the relationship persists in a larger sample, across jurisdictions, and in conjunction with further randomization procedures. This study corroborates and expands on the findings in Doberstein (2016), and further confirms that external policy advice systems are not a pluralistic arena of policy research and advice, but instead is one subject to powerful heuristics that bureaucrats use to sift through policy-relevant information and advice. The sources and implications of such biases among analysts in government are considered.