Survey Design and Forecast Accuracy

Surveys of professional forecasts are a key input in governmental and private sector decisions. This is the first paper to study how the design and survey features affect accuracy. We analyze a new dataset - the Focus Survey... [ view full abstract ]

Surveys of professional forecasts are a key input in governmental and private sector decisions. This is the first paper to study how the design and survey features affect accuracy. We analyze a new dataset - the Focus Survey - which is uniquely suited for our goals: Agents can update any day, so we can analyze what drives their updating. The bank runs a monthly contest that ranks forecasters based on their accuracy the day before a key piece of information is released. We find clear evidence that the contest, rather than information, is the prevailing motive for updates and accuracy improvement. Our theoretical model takes into account the dynamic structure of the forecasting problem, where accuracy improves both due to agents' effort in collecting information and due to the resolution of uncertainty during the forecasting period. Agents use a common forecasting model but face different cost-benefit ratios of effort. We structurally estimate the model to uncover the deep parameters characterizing the cost-benefit distribution across agents and how this changes on contest vs. normal days. The model fits the data well and the estimates allow us to perform counterfactual exercises on alternative survey designs.