Comparing Alternative Survey Methodologies for Human Dimensions Research

Household surveys are one of the most commonly used methodologies in human dimensions research. More broadly, large-scale public surveys have played a foundational role in modern social science and their use has increased... [ view full abstract ]

Household surveys are one of the most commonly used methodologies in human dimensions research. More broadly, large-scale public surveys have played a foundational role in modern social science and their use has increased markedly in recent decades. However, current literature notes increasing challenges for the continued reliance on public survey data in the social sciences. Chief among these is the decline in response rates for more traditional survey modes (e.g., telephone and mail), elevating concern over the potential for non-response bias. Diminishing public participation in surveys and the associated rise in administration costs have led some investigators to seek alternatives. At the same time, advancing technologies provide new opportunities to increase efficiency and reduce data collection costs through online survey administration.

In light of current challenges and opportunities, the comparison of alternate survey methodologies represents a key human dimensions research need. We provide a case example from a longitudinal survey effort to understand public values toward wildlife in the United States. In this project’s pilot phase, we tested alternative survey modes (phone, mail, email panel) in South Dakota, USA. Measures of wildlife value orientations, wildlife-based recreation patterns (including license purchase behavior), and demographics were compared by survey method.

Estimates of participation in wildlife-based recreation suggest congruence with independent sources (National Survey of Fishing, Hunting, & Wildlife-Associated Recreation; South Dakota Game, Fish & Parks license sale data) for the three survey methods. Estimates of wildlife value orientation types varied more widely by survey method, indicating the possibility for method effects to confound longitudinal analysis, where baselines were established through mail survey. These findings led the research team to select a mail survey approach for the nationwide data collection. Results of this case study also suggest the need for increased attention to alternative survey methods to inform the design of long-range human dimensions research programs that rely on longitudinal data.