Sir Humphrey teaches questionnaire design

Classic British TV comedy Yes Prime Minister has important lessons for those who want to interpret questionnaire data. This clip shows two civil servants discussing a policy suggestion. Bernard Woolley, who we see first, thinks the public are in favour of the policy – the minister has had an opinion poll done. Luckily senior civil servant, Sir Humphrey Appleby is there to set him straight:

Fans of cognitive biases, note that Sir Humphrey uses at least three in his illustration of a biased questionnaire: framing, priming, and acquiescence bias.

This example exaggerated, but the moral still holds : questionnaires can be designed to encourage the answers you want. People’s opinions are not objective facts like their height and weight, they change depending on the context and on how they are asked.