Have you ever looked at the reviews for a hotel or resort online? It’s frustrating. You’ll read a glowing review and think: “Let’s go there!” But the next one says the place is horrific and you think again: “Uh, let’s not.”
Online reviews confuse because they’re based on self-selected samples...participants volunteer to offer their “two cents.” So who makes the effort??? Those who are most satisfied or most upset. They’re not representative of all customers’ opinions!
Virtually all surveys are self-selected to some degree…researchers can’t force respondents to participate! But the further we get from a self-selected sample to one that is recruited – one we go out and seek – the closer we get to research that represents a population as a whole.
For decades, telephone surveys were the “gold standard” for recruiting and surveying respondents. Before cellphones, nearly every household had a landline. Random-digit dialing of landline numbers delivered a probability sample – that is, one where almost everyone has an equal chance of being selected, yielding data projectable to the total population.
Things change. Cellphones took off and the percentage of landline households began to drop. Today, the majority of U.S. households are cellphone-only, while fewer than 10% are landline-only. And cell-only households dominate among the young (for example, 71% of Millennials).
But users are typically resistant to spending extended time talking to a stranger on their cellphone for a survey. The kind of in-depth strategic surveys we specialize in typically take 10-20 minutes to complete…tough to get done on a cell!
So the emergence of cellphones and especially cell-only households has made telephone surveys increasingly difficult. Response rates have declined over the past twenty years. And lower response rates mean nonresponse bias – the possible difference between those who respond and those who don’t — is a concern.
And – bottom line – more difficult telephone surveys also mean they take longer to achieve their sample objectives and cost more.
Many turned to the internet as a faster, less-expensive alternative. But many online surveys are based on panels that respondents “opt in” to – that is, they volunteer to participate. For example, surveying your station’s “Loyal Listener Club,” may be a good PR move, but it’s still a self-selected sample – ardent fans…not representative of your audience as a whole.
Of course, there are situations where understanding the preferences of your “hardcores” is exactly what you want!
But when it comes to making big strategic decisions – like: What’s the best direction for your station? Should you consider a significant adjustment to your format? Should you opt for a different format entirely? – it’s crucial to get a broader, more representative view of the audience.
We’ve specialized in answering major strategic questions like these for decades, and telephone surveys have been our “go to.” But given the increased difficulty and expense of telephone surveys, we’ve progressed to a hybrid survey – one that involves telephone recruitment with a more convenient, less costly online option.
In a hybrid survey, every potential respondent gets a phone call – using a random sampling of both listed and unlisted, landline and cell. This is crucial because we’re recruiting participation by way of a real human being on the phone…more difficult to ignore than an email invitation that gets lost in their inbox. This telephone recruitment essentially delivers a probability sample projectable to the population as a whole.
Once we find qualified respondents, we can continue the survey on the phone or give them the option of completing the survey online. If they’d prefer to do it online, we email them the link and a secure login.
Both versions of the survey have identical questions. But the telephone versions have station lists – not read to respondents, but a convenience for interviewers to check off responses to questions like: “Which station has the best morning program?” The online version has no lists, because we’re seeking unaided responses…we don’t want to remind them about stations.
The hybrid approach also means extra work in tabulation, because those unaided, written-in responses have to be coded and merged into the data set.
But the added effort is well worth it, because anything that makes participation more convenient for respondents results in more participation, a more representative sample, less potential for non-response bias and lower costs. This approach is especially beneficial for improving response rates among younger listeners…always a challenge.
The hybrid approach isn’t a “magic wand”…real research is never (and never has been) easy or cheap. But when it comes to major decisions about your radio station, the synergy of telephone with online is helping us deliver true, representative insights into your station, your market and its opportunities.