Do you enjoy hunting around on the web for surveys to fill in? I do, but that’s because I’m researching a book on Surveys That Work and I believe that it’s rather unusual behaviour.
Most people need to be asked to complete a survey, so I talked about better invitations in my previous post.
What about a prenotice? It tells your respondents in advance that a survey is on its way: a pre-invitation, perhaps.
Some examples of prenotices
Prenotices can be anything from very simple to very elaborate. Some examples:
- An instructor asks a class to respond to a feedback survey about the quality of the teaching: person-to-person, word of mouth
- A university sends a letter to a sample of individuals to ask them to respond to an academic survey that will be sent separately: organisation-to-person, paper letter
- A national statistics organisation runs a series of TV advertisements to alert the general public to a census: organisation-to-person, broadcast. The UK 2011 Census campaign starred a purple bus with the slogan “Help tomorrow take shape”.
The Dillman recipe for a prenotice letter that increases response rates
Dillman, Smyth and Christian (2009) quote a number of experiments that tested prenotice letters and achieved an improvement of response rates between 3 and 6 percentage points. Their recipe for a prenotice letter is:
- Use the headed notepaper of a respected institution e.g. a university
- Write to a named person, or failing that a specific address
- Appeal for help with a specific project
- Describe what will happen in general terms
- Say that a small token of appreciation will be offered (but not exactly what it is)
- Say how much effort will be required of the respondent
- Close with the real signature of a respected, named individual
- Provide the full contact details of the named individual
- Send the prenotice letter about a week before the invitation.
One thing I’d add to this list:
- Offer the opportunity to receive the survey in alternative formats, such as large print for a paper survey.
Dillman, Smyth and Christian argue that prenotice letters work better than postcards because letters take longer to deal with than postcards, so have more chance of being remembered when the actual invitation arrives.
A prenotice letter that didn’t work
The general idea of a prenotice is that endorsement by a respected authority lends credibility to the survey, increasing trust, and therefore improving the response rate.
A group of researchers in Ontario, Canada and Minnesota, US, ran an experiment with a survey of orthopaedic surgeons (Bhandari et al. 2003). Half of the surgeons got a prenotice with endorsement of the study by 22 surgeons identified as opinion leaders; the others just got the survey. This time, the prenotice significantly reduced response rates.
The researchers speculate that ‘superspecialists’ might be perceived as arrogant or disrespectful; my own experience of one particular orthopaedic surgeon suggests to me that he would have been furious at any suggestion that his opinion was of lesser value than anyone else’s, and would have therefore perceived the endorsement as insulting. (One of my best decisions was to change to another surgeon, but that’s another story).
So if you do decide on a prenotice: test it! Make sure your named institution and person really are respected by the recipients.
What about prenotices for internet surveys?
You’ll see from the list above that this research all relates to prenotices on paper. I can see some advantages to sending a paper letter asking someone to look out for an email with a link to the survey.
I’m not at all sure whether there is any merit in sending an email to tell someone that they’ll be getting another email in a week. Personally, I’d be a bit annoyed that I’d opened the email just to learn that I had to remember to look out for, and open, another email in a week’s time.
This is a point made by Dillman, Smyth and Christian: They emphasise that each approach to the respondent needs to be as different as possible from the previous one. Example: If the main questionnaire will be a mailed packet, then make the prenotice a letter and the reminder a postcard.
Two letters in a row? Too similar. Two emails in succession? Definitely too similar.
I suspect that’s why national censuses are announced in other media such as the internet advertisement for the Australian Census in 2011 (below) and TV campaigns. Paper census invitation, prenotice in anything other than paper.
How to make an in-person prenotice more effective
I’ll assume that you don’t want to mount a major TV campaign to alert respondents to your survey. But what about a much more familiar situation: Asking attendees at an event to respond to a feedback survey? I’m going to stick to the simple example of teaching a workshop where there’s just one questionnaire to consider.
Before event organisers learned about the internet, we used to hand out the questionnaire right there at the end of each talk. It still works well if you can persuade them to let you do it, and the rules are simple:
- Finish a little early, so that attendees have some time to spare (reduces effort)
- Explain why you, personally, value the feedback (creates a reward – be nice to the speaker)
- Make sure that the questionnaire can be handed in anonymously, and carefully avoid reading the responses in front of other respondents (increases trust).
These days, many events have moved to sending out a post-event email. I’m sure I’m not the only person who has received a grudging prenotice of the survey from a speaker who clearly wasn’t interested in it, knew very little about it, and probably won’t pay much attention to it anyway. Result: a very bad response rate.
Even a good speaker will struggle to get attendees to focus attention on yet another piece of information – after they’ve spent an hour, or a day, or even longer trying to take in as much as they can.
The answer? I’m not sure yet, but here are some ideas that we could try – adapting the idea of changing format from Dillman:
- Get someone other than the speaker to come along and advertise the survey.
- Give out a handout with key details of the survey: the link, or the name and email address of the person who will send the link.
- Consider making the handout something that’s an incentive in its own right: maybe a pen. Or adapt an idea from Dillman and give people an envelope printed with the details and with a dollar bill inside. I’ve even wondered if it might work to adapt an idea from the US Census and hand out mugs.
References
1. Dillman, D. A., Smyth, J. D and Christian, L. M. (2009) “Internet, Mail and Mixed-Mode Surveys: the Tailored Design Method, Third Edition”, John Wiley and Sons Inc, Hoboken New Jersey
2. Bhandari, M., Devereaux, P., Swiontkowski, M. F., Schemitsch,
E. H., Shankardass, K., Sprague, S. and Guyatt, G. H. (2003). “A randomized trial of opinion leader endorsement in a survey of orthopaedic surgeons: effect on primary response rates.” International Journal of Epidemiology 32(4): 634-636.
#surveys #surveysthatwork