How EastSide Arts increased their survey response rates

“Is this a good response to my survey?” 

That’s one of the questions we regularly get here at thrive, and it’s a tougher question to answer than you might think. Too often, by the time we’re asked the question, the survey has closed and the thing you wanted to find out about happened weeks ago. By then, it’s too late to get reliable answers from more people. It’s what you do before and during the survey that often dictates how well you do.  

It is possible to say when you haven’t got enough responses though – drop below a certain bar and it’s unlikely that your findings will be reliable. As a rule of thumb, if you’ve got under 100 responses but a lot more than 100 people took part in whatever you’re researching (your event, using your website etc), the data you’ve gathered will be almost useless.

Think about it this way – when 100 people answer your survey, each person is worth 1% when you start trying to explain the data. The more you drop below 100, the more influence each individual response has on your results. Saying '20% of people did X' when only five people responded to your survey is very misleading because it only takes one person to change their answer for that to become 40%. When small numbers of people can make the data look very different, you end up with something that is at best useless and potentially dangerously misleading.

So what can you do to ensure you get enough responses to be genuinely useful? Here's how EastSide Arts improved their results, along with our top ten tips that you can implement too.

EastSide Arts Festival Survey

In Summer 2017, we worked with EastSide Arts to re-develop their annual post-festival survey for the EastSide Arts Festival and help increase response rates.

Previous surveys had between 40 to 50 responses each year, but by dedicating some time and effort to encourage more responses, EastSide were able to increase the response rates for their 2017 survey to 313 – an increase of over 500%.

How did they do it? Here are the ten things we did with EastSide that you can implement too to make sure your survey is a success.

1. Information is valuable, so prioritise getting it

On average, people don’t spend long reading emails. Figures vary (a recent study from Litmus put it at 11 seconds) but we’re talking seconds here, not minutes. So if your survey link is four articles in, and buried in the bottom left corner of your newsletter or post-festival round-up, you’re not going to get the response you need. If you only do one thing, this is the most important - dedicate an entire email to the survey. Give it the prominence and priority it deserves.

Our tip: Make sure your subject line also clearly shows that you’re giving them the option to have their say, rather than asking them to make a purchase.

Here's how EastSide Arts worded theirs: 

"Tell us what you thought: EastSide Arts Festival Survey 2017"

2. Send it out soon

Facts don’t really change, but memories and opinions shift and blur over time. If you want to know what someone thought of your event, a small gap before you ask can be good as it allows people to reflect and digest. Large gaps are not advisable though, as these could impact on how well they recollect their experience (known as recall error). The effort of trying to remember that far back may also make them less likely to respond at all.

Our tip: Like EastSide, plan ahead so that your survey is designed and ready to go well in advance.

3. Use a recognised bulk email account

Sending bulk emails to hundreds or thousands of people from Outlook is guaranteed to have your emails heading straight for the spam folder. Use a recognised email platform (e.g. MailChimp) to send the email out. That way, the email looks attractive enough to grab people’s attention and lets you see how many people have clicked on your link. 

4. Don’t waffle

The message inside the email should be short and snappy. Remember – you’ve only got seconds to grab their attention, so make it count.

Here's how EastSide worded their email:

"Tell us what you thought to be in with a chance to win £50 worth of Amazon vouchers.

If you made it along to this year’s EastSide Arts Festival, thanks for supporting us and it would be great to hear what you thought by taking 10 to 15 minutes to complete our survey below.

Your opinions help us understand what went well this year, and what we can do to make next year’s festival even better. If you didn’t make it along, we’d still welcome your thoughts on next year too.

Click here to complete the survey>>>"

5. Keep it simple

You want people to complete your survey, so keep that call to action front and centre, ideally as the focal point of your first sentence. Without going into too much detail, do also explain why it’s important for them to take part and what you will do with the results. You’re asking people to give their time to you, so reassure them that it won’t be a waste.

How EastSide did it:

"Thanks for taking the time to complete our 2017 Eastside Arts Festival survey. Your responses help us ensure we're delivering a great festival every year, and help us understand what we can do even better next year."

6. Tell people how long it will take

People are more likely to respond if they know how much time it will take, so tell them you’re not asking for War and Peace. Be truthful about the length though – if you’ve said five minutes and it’s more like 25, you’re going to annoy people and damage your relationship. It can also impact on your results, making responses more negative than they might have been and even causing people to abandon the survey part way through.

7. Include an incentive

The chance of winning something is often enough to entice someone to fill out your survey, so make sure to run a prize draw for completed entries. Depending on what you can afford, the bigger the prize on offer, the better the response you’ll see. Don’t go too big though, or you’re in danger of attracting respondents who are only interested in the prize.

Free tickets for a future show may be cheap, but you want people to be honest, particularly if someone didn’t enjoy their experience. Tickets, experiences or merchandise from your organisation may make people self-conscious about criticising you, or may not even be an incentive to them at all.

Our tip: Vouchers for Amazon or Marks and Spencer are often a safe bet.

8. Follow up with a reminder

Don’t rely on a single e-mail to prompt everybody to respond. If you’ve given people a couple of weeks to respond, a reminder mid-way through, and then again at the end is perfectly acceptable. If you’ve got a good email system, you should be able to segment your mailing list to send follow up emails to just those who haven’t yet opened and/or clicked the previous emails yet.

Our tip: Change your subject line on the follow up emails (e.g. add “Reminder” or “Last Chance” at the beginning of the line) and tweak the wording inside the email. A good approach is to add in something like “Thanks to all those who have already completed this year’s survey. If you haven’t had a chance to yet, there’s still time” at the start of your email.

9. Reach the right people

Who you ask to take part is often as important as what you ask. If you have contact details for everyone you’re interested in, then you shouldn’t need to do more than email them. However, if you don’t have contact details for everyone, you may need to cast your net wider. This is a particular issue for events like EastSide Arts Festival, where there are often free shows and multiple venues.

You can harness your own social media with regular posts about taking part in your survey. You can also work with your partners and peer organisations by asking them to promote your survey as well.

Our tip: Use distinct URLs when posting your survey in different places. You can use different links in your emails, on social media, and for your partner organisations. EastSide used SurveyMonkey which gave us the ability to track distinct URLs and find out which platforms worked best for sharing the survey.

10. Share the good news

While it won’t help with this year’s response, if your customers know that they helped make a change or gave a new idea then they’ll be more invested and likely to respond in future years. Via email and social media, let your customers and partners know they helped make a difference. 

Our tip: Don’t just focus on the numbers – show the impact. Have you secured next year’s funding on the back of this? Is it going to open up a new strand of programming? Are you exploring other venues or making improvements?

The End Result

We'll leave the last word to Rachel Kennedy, EastSide Arts Manager:

“It’s amazing what making just a few small changes to how we structure and send the survey can achieve. Gaining an insight into our customers that we know we can trust has been fantastic this year, and as a small team, this puts us in a great place to build from.

Not only can we plan more effectively for next year’s festival, but we’re committed to growing arts provision in East Belfast in the long term. With some help from thrive, this information has helped shape how we’re going to achieve that strategically.”

If you'd like us to work with you on creating your own survey success story, take a look at our market research products, or drop me an email to find out more.