This document provides guidance on conducting focus groups to test marketing messages. It discusses defining objectives, designing the appropriate research approach, sampling relevant participants, developing a discussion guide, moderating focus groups, analyzing the results, and reporting findings. The key steps outlined are defining objectives, designing the research, recruiting appropriate samples, running focus groups with a discussion guide, analyzing the collected data, and reporting insights to inform decisions. Special considerations are given to internal research among staff to ensure candor and address potential biases. Common pitfalls like inadequate planning, timing, and recruitment are also addressed.
1. Testing your messages with focus groups
December 2010
Tel: 020 7426 8888
Email: michele.madden@nfpsynergy.net
Web: www.nfpsynergy.net
2. Purpose of session
• How can you make sure the messages in your marketing campaigns
are received and understood by the right people?
• Research design
• Deciding what is right for you
• Planning focus groups
• Running focus groups
2
3. The research process
Define Research Sampling &
objectives Design recruitment
Data
Analysis Reporting
collection
3
4. Define Research Sampling &
objectives Design recruitment
Data
Defining objectives
Analysis Reporting
collection
• Absolutely key to the success of your project
• What questions do you want to answer?
• What change will happen as a result?
• Do you have the resources?
- skills
- time
- money
• Planning, how does research fit in with your implementation
timetable?
- who else needs to be involved?
4
5. Setting your objectives
• Clearly defined purpose – shared • Loose sense of purpose or vague
with key players objectives
• Contingency/open-ended plans for
unexpected feedback • Pre-determined action plans with
insufficient flexibility to
incorporate stakeholder views
• Resources in place to put insights
into action • No resources for follow-up
o Eg adequate budgets, personnel o Eg insufficient staff resources
for implementation
Top tip: if you are finding it hard to agree on the key objectives ask each
individual for the 3 key things they want to find out and work from there.
5
6. Define Research Sampling &
objectives Design recruitment
Data
Research design
Analysis Reporting
collection
• Flexibility of methods/approach is critical
• Method of approach needs to reflect the relative investment, and
expertise, of the stakeholder group(s)
• Examples:
o external executives → in-depth telephone interviews
o elderly donors → accompanied in-home interviews
o staff groups → breakfast or lunchtime discussion groups
o low-literacy or ESL communities → computer-aided discussion of visual
stimulus
• Qualitative or quantitative?
6
7. Research design: Quantitative vs Qualitative
Quant Qual
• Quantity • Quality
• counts, quantifies data in different • explore perceptions, motivations
groups and attitudes
• if you want to be able to say that • how and why and deals in
"23% of people said yes to X", or concepts rather than numbers
"a quarter of the population say
they want X “
• roots in psychology, sociology,
semiotics, anthropology and
• Roots in maths, statistics, neurolinguistic programming
advertising
7
8. Other considerations
• Timescales
o When do you need to report back?
o Other timing issues (mailings, key meetings, public holidays)
• Budget
o Incentives
o Transcription
o Travel/expenses
• Resources
o Skills
o Staff time available
o Venue
8
9. Define Research Sampling &
objectives Design recruitment
Data
Who do you need to talk to?
Analysis Reporting
collection
• Who is the campaign/messages targeted at?
• Are there other audiences you need to consider?
• Where are these people?
- do you know them?
- do you have contact details?
- where do they live?
- what is their situation?
• Sampling
- what are the key characteristics of your sample?
- how many from each group?
- in the same groups or separated?
9
10. Recruitment
• Do not underestimate how long this takes!
• Telephone, email or letter
o Allow people sufficient time to respond – but not too long
• Clarify the purpose of the research from the outset
• Be clear what you are asking people to do
• Be clear what will happen with the results
• Incentives/expenses
10
11. Discussion Guide
A guide will have 5 main sections;
1. Introduction – this is essential as it sets up the group or interview, explains
the purpose, rules and parameters of the discussion
2. Warm up – to get everyone talking and building relationships within the group
(or with the researcher if an interview)
3. Warming up to the specific – questions that are easy for the respondent to
answer but that are moving on to the topic in question
4. The Specific – the specific information or questions that the research was
commissioned for. Probably the longest section of the group, and may include
stimulus materials and/or exercises
5. Warm down/conclusion – signalling the end of the group, summarising what
has been learnt, saying thank you.
11
13. Running a focus group?
• What do you need to think about?
13
14. The role of the moderator
• To run a structured discussion to find the answers/responses
• To create a safe, open environment for discussion
• Listen to everyone, encourage all to speak
• Allow time and space for people to respond
• Protect and control if necessary
• Be objective, fair and open
• Keep to time
14
15. Essentials and practicalities
• Essentials
o Obtain consent for research with adults
o Children and young people
o Data protection
o Discussion guide that covers objectives
o Stimulus materials
• Practicalities
o Venues and viewing facilities
o Recordings digital/video
o Refreshments
15
16. Special considerations for research among your own
staff
• Consulting staff from an early stage is a key part of an
organisational change journey
• Encourage candour: take extra steps to ensure anonymity or
confidentiality
• ‘Silo syndrome’ – remember that staff may have extensive expertise
in one area, but also be less aware of other aspects of the
organisation or competing priorities
• Communicate findings and decisions about ideas that are taken up,
and those that are rejected/deferred
16
17. Define Research Sampling &
objectives Design recruitment
Data
Analysis
Analysis Reporting
collection
• Analyse different methods and audiences separately, then pool the
findings
• If possible, have a discussion with someone else who was at the
groups to test your thinking
• Look for themes, then divide these into sub-themes
• Objectivity is key, leave your personal opinions at the door
• Don’t underestimate how much time it takes to do this properly
17
18. Define Research Sampling &
objectives Design recruitment
Data
Reporting
Analysis Reporting
collection
• When using quotes ensure they are representative of the data and
not the ‘strongest’ or ‘weakest’ example of someone’s feelings
• The order of the story may not be that of the discussion guide –
follow whichever makes most sense
• Prioritise findings and recommendations – return to original
objectives to sense-check
• Recommendations may be at different levels – this is fine
18
20. The pros and cons of carrying out the research
internally
Benefits Drawbacks
• Knowledge of • May make assumptions
topic/internal context about the options/context
• May be more cost • Can be more time-
efficient consuming than expected
• High levels of enthusiasm • Lack of expertise/options
about research methods
and analysis
20
21. The pros and cons of carrying out the research with
an external provider
Benefits Drawbacks
• Objective viewpoint: open to • Can be steep learning curve:
hearing different/unexpected organisation needs to commit
views
to bringing agency up to speed
• ‘Outsiders’ may elicit more
candid feedback from • Organisation needs to ‘validate’
participants the external provider’s role
• Expenditure may save internal
staff time/resources • May appear expensive/an
additional budget
• Experts in research methods consideration
options and analysis
21
22. The 5 things we most often see go wrong…
1. Not planning properly – research is not just about the focus group,
the thinking before and after are more important
2. Not building the research into internal timetables – planning,
mailing, budgeting
3. Overestimating how many people will be willing to take part –
ensure sufficient resources are available to recruit
4. Skewed recruitment – need to work hard to get people to
attend/take part who are representative of the wider group
5. Underestimating how long analysis takes
22
23. Key questions to ask yourself
• Are you clear about what and why you’re doing the research?
• Does it fit with your planning cycle?
• Do you have the money, resources and internal political support to
make changes as a result of the research?
• Do you already know the answer? Have you already made the
decision? Are the materials already printed?
23