Evaluating conferences and events: new approaches and initiatives
- 1. Evaluating conferences and events: new approaches and initiatives 10th EES Biennial Conference 5 October 2012 Helsinki, Finland Glenn O’Neil oneil@owlre.com www.owlre.com
- 2. The two case studies • Both conferences undertake evaluation • The conference evaluation differs in the number and type of evaluation tools used, in addition to aims and objectives
- 3. Approaches and ideas from two case studies The International AIDS Conference: biannual, over 20,000 participants The Lift technology conference: annual, some 700 participants
- 4. Commonly used evaluation tools for event/conference evaluation • Face-to-face or phone individual interviews • Focus groups • Online/printed surveys • Structured observations of key sessions and conference areas • Review of online resources/activities • Review of statistical data on conference registration • Comparison with previous conferences
- 5. The question of impact Some main questions posed by conference organisers and funders are: • What is the impact of a conference on participants? • What does it change in the way participants do their jobs?
- 6. International AIDS Conference & impact •Methods used: follow-up survey (online/face-to-face) and action plans Ex: AIDS 2008 follow-up survey (1.5 year after) 1,195 AIDS 2008 delegates completed the survey About 2/3 had learnt something new and had changed some aspects of their work practice Almost half reported that AIDS 2008 had directly influenced their organizations’ HIV work Almost 4 in 10 were aware of AIDS 2008’s influences on HIV work, policies or advocacy in their countries 75% had kept in contact with at least 1 person met Source: http://www.aids2008.org/admin/images/upload/AIDS_2008_follow_up_survey_report.pdf
- 7. Lift Conference and impact •Methods used: follow-up survey (online) and testimonies
- 8. Lift Conference and impact
- 9. Other emerging evaluation tools • Use of conference “instant” feedback systems • Use of the “mystery shopper” approach • Use of network analysis and mapping • Analysis of conference media coverage • Analysis of social media activity
- 10. Use of evaluation findings • Evaluation findings should be “very usable” as conferences are often repeated • Importance of “buy-in” of conference organizers • The quality of content and format of evaluation reports is crucial to attract readers and convince them that evaluation results are reliable and useable • Dissemination of evaluation results should be timely, and use a variety of channels depending on the target audience • Use of follow-up mechanisms with conference organizers and relevant stakeholders is recommended
- 11. Lessons from conference evaluation • How to deal with over-positive feedback • Evaluation findings can be used more for accountability and marketing rather than for learning and improvements • Challenges in getting adequate resources for conference evaluation • Challenges in moving beyond the basic satisfaction measures • Factoring in the importance of participants’ views
- 12. Q&A Thanks to Laetitia Lienart, former evaluation manager, IAS Join the Conference Evaluation Google Group: http://groups.google.com/group/conference_evaluation Download this presentation on my blog www.intelligentmeasurement.wordpress.com oneil@owlre.com @glenn_oneil glennoneil www.owlre.com 12