Evaluating Mental Health First Aid

Mental Health First Aid TUI University By April Showalter Capstone Integrative Project MHD 599 Mod 4 SLP Dr. Afrooz Afghani June 15, 2009 a public health education program

Evaluating the MHFA Program The mission of the program The vision of the program The objectives of the program Why evaluate? Types of evaluation What do we want to evaluate? Ways to evaluate the program Four levels of evaluation information Steps in planning an evaluation References

Mission the MHFA Program To help those in society who have a mental health problem or are in a mental health crisis. (Kitchener, 2008).

Vision of the MHFA Program Better recognition of mental disorders, Changed beliefs to a less negative/stigmatizing view of mental disorders, Decreased social distance from people with mental disorders, Increased help and understanding of those with mental disorders, and Increased confidence in helping a person with a mental disorder. (Kitchener, 2008).

The Objectives of MHFA Training the names of the most common MH problems; the signs & the symptoms of the common MH problems; the potential causes of MH problems; the risk factors for each MH problem; treatment options for each MH problem; action steps on how to help and/or get help; where to get outside help; when to get outside help; other resources about MH. (Kitchener, 2008) Each participant will learn...

Why Evaluate the MHFA Program? Provide accountability to your funders, stakeholders and the community. Improve program quality Provides feedback Evidence that the program is doing what it claims to do Help allocate resources What resources are needed? Are resources being used effectively? (Washington, 1999)

Types of Evaluation Process evaluation Measures and documents all program activity Impact evaluation Measures the impact the program had on participants Measures changes in knowledge, attitude, skills, behavior, policies, or the environment. Outcome evaluation Measures the outcome of the program Asks did the program have an effect on individual or community health? Measures the long term (Washington, 1999)

What Do We Want to Evaluate? Implementation: Were MHFA program activities put into place as originally intended? Effectiveness: Is the MHFA program achieving the goals and objectives it was intended to accomplish? Efficiency: Are the MHFA program’s activities being produced with appropriate use of resources such as budget and staff time? Cost-Effectiveness: Does the value or benefit of achieving the MHFA program’s goals & objectives exceed the cost of producing them? Attribution: Can progress on goals and objectives be shown to be related to the MHFA program,as opposed to other things that are going on at the same time? (CDC, 2005)

Ways to Evaluate the MHFA Program Questionnaires Pre- and post- training knowledge checks Interviews Focus groups Surveys – telephone, internet, mail Program documentation reviews Observation Case studies (Washington, 1999)

Four Levels of Evaluation Information... ...that can be gathered from clients: 1. Reactions and feelings (feelings are often poor indicators that your service made lasting impact) 2. Learning (enhanced attitudes, perceptions or knowledge) 3. Changes in skills (applied the learning to enhance behaviors) 4. Effectiveness (improved performance because of enhanced behaviors) (McNamara, 1997)

Types of Evaluation Design 1 Experimental design - use random assignment to compare the outcome of an intervention on one or more groups with an equivalent group or groups that did not receive the intervention. 2 Quasi-experimental design – make comparisons between nonequivalent groups and do not involve random assignment to intervention and control groups (in community settings it is hard, or sometimes even unethical, to have a true control) 3 Observational design - time–series analysis, cross-sectional surveys, and case studies. 4 Goal-based evaluation - uses predetermined program goals and the underlying program theory as the standards for evaluation, thus holding the program accountable to prior expectations. (CDC, 2005)

Steps in Planning an Evaluation Decide what you want to evaluate Select evaluation measures Design data collection and measurement tools Develop an action plan Share the results (Washington, 1999)

References Centers for Disease Control and Prevention. (2005). Introduction to program evaluation for public health programs: A self-study guide. U.S. Department of Health and Human Services. Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Atlanta, GA; 2005. June 13, 2009 from http://www.cdc.gov/eval/evalguide.pdf Kitchener, B.A. and Jorm, A.F. (2008). Mental Health First Aid Manual . ORYGEN Research Centre, University of Melbourne. Melbourne, Australia. Retrieved June 14, 2009 from http://www.mhfa.com.au/mhfa_manual.pdf McNamara, C. (2002). Basic Guide to Program Evaluation. Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation. Retrieved June 14, 2009 from http://www.managementhelp.org/evaluatn/fnl_eval.htm#anchor1586742 Washington State Department of Health. (1999). Evaluation: Are we making a difference? H.E.R.E. (Health Education Resource Exchange). Health Educator's Toolbox, Program Development and Evaluation . February 1999. Retrieved June 13, 2009 from https://fortress.wa.gov/doh/here/howto/images/eval.html

Evaluating Mental Health First Aid

More Related Content

Evaluating Mental Health First Aid

Editor's Notes