Focus group interviews were conducted to explore socio-cultural, environmental, and policy-relate... more Focus group interviews were conducted to explore socio-cultural, environmental, and policy-related determinants of physical activity among sedentary American Indian women. Thirty women aged 20 to 50 years (mean = 37.4 ± 10.6 years) participated. Three sessions were conducted with women aged 20 to 34 years and three with women aged 35 to 50 to evaluate response differences by age. Because
BackgroundMental health is a critical component of wellness. Public policies present an opportuni... more BackgroundMental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures.MethodGuided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995–2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale.ResultsWe identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n = 20, 57%) and other resources (n = 12, 34%), actor relationships/networks (n = 15, 43%), and organizational culture and climate (n = 11, 31%). Fidelity was the most prevalent implementation outcome (n = 9, 26%), followed by penetration (n = 8, 23%) and acceptability (n = 7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools.ConclusionsThis work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap.Plain Language SummaryMental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.
Objectives. There are sparse data showing the extent to which the process of evidence-based publi... more Objectives. There are sparse data showing the extent to which the process of evidence-based public health is occurring among local health departments (LHDs). The study objective was to describe the patterns and predictors of administrative evidence-based practices (structures and activities that are associated with performance) in a representative sample of LHDs. Methods. A cross-sectional study of 517 LHD directors was conducted. The questions on administrative evidence-based practices included 19 items based on a recent literature review (five domains: workforce development, leadership, organizational climate/culture, relationships/partnerships, financial processes). Results. There was a wide range in attainment of administrative evidence-based practices; mean values were lowest for organizational climate/culture (49.9%) and highest for relationships/partnerships (77.1%). Variables associated with attaining the highest tertile of administrative evidence-based practices included ha...
Objectives: Evidence-based decision making (EBDM) capacity in local public health departments is ... more Objectives: Evidence-based decision making (EBDM) capacity in local public health departments is foundational to meeting both organizational and individual competencies and fulfilling expanded roles. In addition to on-the-job training, organizational supports are needed to prepare staff; yet, less is known in this area. This qualitative study explores supportive management practices instituted as part of a training and technical assistance intervention. Design: This qualitative study used a semistructured interview guide to elicit participants' descriptions and perceptions via key informant interviews. Verbatim transcripts were coded and thematic analyses were conducted. Setting: Local public health departments in a US Midwestern state participated in the project. Participants: Seventeen middle managers and staff from 4 local health departments participated in remote, audio-recorded interviews. Intervention: Following delivery of a 3½-day in-person training, the study team met w...
IntroductionThe dissemination of evidence-based interventions (i.e., programs, practices, and pol... more IntroductionThe dissemination of evidence-based interventions (i.e., programs, practices, and policies) is a core function of US state health departments (SHDs). However, interventions are originally designed and tested with a specific population and context. Hence, adapting the intervention to meet the real-world circumstances and population's needs can increase the likelihood of achieving the expected health outcomes for the target population from the implemented intervention. This study identified how SHD employees decide to adapt public health programs and what influences decisions on how to adapt them.Materials and methodsSHD employees (n = 45) were interviewed using a qualitative semi-structured interview guide. Telephone interviews were audio-recorded and transcribed verbatim. The transcripts were consensus-coded and themes were identified using thematic analysis. Several themes aligned with the Model for Adaptation Design and Impact.ResultsData, outcomes, and health depa...
BackgroundLocal health departments (LHDs) in the United States are charged with preventing diseas... more BackgroundLocal health departments (LHDs) in the United States are charged with preventing disease and promoting health in their respective communities. Understanding and addressing what supports LHD's need to foster a climate and culture supportive of evidence-based decision making (EBDM) processes can enhance delivery of effective practices and services.MethodsWe employed a stepped-wedge trial design to test staggered delivery of implementation supports in 12 LHDs (Missouri, USA) to expand capacity for EBDM processes. The intervention was an in-person training in EBDM and continued support by the research team over 24 months (March 2018–February 2020). We used a mixed-methods approach to evaluate: (1) individuals' EBDM skills, (2) organizational supports for EBDM, and (3) administered evidence-based interventions. LHD staff completed a quantitative survey at 4 time points measuring their EBDM skills, organizational supports, and evidence-based interventions. We selected 4 ...
This case study provides an example of a collaboration between a university and a public health a... more This case study provides an example of a collaboration between a university and a public health agency to build organizational capacity to spread data-driven decision-making, implementation, and evaluation of evidence-based cancer prevention strategies. The Office of Chronic Disease Prevention and Health Promotion at the South Dakota Department of Health provided the key management practices for scale-up of evidence-based decision-making (EBDM): leadership support, training, a supportive organizational climate and culture, inclusion of partners, and outcomes-based contracting with partnering organizations. A pre–post survey showed increased use of research evidence for several job tasks, including selection of interventions and evaluation. Perceived work unit access to skills in prioritization and adapting interventions also increased. The 16 staff and partners interviewed perceived leadership support, federal funding requirements, and an initial multi-day training as the key facili...
Background: Cancer mortality rates in the United States are higher in rural than urban areas, esp... more Background: Cancer mortality rates in the United States are higher in rural than urban areas, especially for colorectal cancer. Modifiable cancer risks (e.g., tobacco use, obesity) are more prevalent among U.S. rural than urban residents. Social network analyses are common, yet rural informal collaborative networks for cancer prevention and control and practitioner uses of network findings are less well understood. Methods: In five service areas in rural Missouri and Illinois, we conducted a network survey of informal multisector networks among agencies that address cancer risk (N = 152 individuals). The survey asked about contact, collaborative activities, and referrals. We calculated descriptive network statistics and disseminated network visualizations with rural agencies through infographics and interactive Network Navigator platforms. We also collected feedback on uses of network findings from agency staff (N = 14). Results: Service areas had more connections (average degree) f...
Focus group interviews were conducted to explore socio-cultural, environmental, and policy-relate... more Focus group interviews were conducted to explore socio-cultural, environmental, and policy-related determinants of physical activity among sedentary American Indian women. Thirty women aged 20 to 50 years (mean = 37.4 ± 10.6 years) participated. Three sessions were conducted with women aged 20 to 34 years and three with women aged 35 to 50 to evaluate response differences by age. Because
BackgroundMental health is a critical component of wellness. Public policies present an opportuni... more BackgroundMental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures.MethodGuided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995–2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale.ResultsWe identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n = 20, 57%) and other resources (n = 12, 34%), actor relationships/networks (n = 15, 43%), and organizational culture and climate (n = 11, 31%). Fidelity was the most prevalent implementation outcome (n = 9, 26%), followed by penetration (n = 8, 23%) and acceptability (n = 7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools.ConclusionsThis work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap.Plain Language SummaryMental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.
Objectives. There are sparse data showing the extent to which the process of evidence-based publi... more Objectives. There are sparse data showing the extent to which the process of evidence-based public health is occurring among local health departments (LHDs). The study objective was to describe the patterns and predictors of administrative evidence-based practices (structures and activities that are associated with performance) in a representative sample of LHDs. Methods. A cross-sectional study of 517 LHD directors was conducted. The questions on administrative evidence-based practices included 19 items based on a recent literature review (five domains: workforce development, leadership, organizational climate/culture, relationships/partnerships, financial processes). Results. There was a wide range in attainment of administrative evidence-based practices; mean values were lowest for organizational climate/culture (49.9%) and highest for relationships/partnerships (77.1%). Variables associated with attaining the highest tertile of administrative evidence-based practices included ha...
Objectives: Evidence-based decision making (EBDM) capacity in local public health departments is ... more Objectives: Evidence-based decision making (EBDM) capacity in local public health departments is foundational to meeting both organizational and individual competencies and fulfilling expanded roles. In addition to on-the-job training, organizational supports are needed to prepare staff; yet, less is known in this area. This qualitative study explores supportive management practices instituted as part of a training and technical assistance intervention. Design: This qualitative study used a semistructured interview guide to elicit participants' descriptions and perceptions via key informant interviews. Verbatim transcripts were coded and thematic analyses were conducted. Setting: Local public health departments in a US Midwestern state participated in the project. Participants: Seventeen middle managers and staff from 4 local health departments participated in remote, audio-recorded interviews. Intervention: Following delivery of a 3½-day in-person training, the study team met w...
IntroductionThe dissemination of evidence-based interventions (i.e., programs, practices, and pol... more IntroductionThe dissemination of evidence-based interventions (i.e., programs, practices, and policies) is a core function of US state health departments (SHDs). However, interventions are originally designed and tested with a specific population and context. Hence, adapting the intervention to meet the real-world circumstances and population's needs can increase the likelihood of achieving the expected health outcomes for the target population from the implemented intervention. This study identified how SHD employees decide to adapt public health programs and what influences decisions on how to adapt them.Materials and methodsSHD employees (n = 45) were interviewed using a qualitative semi-structured interview guide. Telephone interviews were audio-recorded and transcribed verbatim. The transcripts were consensus-coded and themes were identified using thematic analysis. Several themes aligned with the Model for Adaptation Design and Impact.ResultsData, outcomes, and health depa...
BackgroundLocal health departments (LHDs) in the United States are charged with preventing diseas... more BackgroundLocal health departments (LHDs) in the United States are charged with preventing disease and promoting health in their respective communities. Understanding and addressing what supports LHD's need to foster a climate and culture supportive of evidence-based decision making (EBDM) processes can enhance delivery of effective practices and services.MethodsWe employed a stepped-wedge trial design to test staggered delivery of implementation supports in 12 LHDs (Missouri, USA) to expand capacity for EBDM processes. The intervention was an in-person training in EBDM and continued support by the research team over 24 months (March 2018–February 2020). We used a mixed-methods approach to evaluate: (1) individuals' EBDM skills, (2) organizational supports for EBDM, and (3) administered evidence-based interventions. LHD staff completed a quantitative survey at 4 time points measuring their EBDM skills, organizational supports, and evidence-based interventions. We selected 4 ...
This case study provides an example of a collaboration between a university and a public health a... more This case study provides an example of a collaboration between a university and a public health agency to build organizational capacity to spread data-driven decision-making, implementation, and evaluation of evidence-based cancer prevention strategies. The Office of Chronic Disease Prevention and Health Promotion at the South Dakota Department of Health provided the key management practices for scale-up of evidence-based decision-making (EBDM): leadership support, training, a supportive organizational climate and culture, inclusion of partners, and outcomes-based contracting with partnering organizations. A pre–post survey showed increased use of research evidence for several job tasks, including selection of interventions and evaluation. Perceived work unit access to skills in prioritization and adapting interventions also increased. The 16 staff and partners interviewed perceived leadership support, federal funding requirements, and an initial multi-day training as the key facili...
Background: Cancer mortality rates in the United States are higher in rural than urban areas, esp... more Background: Cancer mortality rates in the United States are higher in rural than urban areas, especially for colorectal cancer. Modifiable cancer risks (e.g., tobacco use, obesity) are more prevalent among U.S. rural than urban residents. Social network analyses are common, yet rural informal collaborative networks for cancer prevention and control and practitioner uses of network findings are less well understood. Methods: In five service areas in rural Missouri and Illinois, we conducted a network survey of informal multisector networks among agencies that address cancer risk (N = 152 individuals). The survey asked about contact, collaborative activities, and referrals. We calculated descriptive network statistics and disseminated network visualizations with rural agencies through infographics and interactive Network Navigator platforms. We also collected feedback on uses of network findings from agency staff (N = 14). Results: Service areas had more connections (average degree) f...
Uploads
Papers by Peg Allen