Review
Public Health Genomics 2009;12:233–244
DOI: 10.1159/000203779
Received: July 3, 2008
Accepted after revision: December 3, 2008
Published online: February 20, 2009
A Health Services Research Agenda
for Cellular, Molecular and Genomic
Technologies in Cancer Care
Louise Wideroff a Kathryn A. Phillips b Gurvaneet Randhawa c Anita Ambs a
Katrina Armstrong d Charles L. Bennett e Martin L. Brown a Molla S. Donaldson f
Michele Follen g Sue J. Goldie h Robert A. Hiatt b Muin J. Khoury j Graham Lewis p
Howard L. McLeod k Margaret Piper l Isaac Powell m Deborah Schrag i Kevin A. Schulman n
Joan Scott o
a
National Cancer Institute, Bethesda, Md., b University of California at San Francisco, San Francisco, Calif.,
Agency for Healthcare Research and Quality, Rockville, Md., d University of Pennsylvania, Philadelphia, Pa.,
e
Northwestern University, Evanston, Ill., f George Washington University, Washington, D.C., g M. D. Anderson
Cancer Center, Houston, Tex., h Harvard University and i Dana Farber Cancer Institute, Boston, Mass., j Centers for
Disease Control and Prevention, Atlanta, Ga., k University of North Carolina, Chapel Hill, N.C., l Blue Cross Blue Shield
Association Technology Evaluation Center, Chicago, Ill., m Wayne State University, Detroit, Mich., n Duke University,
Durham, N.C., and o Johns Hopkins University, Baltimore, Md., USA; p University of York, Heslington, UK
c
Key Words
Genomics ⴢ Health services research ⴢ Emerging
technologies ⴢ Translational research
Abstract
Background: In recent decades, extensive resources have
been invested to develop cellular, molecular and genomic
technologies with clinical applications that span the continuum of cancer care. Methods: In December 2006, the National Cancer Institute sponsored the first workshop to
uniquely examine the state of health services research on
cancer-related cellular, molecular and genomic technologies and identify challenges and priorities for expanding the
evidence base on their effectiveness in routine care. Results:
This article summarizes the workshop outcomes, which included development of a comprehensive research agenda
that incorporates health and safety endpoints, utilization
patterns, patient and provider preferences, quality of care
© 2009 S. Karger AG, Basel
1662–4246/09/0124–0233$26.00/0
Fax +41 61 306 12 34
E-Mail karger@karger.ch
www.karger.com
Accessible online at:
www.karger.com/phg
and access, disparities, economics and decision modeling,
trends in cancer outcomes, and health-related quality of life
among target populations. Conclusions: Ultimately, the successful adoption of useful technologies will depend on understanding and influencing the patient, provider, health
care system and societal factors that contribute to their uptake and effectiveness in ‘real-world’ settings.
Copyright © 2009 S. Karger AG, Basel
Introduction
Within the past 30 years, extensive resources have
been invested to develop cellular, molecular and genomic (CMG) technologies with clinical applications that
span the continuum of cancer care [1]. The capacity for
innovation stems largely from our rapidly expanding
knowledge base in molecular carcinogenesis [2] and biotechnology. However, there are many challenges to transLouise Wideroff
Epidemiology Research Branch, Division of Epidemiology, Services,
and Prevention Research, National Institute on Drug Abuse
6001 Executive Blvd., Suite 5153 MSC 9589, Bethesda, MD 20892-9589 (USA)
Tel. +1 301 451 8663, Fax +1 301 443 2636, E-Mail Wideroff@nih.gov
lating this knowledge into clinically useful and well-implemented interventions for cancer prevention, early detection, diagnosis and treatment [3].
The National Institutes of Health (NIH) have emphasized that translation and integration of new technologies
are major priorities. Currently, 2 of 4 components in the
NIH Core Strategic Vision directly address integration of
research innovations into health care delivery, emphasizing the need to ‘accelerate translation of findings from the
bench to the bedside to the community’ as well as ‘provide
the evidence and knowledge base to allow for a rational
transformation of our healthcare system’ (http://www.
nih.gov/about/director/newsletter/Spring2007.htm).
In December 2006, the National Cancer Institute
(NCI) sponsored the first workshop of researchers to focus on the delivery of CMG interventions in cancer care.
The goals were to examine the current state of health
services research (HSR) on CMG interventions and to
identify research priorities and challenges to expand our
limited knowledge base about their effectiveness in ‘realworld’ settings. Participants were primarily from academia and government, with expertise in HSR, including
clinical decision making, economics, clinical and translational medicine, technology assessment, policy, and
other areas. The workshop addressed cutting edge issues
from a wide range of perspectives and also developed
recommendations for advancing a comprehensive HSR
agenda on cancer-related CMG interventions. The objective of this report is to summarize the main workshop
conclusions, beginning with a discussion of knowledge
gaps surrounding the use of CMG interventions, followed
by research needs and challenges, and ending with recommendations for a comprehensive HSR agenda to advance our understanding of their effectiveness. Several
relevant areas are not addressed in this report because of
limited coverage during the workshop. Their importance,
however, is acknowledged in the section on workshop
limitations.
lists the main areas of HSR, as described by the Blue Cross
Blue Shield Association (http://www.bcbs.com/about/
foundation/health-services-research-definition.html?
templateName=template-28719196&print=t).
Box 1: Components of Health Services Research
• Costs, cost-effectiveness, cost-benefit and other economic aspects of health care
• Patient and population health status/quality of life
• Outcomes of health care technologies/interventions
• Practice patterns and diffusion of technologies/interventions
• Quality assurance programs/techniques designed to
test generalizable attributes
• Guidelines, standards and criteria for health care
• Patient compliance with treatment
• Need and demand for health care
• Availability and accessibility of health care
• Utilization of health care
• Patient preferences for treatments, providers, settings and so forth
• Organization and delivery of health care (for example, managed care vs. fee for service)
• Health care workforce
• Financing of health care (for example, public and private third-party payment, capitation)
• Health care administration and management
• Health education and patient instruction
• Health professions education
• Health planning and forecasting
• Legal and regulatory changes affecting the health
care system (for example, anti-trust laws)
• Data and information needed for health care decision making (for example, report cards)
• Studies of whether new health care technologies/interventions (including randomized controlled trials)
can produce a desired outcome in real-world settings
of general or routine clinical practice
The Scope of HSR
HSR is defined by AcademyHealth (the primary professional organization for HSR) as ‘the multidisciplinary
field of scientific investigation that studies how societal
factors, financing systems, organizational structures and
processes, health technologies, and personal behaviors
affect access to health care, the quality and cost of health
care, and ultimately, our health and well-being’ (http://
www.academyhealth.org/about/whatishsr.htm). Box 1
234
Public Health Genomics 2009;12:233–244
The traditional methods employed in HSR to evaluate
familiar interventions like mammography or chemotherapy are, for the most part, applicable to research on
CMG technologies. However, the latter may have distinguishing features that require new methodologies or adaptations to existing ones. Such features may include:
limited insurance coverage (for example, genetic counseling) and inadequate coding systems (for example, laboratory genetic testing) that make it difficult to track or
Wideroff et al.
identify an intervention in claims databases, the need to
analyze and condense massive amounts of data points
(for example, microarray-based gene polymorphism and
expression assays), low power due to risk stratification of
patients into multiple groups (for example, pharmacogenetics), rapid translation to clinical use with little clinical
outcome data (for example, laboratory genetic testing),
heightened concerns about genetic discrimination, and
the complexities inherent to studying families rather
than individuals (for example, predictive genetic testing
for cancer risk assessment). Clearly, the strategies for HSR
will vary by technology.
The Scope of CMG Technologies in Cancer Care
Public and private funding has supported extensive
research to develop a broad spectrum of CMG clinical
applications, although progress has been uneven across
cancer sites and technologies. Applications in cancer care
range from primary prevention and early detection of
malignancy to confirming diagnosis, selecting type and
dose of treatment, and monitoring treatment effectiveness. Commercially available tools for primary prevention include predictive genetic tests for hereditary highrisk mutations [4], nucleic acid and protein-based tests for
oncogenic infectious agents [5–7], and recombinant prophylactic vaccines against human papillomavirus [8] and
hepatitis B [9].
There has also been considerable research on biomarker tests for early detection of cancer, where the challenges of developing highly sensitive and specific tests
have limited the number of clinical applications currently in practice [10, 11]. A recent ‘horizon scan’ on genetic
tests for cancer performed by the Agency for Healthcare
Research and Quality (AHRQ) technology assessment
program lists 11 screening tests that rely on DNA or protein-based technologies, including the widely used prostate-specific antigen test (http://www.ahrq.gov/clinic/ta/
gentests/gentests.pdf).
The majority of cancer biomarker tests, including 54
listed in the AHRQ report, are used for the diagnosis and/
or treatment of cancer. Some diagnostic tests identify abnormal gene expression in tumors that may respond to
targeted therapies, such as trastuzumab for HER2-positive breast cancers [12]. The BCR-ABL gene test can be
used to diagnose chronic myelogenous leukemia as well
as monitor response to the tyrosine kinase inhibitor drug
imatinib that targets the BCR-ABL protein [13]. Several
gene expression-profiling tests detect and analyze patResearch Agenda for CMG Technologies
in Cancer Care
terns of multiple genes expressed in early-stage breast tumors, providing a prognostic score to identify potentially
more aggressive tumors for which adjuvant chemotherapy is indicated [14, 15]. Other genetic tests identify germline polymorphisms associated with drug toxicity or
nonresponse, enabling early dose adjustment or selection
of alternative therapies [16, 17].
CMG applications also include targeted and supportive therapies to treat cancer. Targeted therapies, such as
monoclonal antibodies or drugs, are designed to interfere
with specific molecular pathways important to the growth
and survival of tumor cells. Several targeted therapies that
inhibit kinase enzyme activity (such as trastuzumab, imatinib, erlotinib in epidermal growth factor receptor-positive non-small-cell lung cancer) have received Food and
Drug Administration (FDA) approval and are in clinical
use. Promising approaches to targeted drug delivery include chemotherapy molecules conjugated to natural or
synthetic nanoparticles, which are efficiently taken up by
tumor cells with fewer toxic side effects than drug formulations requiring solvents [18–20]. Supportive care therapies, such as hematopoietic stem cell transplants and
growth factor therapies, are used to replace stem cells or
stimulate production of red and white blood cells destroyed by chemotherapy or radiotherapy [21–23].
Functional imaging and optical technologies have diverse applications, ranging from detection of early lesions
and metastatic disease to monitoring response to treatment. In vivo functional imaging techniques enable visualization of molecular pathway activities, enhancing
characterization of the disease state when used in conjunction with structural imaging data [24, 25].
Not unexpectedly, the number of CMG innovations in
the developmental pipeline that are described in the literature exceeds the number currently used in clinical
care. Although a strong scientific rationale often exists
for their use, some technologies are not sufficiently developed to yield clinically useful results or improvements
over existing interventions [26]. Nevertheless, contextual
factors, such as direct-to-consumer and provider marketing [27, 28], press coverage, or advocacy and lobbying activities, may drive their diffusion into clinical practice
before their safety and efficacy are well understood.
Gaps in Efficacy Research
A major concern for many emerging CMG interventions is the limited amount of published premarketing
clinical data about safety and efficacy, which provides the
Public Health Genomics 2009;12:233–244
235
basic evidence that new technologies have net benefit
(that is, improve health outcomes). Premarketing safety
and efficacy data are evaluated in technology assessments, which are critical for decision making about appropriate clinical uses of new technologies, coverage and
reimbursement by insurers, and evidence-based guidelines. Technology assessments also identify existing gaps
in both premarketing efficacy data and real-world effectiveness data. Gaps in premarketing research on CMG
technologies are summarized in box 2. Three areas with
significant gaps include targeted therapies, screening interventions and gene-based tests.
Box 2: Research Gaps for CMG Technologies
Limited published data on analytic and clinical validity as well as clinical utility of diagnostic and predictive tests, and devices:
• Lack of gold standards with which to assess analytic
validity
• Studies of analytic validity often not published or
otherwise publicly available
• Studies of clinical validity often small and not designed to yield definitive results
• Studies on clinical utility (that is, impact of diagnostic tests on subsequent treatment decisions or patient
outcomes) seldom done
• In the United States, evidentiary standards for regulatory (that is, FDA) approval of laboratory tests and
devices are less stringent than those for approval of
therapeutics; in addition, laboratory tests may avoid
FDA regulation entirely if developed in a laboratory
and offered as a service by the same lab
Limited published randomized controlled trial data
on safety and efficacy of therapeutics:
• Studies often have small sample size, restrictive inclusion criteria, short duration and surrogate outcomes
• Concerns about market segmentation may serve as
disincentives for conducting pharmacogenetic drug
trials
Limited data on outcomes and effectiveness in routine clinical care:
• Standards of care needed in the early stages of diffusion cannot be developed in the absence of clinical
data
236
Public Health Genomics 2009;12:233–244
• Minimal HSR has been done to examine whether appropriate levels of care are being provided, what the
determinants are of disparities in outcomes and care,
and what organizational or other changes are needed
to optimize care
• Minimal surveillance has been conducted to monitor
long-term outcomes of new interventions related to
health, safety and quality of life
• Minimal economic studies have been done to assess
incremental costs and cost-effectiveness of new technologies
With respect to therapeutics, FDA approval requires
data from randomized controlled trials (RCTs) to establish the safety and efficacy of new oncology drugs and
biological therapies. Yet, RCTs of novel therapeutics often
lack the power to detect uncommon adverse events and
are not designed to assess off-label uses [29]. Short trial
duration and use of surrogate markers as primary endpoints can also limit understanding of therapeutic outcomes. Furthermore, the generalizability of trial results
may be affected by restrictive inclusion criteria that cause
key demographic subgroups to be underrepresented. Unfortunately, these subgroups, such as elderly patients or
those with comorbidities, may actually comprise a large
proportion of users in routine care. In addition, incentives to assess pharmacogenomic differences in drug response may be tempered by industry concerns about
market segmentation if fewer patients are found to benefit from treatment [30].
Gene-based tests to detect abnormalities in a single
gene or a broader array of genes represent an area of
growing importance for treatment decision making. Data
from observational studies are typically used to obtain
FDA approval for in vitro diagnostic test kits. However,
laboratory-developed tests that are not marketed as kits,
such as breast cancer (BRCA) tests for breast/ovarian
cancer risk assessment and OncotypeDXTM for breast
cancer prognosis, are not currently subject to FDA clearance. In contrast to test kits, laboratory-developed tests
may enter the market with minimal published information about analytic and clinical validity [31, 32].
With the widespread availability of different highthroughput genomic and proteomic platforms and analytic techniques, and the lack of a gold standard for comparison to genomic expression data (or with current gold
standards that are not adequately sensitive and/or specific), it can be difficult to establish analytic validity of a
test. Furthermore, although the association of new biomarkers with patient outcomes (that is, clinical validity)
Wideroff et al.
often provides the incentive for test development, the predictive value of a test is likely to differ in high- versus lowrisk populations, thus requiring costly large samples to
assess genetic, demographic or other variation. Other issues can limit validation studies as well. As an example,
a recent AHRQ evidence report on genomic tests for
ovarian cancer found that studies had small sample sizes
and spectrum biases, which affected the estimates of test
sensitivity and specificity. These studies also had unrealistically high disease prevalence, which affected predictive value (http://www.ahrq.gov/clinic/tp/genovctp.
htm).
Finally, validation studies alone cannot assess the clinical utility of diagnostic tests, which addresses test impact
on subsequent treatment decisions and patient outcomes.
Most tests lack information on clinical utility when first
introduced into clinical practice. The same AHRQ evidence report on ovarian cancer found that the use of genomic tests to guide management decisions, and improvements in patient outcomes (compared to standard
management), had not been evaluated in the context of
diagnosis, or primary or secondary prevention. These
limitations are not unique to genomic tests. Validation
studies of optical technologies and molecular imaging
devices also require costly large samples to examine
sources of variability, and there are similar challenges to
selecting appropriate gold standards and surrogate endpoints [33, 34].
Gaps in Effectiveness Research
Even with promising premarket safety and efficacy
data about many CMG interventions, the actual net benefit observed in real-world settings is typically less than
the expected net benefit obtained from RCTs, for several
reasons. First, the former includes a more diverse, often
sicker, patient population exhibiting less adherence and
persistence with therapy, drug-drug interactions or pharmacogenetic factors, and variations in delivery of care
due to provider practices, coverage and reimbursement,
or access. Furthermore, the short-term surrogate endpoints examined in RCTs may correlate imperfectly (or
even poorly) with long-term health outcomes, especially
if the natural history of the disease is not well understood
or the clinical course is heterogeneous. The potential for
differences between RCT and clinical practice results underscores the need to monitor the effectiveness of CMG
interventions in routine care. Research on effectiveness
also evaluates whether appropriate care is being provided
Research Agenda for CMG Technologies
in Cancer Care
to those who need it, resulting in improved health and
health-related quality of life.
Gaps in effectiveness research are summarized in box
2. For many CMG interventions, relatively few studies
have examined uptake and patterns of care or associated
clinical outcomes in routine care [35]. Similarly, the incremental costs, life expectancy and quality-adjusted life
expectancy gains associated with CMG interventions,
relative to clinical practice without their use, have not
been evaluated; these will be necessary to conduct economic evaluations, such as cost-effectiveness and costutility analyses [36].
One explanation for knowledge gaps about effectiveness is that funded cancer research focusing on the delivery of CMG interventions has been minimal. In a search
of the NCI cancer research portfolio database (http://
researchportfolio.cancer.gov/) from 2000 to 2006, only
12 HSR or related grants were identified that focused on
delivery of care involving CMG technologies [Ambs and
Wideroff, pers. commun.]. These grants, which totaled
USD 12 million, were found by independently entering
19 biotechnology search terms (such as genetic tests,
pharmacogenetics, biological therapies, cancer vaccines
and nanotechnology) and restricting results to grants
and contracts with research-type codes for patient care
and survivorship, surveillance, cost analyses, health care
delivery, education, communication, resources or infrastructure. No additional grants were found when the
search term ‘health disparities’ was added to each technology. One caveat is that projects conducted within large
research infrastructures, such as cancer centers or P01
grants, cannot be individually identified in abstracts,
which possibly results in some underestimation. Nevertheless, to put this in perspective, total NCI research expenditures for the same period were approximately USD
18.29 billion (NCI Annual Fact Book, http://obf.cancer.
gov/financial/factbook.htm; see 2001 link: Total Research
Grants, page 6/iv, for fiscal year 2000 and 2001 expenditures; see 2003 link: Total Research Grants, page 7/v, for
2002 and 2003; see 2004 link: Subtotal Research Grants,
page 41/B-4, for 2004; see 2005 link: Subtotal Research
Grants, pg. 53/B-4, for 2005; see 2006 link: Subtotal Research Grants, page 53/B-4, for 2006).
How to Address HSR Gaps
A strong HSR agenda is needed to evaluate clinical
utility and reduce uncertainty about the net benefits of
CMG interventions. Box 3 summarizes ways that HSR
Public Health Genomics 2009;12:233–244
237
can contribute to the knowledge base for CMG interventions. The HSR agenda should include postmarketing
surveillance to systematically monitor adverse events of
new therapies, devices and tests. Such research is needed to determine the balance of risks versus benefits of
emerging CMG interventions among the heterogeneous
populations that receive care in community settings
(http://www.nap.edu/catalog.php?record_id=11897#toc).
For example, postmarketing surveillance research has
yielded valuable information about the risks and benefits
of hematopoietic growth factors used in association with
adjuvant chemotherapy [37, 38].
Box 3: Ways to Address Research Gaps in CMG
Technologies
Conduct postmarketing surveillance to systematically assess CMG technologies within well-defined target populations:
• Monitor clinical endpoints to assess safety and health
outcomes
• Monitor patient-reported outcomes, such as functional impact and psychosocial and economic burden
Broaden research to evaluate the effectiveness and
clinical utility of CMG technologies, by determining if
appropriate care is being delivered to the target populations, with improvements in health and health-related
quality of life:
• Assess prevalence and determinants of patient and
provider use, knowledge, and attitudes
• Monitor trends in health plan and government policy toward coverage
• Determine economic impact, including incremental
cost, cost-effectiveness and cost-utility
• Assess the multiple dimensions of access to care, particularly among medically underserved and target
populations (that is, affordability, availability, acceptability, access by target populations, and accommodation of practice-to-patient constraints and preferences)
• Determine the occurrence, determinants and consequences of disparities in care, and test interventions
to reduce or eliminate them
• Evaluate net benefit based on population trends in
incidence, mortality, survival, and quality of life
238
Public Health Genomics 2009;12:233–244
Expand use of decision analysis in premarketing and
postmarketing research to model the effectiveness of
CMG technologies that add to or substitute existing
standards of care:
• Change the model parameters to assess relative costs
and benefits of interventions
• Compare interventions in populations with varying
economic and health care resources and sociocultural practices
Yet, HSR must go well beyond postmarketing surveillance to assess the adoption and impact of emerging
CMG interventions at the patient, provider, health system and population levels. A wide range of outcomes
should be studied, including patient-reported quality of
life indicators (such as functional impact, psychosocial
and economic burden) [39], trends in patient and provider uptake and use, knowledge, attitudes and other determinants that might facilitate or hinder adoption and
appropriate use in specific settings [40], trends in health
plan and government policy toward coverage and regulation, incremental cost, cost-effectiveness and cost-utility
[41, 42], and net benefit based on population trends in
incidence, mortality, survival and quality of life [43].
The knowledge obtained from this approach would be
useful for defining standards of care and measuring the
quality of care involving CMG interventions. In this regard, the Institute of Medicine described 6 aims for health
care: safety, effectiveness, patient-centeredness, timeliness, efficiency and equity (http://www.iom.edu/CMS/
8089/5432.aspx). To measure how well those goals are
being met, a link must be made between a given process
of care, such as administration of a genomic test or treatment, and an outcome. These links can be difficult to
establish. Additionally, the development and testing of
quality measures is resource intensive, particularly for
interventions that improve decision making and knowledge (http://mdm.sagepub.com/content/vol27/issue5/)
[44, 45]. However, these links are essential for efficient
identification, adoption and promotion of new CMG
technologies that can positively impact patients’ lives.
Access to health care will remain an important factor
in improving overall population health care quality and
outcomes. Broadly defined, access covers the dimensions
of affordability, availability of services and their access by
the target population, acceptability to patients, providers
and the health care system, and accommodation of practice to patient constraints and preferences [46, 47]. Although there is currently little research about access to
most CMG technologies, many of the issues are very relWideroff et al.
evant [36]. High costs and limited insurance coverage
may impose particular constraints on affordability. For
example, biological therapies currently represent some of
the highest expenditures in cancer treatment [48]. The
monoclonal antibody trastuzumab, indicated for metastatic HER2-positive breast cancer [49], was estimated to
cost USD 50,000 per year for a 70-kg woman [50], which
potentially limits access for uninsured and underinsured
patients.
Availability of health services is another notable constraint to access, particularly for medical specialties with
existing or projected work force supply deficits, such as
genetic counseling and oncology [51, 52]. With respect to
genomic tests, it will be important to have access to highquality clinical labs that deliver results rapidly, so that the
results can influence treatment decisions as near to real
time as feasible. This is particularly challenging for single-source tests offered by labs that conduct only 1 or a
few tests and do not have an established shipping network. At the patient level, cultural and other factors, perhaps undefined, may also influence access. One study
noted a 5-fold higher uptake of genetic counseling for
breast cancer 1 and 2 testing in white women, relative to
African Americans, that was not explained by socioeconomic or behavioral variables [53].
Inequities in access to care due to socioeconomic, geographic, cultural, racial-ethnic or other factors can ultimately lead to disparities in cancer incidence, mortality
and health-related quality of life [54]. Therefore, studies
that describe the occurrence, determinants and consequences of disparities, as well as those that test strategies
to reduce or eliminate disparities, should be an integral
part of an HSR agenda [55]. Although some research has
addressed disparities in genetic testing and follow-up
care [56], little is currently known about disparities in
utilization of most CMG technologies.
Decision-analytic modeling can play an important
role in evaluating the incremental costs and cost-effectiveness of an emerging CMG intervention [57]. In a
model-based decision analysis, information about the
disease, effectiveness of treatment, performance and
costs of both new and existing technologies are combined
with other relevant demographic and epidemiological
characteristics of a target population. These models can
be used to comparatively assess the relative costs and benefits of technologies that add to or substitute existing
standards of care [58, 59]. Decision analysis can also be
used to compare public health strategies in countries or
populations with varying economic and health care resources and sociocultural practices [60].
Research Agenda for CMG Technologies
in Cancer Care
Challenges in Addressing Gaps
Although the rationale to expand HSR on emerging
CMG technologies is clear, there is less clarity about how
to do this with the available resources. Administrative
claims databases have been a mainstay of HSR [61], but
may require substantial adaptations to accommodate
CMG information, such as: specific standardized coding for new test procedures, and information on patient
diagnosis and intent of the test; electronic linkages to additional data sources; incorporation of genetic and family history data. Cancer registries may need to be modified to allow their use for HSR research on CMG
technologies, by incorporating data on tumor biomarkers, pharmacy claims and other data elements [62]. Images and multiplex molecular test results may need to be
incorporated into electronic medical records systems
[63], which are already facing the complexities of interoperability across systems as well as tensions between patient privacy and accessibility to researchers [64, 65]. The
more expansive the data required to personalize care
based on hereditary and molecular tumor characteristics, the more complex and costly the data systems can
become, with potentially long lag times required to implement changes. Even with extensive informatics adaptations, some research questions can only be answered by
collecting questionnaire data or other types of information not recorded in the usual databases [66].
Another challenge concerns selection of the appropriate time to initiate HSR in the developmental trajectory
of a new CMG technology. Unfortunately, the opportunity to generate published HSR in the technical development and testing phase is often missed because translational studies are typically restricted in scope and do not
incorporate HSR disciplines [67, 68]. Consequently, data
that directly address health care delivery are often absent
from translational medicine conferences and publications. Several conceptual models have described the
translation of emerging CMG technologies into health
practice, emphasizing the continuum of translational research that could incorporate HSR at various stages [68,
69]. Certain types of HSR, such as decision analysis and
assessment of quality of life and acceptability, could be
readily integrated into premarket clinical studies. In fact,
decision-analytic models can be used to extend the
knowledge from empirical studies of CMG technologies
to a broader array of clinical situations, and can allow for
the extrapolation of costs and health effects beyond the
time horizon of a single study. Models can formally relate
biological and clinical information, provide quantitative
Public Health Genomics 2009;12:233–244
239
insight into the relative importance of different uncertain
variables, and investigate how results will change if values of key parameters are changed. By identifying the
most influential parameters, they can be used to help prioritize and guide data collection efforts as well as assist
with the design of randomized trials [70].
Data on costs can certainly be collected during clinical
trials, although the highly controlled experimental conditions may not mirror the real-world practice conditions
needed for economic analyses [71]. Model-based analyses
inevitably rely on uncertain assumptions and parameters; as such, it is imperative that analysts be as transparent as possible and assess the impact of parameter and
model uncertainty. The purpose and stage of development of CMG technologies will greatly influence the economic evaluation, due to differences in target populations, amount of available data on clinical outcomes and
other variables of interest [72]. Standardization of cost
measures and analytic methods can ensure better comparability across economic studies [73].
Incorporation of HSR into premarket clinical studies,
and early in the diffusion curve when clinical guidelines
and public health policies are nascent, can best be achieved
through a scientific, team-based approach that brings together collaborating experts in translational and applied
research. Early collaboration could facilitate timely modifications of databases used for postmarketing surveillance to rapidly respond to information needs. Challenges to implementing a transdisciplinary team approach
include cross-disciplinary differences in terminology
and research culture, complexities in working across institutions, and competing needs of tenure-track investigators to obtain grants and publications in their own specific fields [74, 75]. Also, due to increasing data complexity, the teams may require expansion to include disciplines
not previously considered, such as bioinformatics, systems biology and the social sciences.
Understanding the use and impact of CMG technologies among medically underserved populations, such
as racial or ethnic minorities and the uninsured, presents further challenges. These populations are often
underrepresented in research due to limited access to
health care, prioritization of social needs such as food
or housing, distrust in the health care system, or other
factors [76–78]. Tailored and community-based approaches to recruiting may improve representation in
various types of research [79]. Inclusion of underserved
populations in basic and translational cancer research
can help ensure that HSR addresses relevant questions
about health disparities. This is particularly important
240
Public Health Genomics 2009;12:233–244
when the underserved populations are at highest risk of
incidence and/or mortality, as is well-illustrated in the
case of African American men and prostate cancer
[80].
CMG technologies are evolving in a changing regulatory and policy environment that may or may not facilitate HSR [81]. Regulatory changes favoring pharmacogenomic information on drug labels may lead to increased
opportunities for collaborative HSR. In contrast, exclusion of interventions such as genetic counseling and testing from insurance coverage limits availability of data in
claims databases. Ultimately, the growth of HSR on CMG
technologies hinges on its perceived importance, the
availability of interested research groups and funding resources.
One of the greatest challenges to expanding HSR is
constrained government funding [82]. Creative mechanisms for funding HSR research, such as public-private
partnerships or interagency federal initiatives, must be
explored during this period of tightened budgets. Furthermore, funding agencies should promote initiatives
that specifically prioritize health care delivery research
on CMG technologies, in both the translational and postmarketing phase. The NIH program announcement,
‘Understanding the Effects of Emerging Cellular, Molecular, and Genomic Technologies on Cancer Health Care
Delivery’, is one such example that aims to stimulate
grant applications (http://grants.nih.gov/grants/guide/
notice-files/NOT-CA-06-039.html;
http://grants.nih.
gov/grants/guide/pa-files/PA-06-281.html).
In addition, AHRQ has started a new DEcIDE (Developing Evidence to Inform Decisions about Effectiveness)
research network as part of its Effective Health Care program to conduct accelerated studies about the outcomes,
comparative clinical effectiveness, safety and appropriateness of health care services (http://effectivehealthcare.
ahrq.gov/aboutUs.cfm?abouttype=decidecert). AHRQ
has also recently solicited proposals to fund demonstration projects that advance understanding of how best to
incorporate clinical decision support into the delivery
of health care (http://www.fbo.gov/spg/HHS/AHRQ/
DCM/AHRQ%2D07%2D10045/Attachments.html).
Such projects could specifically address funding needs
for emerging CMG interventions. Since the workshop described in this article, the Centers for Disease Control
and Prevention National Office of Public Health Genomics released several other relevant funding initiatives, under the Genomic Applications in Practice and Prevention
program, including one entitled ‘Translation Research’
(http://www.cdc.gov/od/pgo/funding/GD08-001.htm)
Wideroff et al.
and a second one entitled ‘Translation Programs in Education, Surveillance, and Policy’ (http://www.cdc.gov/od/
pgo/funding/GD08-801.htm).
Recommendations for Advancing HSR on CMG
Technologies
Box 4 summarizes the recommendations of workshop
participants for strengthening HSR on emerging CMG
technologies, with recognition that specific strategies depend on the type of technology and its stage of development. HSR should systematically examine the benefits
and harms of CMG technologies in routine clinical care.
A wide range of outcomes must be studied, including
preferences, utilization, access and disparities, quality of
care, cost as well as trends in incidence, mortality, survival and health-related quality of life among target populations. Demographically and biologically heterogeneous populations must be studied, using tailored community-based approaches to facilitate recruitment of the
medically underserved.
Box 4: A Proposed Health Services Research
Agenda for CMG Technologies
• Certain HSR areas, such as assessment of patient and
provider preferences, patient-reported outcomes,
economic modeling and decision analysis, should be
routinely incorporated in the premarketing phases of
product development and efficacy testing. The methodologies should be transparent, and the findings
should be made publicly available.
• Outcomes should include: patient-reported quality
of life indicators; prevalence and determinants of
patient and provider use, knowledge, and attitudes;
health plan and government policy toward coverage
and regulation; incremental cost, cost-effectiveness
and cost-utility; net benefit based on population
trends in incidence, mortality, survival and quality
of life.
• Disparities in access to care and health outcomes
should be routinely monitored, with special attention
given to inclusion of racial/ethnic minorities and
other underserved populations in studies.
• A full range of outcomes should be considered when
developing and evaluating quality measures of progress related to CMG technologies.
Research Agenda for CMG Technologies
in Cancer Care
• Administrative databases of patient records should
be systematically evaluated and modified to include
infrastructure, data elements and electronic linkages
that facilitate HSR. Standard HSR data sources may
need modifications to be effectively used.
• Funding agencies and investigators should explore
ways to further transdisciplinary team science that
bridges discovery, development and delivery research.
• Funding agencies and researchers should explore
ways to fund HSR, with creative approaches emphasized during times of budget restrictions. Mechanisms for priority funding should be considered to
encourage HSR in both pre- and postmarketing studies.
A transdisciplinary team science approach can help
ensure that these critical research needs are met. Certain
areas, such as evaluation of patient and provider preferences as well as decision-analytic modeling, can be successfully incorporated into both pre- and postmarketing
clinical research. Other areas, such as utilization, access
to care and population trends in cancer outcomes are inherently suited for the postmarketing phase. Adaptations
may be needed to effectively use standard HSR data
sources, such as claims databases, population-based cancer registries and health systems’ electronic medical
records.
Workshop Limitations
Given time limitations, some areas were not addressed
but should be a focus of further research. Although evidence synthesis and its application in practice are valid
forms of HSR, the workshop and its recommendations focused mostly on the primary research enterprise that allows data to be collected for this purpose. Other areas that
were not covered include dissemination and communications research, the role of practical clinical trials in building the evidence base for technology assessment [83], and
reimbursement policy where evidence is limited. International comparisons were also not addressed, but could
shed light on variability in adoption and effectiveness of
CMG technologies in countries with different research
priorities, clinical practices and health care systems. Specific bioinformatics tools, such as natural language processing, were not discussed in detail, although such tools
may be critical to improving the availability and validity
of electronic medical records for HSR on emerging CMG
Public Health Genomics 2009;12:233–244
241
technologies. Institutional review board and privacy policies were not addressed in depth, although their impact
on the research climate is significant.
Conclusions
This workshop identified a comprehensive research
agenda to better understand the delivery of care for
emerging CMG interventions. Meaningful expansion of
HSR on CMG technologies hinges on its perceived importance, the availability of interested research groups
and funding resources. Ultimately, the successful adoption of appropriate technologies will depend on understanding and informing the patient, provider, health care
system as well as societal factors that contribute to effectiveness and uptake in routine clinical care.
Acknowledgements
We would like to thank Darrell Anderson and Marcia Feinleib
of the Scientific Consulting Group Inc., Gaithersburg, Md., USA
for editorial assistance.
The findings and conclusions in this report are those of the
authors and do not necessarily represent the views of the NCI,
Centers for Disease Control and Prevention as well as the
AHRQ.
The study was funded by the NCI (263-MQ-610769 to
K.A.P.; 263-MQ-612315 to K.A.; 263-MQ-612317 to C.L.B.; 263MQ-700870 to M.S.D.; 263-MQ-701381 to M.F.; 263-MQ612321 to S.J.G.; 263-MQ-611969 to R.A.H.; 263-MQ-612316 to
H.L.M.; 263-MQ-612323 to I.P.; 263-MQ-611964 to D.S.; 263MQ-612324 to K.A.S.; 263-MQ-611967 to the University of York;
HHSN261200700005C to Scientific Consulting Group for organizational and editorial assistance).
References
1 Moses H, Dorsey ER, Matheson DH, Thier
SO: Financial anatomy of biomedical research. JAMA 2005;294:1333–1342.
2 Kelloff GJ, Lippman SM, Dannenberg AJ,
Sigman CC, Pearce HL, Reid BJ, et al: Progress in chemoprevention drug development:
the promise of molecular biomarkers for prevention of intraepithelial neoplasia and cancer – a plan to move forward. Clin Cancer
Res 2006;12:3661–3697.
3 Deverka PA, Doksum T, Carlson RJ: Integrating molecular medicine into the US
health-care system: opportunities, barriers,
and policy challenges. Clin Pharmacol Ther
2007;82: 427–434.
4 American Society of Clinical Oncology:
American Society of Clinical Oncology policy statement update: genetic testing for cancer susceptibility. J Clin Oncol 2003; 21:
2397–2406.
5 Mueller NE, Birmann BM, Parsonnet J,
Schiffman MH, Stuver SO: Infectious agents;
in Schottenfeld D, Fraumeni JF (eds): Cancer
Epidemiology and Prevention. New York,
Oxford University Press, 2006, pp 507–548.
6 Gish RG, Locarnini SA: Chronic hepatitis B:
current testing strategies. Clin Gastroenterol Hepatol 2006;4:666–676.
7 Wheeler CM: Advances in primary and secondary interventions for cervical cancer: human papillomavirus prophylactic vaccines
and testing. Nat Clin Pract Oncol 2007; 4:
224–235.
8 Collins Y, Einstein MH, Gostout BS, Herzog
TJ, Massad LS, Rader JS, et al: Cervical cancer prevention in the era of prophylactic vaccines: a preview for gynecologic oncologists.
Gynecol Oncol 2006;102:552–562.
242
9 Kundi M: New hepatitis B vaccine formulated with an improved adjuvant system. Expert Rev Vaccines 2007;6:133–140.
10 Wagner PD, Verma M, Srivastava S: Challenges for biomarkers in cancer detection.
Ann NY Acad Sci 2004;1022:9–16.
11 O’Connell CD, Atha DH, Jakupciak JP: Standards for validation of cancer biomarkers.
Cancer Biomark 2005; 1:233–239.
12 Slamon DJ, Leyland-Jones B, Shak S, Fuchs
H, Paton V, Bajamonde A, et al: Use of chemotherapy plus a monoclonal antibody
against HER2 for metastatic breast cancer
that overexpresses HER2. N Engl J Med
2001;344:783–792.
13 Sherbenou DW, Druker BJ: Applying the discovery of the Philadelphia chromosome. J
Clin Invest 2007;117:2067–2074.
14 Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, et al: A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 2004; 351:
2817–2826.
15 Buyse M, Loi S, van’t Veer L, Viale G, Delorenzi M, Glas AM, et al: Validation and clinical utility of a 70-gene prognostic signature
for women with node-negative breast cancer
J Natl Cancer Inst 2006;98:1183–1192.
16 Miller CR, McLeod HL: Pharmacogenomics
of cancer chemotherapy-induced toxicity. J
Support Oncol 2007;5:9–14.
17 Swen JJ, Huizinga TW, Gelderblom H, de
Vries EG, Assendelft WJ, Kirchheiner J, et al:
Translating pharmacogenomics: challenges
on the road to the clinic. PLoS Med 2007; 4:
e209.
Public Health Genomics 2009;12:233–244
18 Brannon-Peppas L, Blanchette JO: Nanoparticle and targeted systems for cancer therapy.
Adv Drug Deliv Rev 2004;56:1649–1659.
19 Vicent MJ, Duncan R: Polymer conjugates:
nanosized medicines for treating cancer.
Trends Biotechnol 2006; 24:39–47.
20 Gradishar WJ, Tjulandin S, Davidson N,
Shaw H, Desai N, Bhar P, et al: Phase III trial
of nanoparticle albumin-bound paclitaxel
compared with polyethylated castor oilbased paclitaxel in women with breast cancer. J Clin Oncol 2005;23:7794–7803.
21 Smith TJ, Khatcheressian J, Lyman GH, Ozer
H, Armitage JO, Balducci L, et al: 2006 update of recommendations for the use of white
blood cell growth factors: an evidence-based
clinical practice guideline. J Clin Oncol
2006;24:3187–3205.
22 Rodgers GM: Guidelines for the use of erythropoietic growth factors in patients with
chemotherapy-induced anemia. Oncology
2006;20(suppl 6):12–15.
23 Ljungman P, Urbano-Ispizua A, CavazzanaCalvo M, Demirer T, Dini G, Einsele H, et al:
Allogeneic and autologous transplantation
for haematological diseases, solid tumours
and immune disorders: definitions and current practice in Europe. Bone Marrow Transplant 2006;37:439–449.
24 Torigian DA, Huang SS, Houseni M, Alavi A:
Functional imaging of cancer with emphasis
on molecular techniques. CA Cancer J Clin
2007;57:206–224.
25 Sokolov K, Aaron J, Hsu B, Nida D, Gillenwater A, Follen M, et al: Optical systems for
in vivo molecular imaging of cancer. Technol Cancer Res Treat 2003;2:491–504.
Wideroff et al.
26 Schrag D, Garewal HS, Burstein HJ, Samson
DJ, Von Hoff DD, Somerfield MR: American
Society of Clinical Oncology technology assessment: chemotherapy sensitivity and resistance assays. J Clin Oncol 2004; 22: 3631–
3638.
27 Jacobellis J, Martin L, Engel J, VanEenwyk J,
Bradley LA, Kassim S, et al: Genetic testing
for breast and ovarian cancer susceptibility:
evaluating direct-to-consumer marketing –
Atlanta, Denver, Raleigh-Durham, and Seattle, 2003. MMWR Morb Mortal Wkly Rep
2004;53:603–606.
28 Vadaparampil ST, Wideroff L, Olson L,
Viswanath V, Freedman AN: Physician exposure to and attitudes toward advertisements for genetic tests for inherited cancer
susceptibility. Am J Med Genet A 2005; 135:
41–46.
29 Bennett CL, Nebeker JR, Lyons EA, Samore
MH, Feldman MD, McKoy JM, et al: The research on adverse drug events and reports
(RADAR) project. JAMA 2005; 293: 2131–
2140.
30 Ginsburg GS, Konstance RP, Allsbrook JS,
Schulman KA: Implications of pharmacogenomics for drug development and clinical
practice. Arch Intern Med 2005; 165: 2331–
2336.
31 Winn-Deen ES: Standards and controls for
genetic testing. Cancer Biomark 2005;1:217–
220.
32 Burke W, Atkins D, Gwinn M, Guttmacher
A, Haddow J, Lau J, et al: Genetic test evaluation: information needs of clinicians, policy
makers, and the public. Am J Epidemiol
2002;156:311–318.
33 Schoenhagen P: The role of coronary CT angiography (CTA) for patients presenting
with acute chest pain: defining problem-specific, evidence-based indications of a novel
imaging modality. Int J Cardiovasc Imaging
2007;23:429–432.
34 Vaucher YE, Pretorius DH: Brain imaging in
neonatal clinical trials: in search of a gold
standard. J Pediatr 2007;150:575–577.
35 Phillips KA, Van Bebber SL, Issa AM: Initial
development of an evidence base for personalized medicine’s translation to clinical
practice and health policy. Personalized Med
2006;3:411–414.
36 Phillips KA, Veenstra DL, Ramsey SD, Van
Bebber SL, Sakowski J: Genetic testing and
pharmacogenomics: issues for determining
the impact to healthcare delivery and costs.
Am J Manag Care 2004;10:425–432.
37 Hershman D, Neugut AI, Jacobson JS, Wang
J, Tsai WY, McBride R, et al: Acute myeloid
leukemia or myelodysplastic syndrome following use of granulocyte colony-stimulating factors during breast cancer adjuvant
therapy. J Natl Cancer Inst 2007; 99: 196–
205.
Research Agenda for CMG Technologies
in Cancer Care
38 Bennett CL, Evens AM, Andritsos LA, Balasubramanian L, Mai M, Fisher MJ, et al: Haematological malignancies developing in previously healthy individuals who received
haematopoietic growth factors: report from
the research on adverse drug events and reports (RADAR) project. Br J Haematol 2006;
135:642–650.
39 Lipscomb J, Donaldson MS, Hiatt RA, et al:
Cancer outcomes research and the arenas of
application. J Natl Cancer Inst Monogr 2004;
33:1–7.
40 Webster A, Martin P, Lewis G, Smart A: Integrating pharmacogenetics into society: in
search of a model. Nat Rev Genet 2004; 5:
663–669.
41 Constable S, Johnson MR, Pirmohamed M:
Pharmacogenetics in clinical practice: considerations for testing. Expert Rev Mol Diagn 2006;6:193–205.
42 Dervieux T, Bala MV: Overview of the pharmacoeconomics of pharmacogenetics. Pharmacogenomics 2006;7:1175–1184.
43 Hiatt RA, Rimer BK: A new strategy for cancer control research. Cancer Epidemiol Biomarkers Prev 1999;8:957–964.
44 Guttmacher AE, Porteous ME, McInerney
JD: Educating health-care professionals
about genetics and genomics. Nat Rev Genet
2007;8:151–157.
45 McInerney J:Education in a genomics world.
J Med Philos 2002;27:369–390.
46 Penchansky R, Thomas JW: The concept of
access: definition and relationship to consumer satisfaction. Med Care 1981; 19: 127–
140.
47 McLaughlin CG, Wyszewianski L: Access to
care: remembering old lessons. Health Serv
Res 2002;37:1441–1443.
48 Meropol NJ, Schulman KA: Cost of cancer
care: issues and implications. J Clin Oncol
2007;25:180–186.
49 Kim P: Cost of cancer care: the patient perspective. J Clin Oncol 2007;25:228–232.
50 Kabe KL, Kolesar JM: Role of trastuzumab in
adjuvant therapy for locally invasive breast
cancer. Am J Health Syst Pharm 2006; 63:
527–533.
51 Eng C, Iglehart D: Decision aids from genetics to treatment of breast cancer: long-term
clinical utility or temporary solution? JAMA
2004;292:496–498.
52 Salsberg E, Erikson C: The changing physician workforce landscape: implications for
physical medicine and rehabilitation. Am J
Phys Med Rehabil 2007;86:838–844.
53 Armstrong K, Micco E, Carney A, Stopfer J,
Putt M: Racial differences in the use of
BRCA1/2 testing among women with a family history of breast or ovarian cancer. JAMA
2005;293:1729–1736.
54 Krieger N: Defining and investigating social
disparities in cancer: critical issues. Cancer
Causes Control 2005; 16:5–14.
55 Kilbourne AM, Switzer G, Hyman K, Crowley-Matoka M, Fine MJ: Advancing health
disparities research within the health care
system: a conceptual framework. Am J Public Health 2006;96:2113–2121.
56 Hall MJ, Olopade OI: Disparities in genetic
testing: thinking outside the BRCA box. J
Clin Oncol 2006;24:2197–2203.
57 Phillips KA, Veenstra D, Van Bebber S, Sakowski J: An introduction to cost-effectiveness and cost-benefit analysis of pharmacogenomics. Pharmacogenomics 2003; 4:
231–239.
58 Elkin EB, Weinstein MC, Winer EP, Kuntz
KM, Schnitt SJ, Weeks JC: HER-2 testing and
trastuzumab therapy for metastatic breast
cancer: a cost-effectiveness analysis. J Clin
Oncol 2004;22:854–863.
59 Goldie SJ, Levin AR: Genomics in medicine
and public health: role of cost-effectiveness
analysis. JAMA 2001;286:1637–1638.
60 Goldie S: A public health approach to cervical cancer control: considerations of screening and vaccination strategies. Int J Gynaecol Obstet 2006;94(suppl 1):S95–S105.
61 Wingo PA, Howe HL, Thun MJ, Ballard-Barbash R, Ward E, Brown ML, et al: A national
framework for cancer surveillance in the
United States. Cancer Causes Control 2005;
16:151–170.
62 Hiatt RA: The future of cancer surveillance.
Cancer Causes Control 2006; 17:639–646.
63 Haux R: Health information systems – past,
present, future. Int J Med Inform 2006; 75:
268–281.
64 Lobach DF, Detmer DE: Research challenges
for electronic health records. Am J Prev Med
2007;32:S104–S111.
65 Deapen D: Cancer surveillance and information: balancing public health with privacy
and confidentiality concerns (United States).
Cancer Causes Control 2006; 17:633–637.
66 McDonald IG: Quality assurance and technology assessment: pieces of a larger puzzle.
J Qual Clin Pract 2000;20:87–94.
67 Sussman S, Valente TW, Rohrbach LA, Skara
S, Pentz MA: Translation in the health professions: converting science into action. Eval
Health Prof 2006;29:7–32.
68 Khoury MJ, Gwinn M, Yoon PW, Dowling
N, Moore CA, Bradley L: The continuum of
translation research in genomic medicine:
how can we accelerate the appropriate integration of human genome discoveries into
health care and disease prevention? Genet
Med 2007;9:665–674.
69 Col NF: The use of gene tests to detect hereditary predisposition to chronic disease: is
cost-effectiveness analysis relevant? Med
Decision Making 2003;23:441–448.
Public Health Genomics 2009;12:233–244
243
70 Goldie SJ, Goldhaber-Fiebert JD, Garnett
GP: Chapter 18: public health policy for cervical cancer prevention: the role of decision
science, economic evaluation, and mathematical modeling. Vaccine 2006;24(suppl 3):
S155–S163.
71 O’Sullivan AK, Thompson D, Drummond
MF: Collection of health-economic data
alongside clinical trials: is there a future for
piggyback evaluations? Value Health 2005;
8:67–79.
72 Rogowski W: Current impact of gene technology on health care: a map of economic assessments. Health Policy 2007;80:340–357.
73 Torti FM Jr, Reed SD, Schulman KA: Analytic considerations in economic evaluations
of multinational cardiovascular clinical trials. Value Health 2006;9:281–291.
244
74 Sellers TA, Caporaso N, Lapidus S, Petersen
GM, Trent J: Opportunities and barriers in
the age of team science: strategies for success.
Cancer Causes Control 2006; 17:229–237.
75 McGuire DB: Building and maintaining an
interdisciplinary research team. Alzheimer
Dis Assoc Disord 1999;13(suppl 1):S17–S21.
76 Halbert CH, Armstrong K, Gandy OH Jr,
Shaker L: Racial differences in trust in health
care providers. Arch Intern Med 2006; 166:
896–901.
77 Corbie-Smith G, Thomas SB, St George DM:
Distrust, race, and research. Arch Intern
Med 2002;162:2458–2463.
78 Shavers VL, Lynch CF, Burmeister LF: Racial
differences in factors that influence the willingness to participate in medical research
studies. Ann Epidemiol 2002; 12:248–256.
Public Health Genomics 2009;12:233–244
79 Royal C, Baffoe-Bonnie A, Kittles R, Powell
I, Bennett J, Hoke G, et al: Recruitment
experience in the first phase of the African American hereditary prostate cancer
(AAHPC) study. Ann Epidemiol 2000; 10(8
suppl):S68–S77.
80 Powell IJ: Epidemiology and pathophysiology of prostate cancer in African-American
men. J Urol 2007;177:444–449.
81 Hopkins MM, Ibarreta D, Gaisser S, Enzing
CM, Ryan J, Martin PA, et al: Putting pharmacogenetics into practice. Nat Biotechnol
2006;24:403–410.
82 Zerhouni EA: Research funding. NIH in the
post-doubling era: realities and strategies.
Science 2006;314:1088–1090.
83 Tunis SR, Stryer DB, Clancy CM: Practical
clinical trials: increasing the value of clinical
research for decision making in clinical and
health policy. JAMA 2003;290:1624–1632.
Wideroff et al.