TOXICOLOGICAL SCIENCES 122(2), 223–234 (2011)

doi:10.1093/toxsci/kfr113
Advance Access publication May 10, 2011

Toxicology and Epidemiology: Improving the Science with a Framework

for Combining Toxicological and Epidemiological Evidence to Establish
Causal Inference

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
Hans-Olov Adami,*,† Sir Colin L. Berry,‡ Charles B. Breckenridge,§ Lewis L. Smith,{,k James A. Swenberg,kj
,
Dimitrios Trichopoulos,* Noel S. Weiss,kk,# and Timothy P. Pastoor §,1
*Department of Epidemiology, Harvard School of Public Health, Boston, Massachusetts 02215; †Department of Medical Epidemiology and Biostatistics,
Karolinska Intitutet, Stockholm, Sweden; ‡Queen Mary University, London, UK; §Toxicology & Health Sciences Department, Syngenta Crop Protection,
Greensboro, North Carolina 27419; {MRC Toxicology Unit The College of Medicine, University of Leicester, Leicester, UK; kSyngenta Crop Protection Ltd,
Basle, Switzerland; kjDepartment of Environmental Sciences and Engineering, University of North Carolina, Chapel Hill, North Carolina 27599; kkDepartment
of Epidemiology, University of Washington, Seattle, Washington 98195; and #The Fred Hutchinson Cancer Research Center, Seattle, Washington 98109

1
To whom correspondence should be addressed at PO Box 18300, Greensboro, NC 27419. Fax: (336) 632-2226. E-mail: tim.pastoor@syngenta.com.

Received March 16, 2011; accepted May 4, 2011

by chimney soot (Potter, 1962). It was almost 140 years before

Historically, toxicology has played a significant role in verifying experimental confirmation of this was produced by Yamagiwa
conclusions drawn on the basis of epidemiological findings. Agents
and Ichikawa (1918). By repeated painting of rabbit ears with
that were suggested to have a role in human diseases have been tested
in animals to firmly establish a causative link. Bacterial pathogens are a coal tar extract, they produced epithelial skin tumors,
perhaps the oldest examples, and tobacco smoke and lung cancer and powerfully corroborating what Pott had seen in humans. In
asbestos and mesothelioma provide two more recent examples. With this case, an inference of causation in humans was arrived at
the advent of toxicity testing guidelines and protocols, toxicology took through a combination of the two scientific disciplines.
on a role that was intended to anticipate or predict potential adverse Subsequently, animal studies were used to verify other
effects in humans, and epidemiology, in many cases, served a role in epidemiological findings, serving to establish Koch’s third
verifying or negating these toxicological predictions. The coupled role postulate: the agent should cause the disease when introduced
of epidemiology and toxicology in discerning human health effects by
into a healthy organism (Koch, 1884, 1893). Although Koch’s
environmental agents is obvious, but there is currently no systematic
original intent was proving disease causation by microbiolog-
and transparent way to bring the data and analysis of the two dis-
ciplines together in a way that provides a unified view on an adverse ical pathogens, this third postulate has also been applied to
causal relationship between an agent and a disease. In working to corroborating chemical-related epidemiological findings in
advance the interaction between the fields of toxicology and epi- humans, by testing in animals.
demiology, we propose here a five-step ‘‘Epid-Tox’’ process that would Although there are a number of examples of how epidemiology
focus on: (1) collection of all relevant studies, (2) assessment of their and toxicology intersected over time, perhaps the most notable
quality, (3) evaluation of the weight of evidence, (4) assignment of case is tobacco smoke and lung cancer. By 1964, there was ample
a scalable conclusion, and (5) placement on a causal relationship grid. epidemiological evidence for a causal connection between lung
The causal relationship grid provides a clear view of how epidemiolog-
cancer and smoking tobacco products; at that point, the U.S.
ical and toxicological data intersect, permits straightforward conclu-
sions with regard to a causal relationship between agent and effect, and
Surgeon General (Bayne-Jones et al., 1964) accepted the
can show how additional data can influence conclusions of causality. relationship as causal. Yet at that time, toxicologists could not
Key Words: epidemiology; causation; framework. reproduce similar tumors in animal models. This lack of
concordance emphasized the difficulty of using Koch’s postulates
that were established for infectious disease in chemical-related
THE INTERSECTION OF EPIDEMIOLOGY pathogenesis. Toxicological corroboration of epidemiological
AND TOXICOLOGY evidence later became the element of ‘‘biological plausibility’’ in
Hill’s guidelines for establishing causality (Hill, 1965). Hill and
In 1775, Percivall Pott concluded, on the basis of clinical others (Bayne-Jones et al., 1964) effectively modified Koch’s
observations, that scrotal cancer in chimney sweeps was caused third postulate from an orientation of proof to one of plausibility.
Ó The Author 2011. Published by Oxford University Press on behalf of the Society of Toxicology.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.5), which permits
unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
224 ADAMI ET AL.

Whereas the concordance was high between cancer-causing fields can derive conclusions based on paradigms illustrated in
agents initially discovered in humans and positive results in Figure 1. Together, conclusions of causality can be more firmly
animal studies (Tomatis et al., 1989; Wilbourn et al., 1984), the based, further investigations can be clearly identified, and
same could not be said for the reverse relationship: carcinogenic improvements in human health protection can be achieved. In
effects in animals frequently lacked concordance with overall addition to highlighting the history of relevant developments in
patterns in human cancer incidence (Pastoor and Stevens, 2005). the fields, we suggest a way that the two disciplines can come
This lack of concordance between toxicology and epidemiology together to better understand the impact, potential or real, of
might arise because the high doses used in animal studies to agents on human health.
produce tumors are not typically seen in human populations.
Nonetheless, toxicology took on a predictive rather than
a confirmatory role by providing alerts for potential effects in CAUSAL INFERENCE

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
humans, whether carcinogenic, neurotoxic, hepatotoxic, or any
other adverse outcome. These alerts became the basis for Process for Causal Inference
regulating chemical exposure to humans. The underlying The disciplines of toxicology and epidemiology ask the
assumption was that restricting exposure well below levels at question: can a substance cause a particular effect in humans.
which adverse effects were seen in animals would prevent The data obtained in toxicological and epidemiological studies
harmful outcomes in humans. do not always lead to a straightforward interpretation, and often
Thus, the relationship between epidemiology and toxicology different observers will differ in their conclusions. Even for
has shifted over time. Both disciplines seek to contribute data associations that are widely regarded as causal today—such as
relating to the causes of human disease and occasionally lean ingestion of water contaminated with the bacterium Vibrio
on each other to support propositions of causality. Toxicolo- cholerae and the incidence of cholera or cigarette smoking and
gists and epidemiologists alike spend considerable time and the incidence of lung cancer—for some years after relevant
effort characterizing the relationship between the putative data became available, there was considerable disagreement as
causal agent and a response (Fig. 1). Many of the same to the presence of a cause-effect relation in each instance.
fundamental considerations are part of the evidence-based Indeed, a principle underlying the philosophy of science is that
analysis that takes place by scientists in the two disciplines. causality cannot be ‘‘proven’’; it can only be inferred with
However, the two fields could arguably be said to work in different degrees of certainty.
parallel rather than in concert. Can toxicological experimen- Epidemiological investigation of a null hypothesis that
tation augment a weak positive epidemiological finding? postulates that a variable has no effect on a health outcome can
Conversely, when and how does low biological plausibility never be established to be true (Popper, 1959); there can only
influence a positive epidemiological finding? Separately, the be a failure to show that the null hypothesis is false within the

FIG. 1. Contribution of toxicology and epidemiology data to causal inference. Many of the same principles contribute to evidence-based decisions in the two
fields. Together, causation can be more accurately inferred.
 TOXICOLOGY AND EPIDEMIOLOGY 225

limits of specific study designs. Theories that integrate example, geographical differences or changes in disease
observations from multiple studies, or rely on other biological incidence over time within a population.
considerations, are useful when they make testable predictions. Ecologic studies have the potential to contribute to our
Hypotheses that are not testable do not fall within the realm of understanding of exposure-disease relationships if . . .
science. Likewise, expert opinion should be supported by
the environmental exposure level can be ascertained with
evidence for rational science-based decision making (Guzelian
reliability,
et al., 2005).
there are large differences in exposure,
Because Hill (1965) and others (Bayne-Jones et al., 1964)
the incidence of the disease is ascertained in a comparable
articulated their perspectives on causal inference, scientists
manner, and
have further described methods to systematically review and
there is little or no difference in the presence of other causes of
characterize the evidence that might be used to support an

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
the disease.
inference of causality (Cole, 1997; ECETOC, 2009; Kundi,
2007; Phillips and Goodman, 2004; Rothman 1976; Rothman For example, aflatoxin (a toxic product of Aspergillus flavus)
and Greenland, 2005; Susser, 1986; Weed, 2005). We suggest was found early on in experimental evaluations in animals to be
an expert judgment process for integrating the totality of the an extremely potent carcinogen. At that time, the only relevant
epidemiological findings in a weight of evidence framework. data in humans took the form of correlations of liver cancer
This integration takes note of the literature cited above but mortality rates across population groups with marked differ-
extends this thinking by offering a method to systematically ences in estimated aflatoxin intake. The positive correlation
consider biological plausibility and epidemiological evidence observed in these studies was open to alternative interpreta-
in a process to unite epidemiology and toxicology in tions—the populations with the highest rates differed in ways
a framework to infer causality. other than exposure to aflatoxin such as the prevalence of
Hepatitis B infection—so it was largely the strength of the
Applications of Causal Inference in Epidemiology laboratory evidence that served as a basis for a tentative causal
Epidemiological studies document the occurrence of illness inference. Later, stronger epidemiologic data became available
or injury in human populations. Depending on the design, supporting a causal effect. In particular, there were ecologic
epidemiological studies can provide evidence bearing on studies with less potential for confounding and nested case-
a causal relationship. For example, quantification of the control studies in which prediagnosis urinary markers of
efficacy of pharmaceutical agents in humans is often based aflatoxin intake could be assessed (Qian et al., 1994; Wang
on randomized controlled studies, where ‘‘exposed’’ and et al., 1996).
‘‘nonexposed’’ persons are similar with regard to other In practice, causal inferences that can be drawn from the
characteristics that bear on the outcome in question. However, results of ecologic studies may be limited because:
the focus of this paper is on causal inference for environmental Within a given population, exposure characterization may
agents (primarily synthetic chemicals). Because randomized not have been carried out (or carried out well and in a similar
trials with environmental agents are rarely feasible or perhaps way) over time or among population subgroups. Furthermore,
ethical, this study design will not be discussed here. within a population, the actual variation in exposure levels may
Studies aimed at evaluating environmental chemicals and be small, making it difficult for an epidemiological study to
other environmental factors are generally nonrandomized reliably document the differences in occurrence of disease. The
observational studies with an ecologic, case-control, or cohort weaker the association, the more difficult it is to distinguish it
design. Although these studies are fundamental in gauging from an association that arises by chance or confounding.
possible human health effects, their design may limit the extent The completeness of ascertainment of the disease condition
to which inferences about causality can be drawn. Because can vary by place and time. This is particularly a problem for
observational studies do not randomly allocate subjects to a condition in which diagnostic criteria are difficult to apply
exposure, interpretation of the results of these studies must take consistently (e.g., autism, Parkinson’s disease, non-Hodgkin
into account any differences, or the possibility of differences, lymphoma) but can also be present when differential disease
between exposed and nonexposed subjects. A brief description screening occurs as a function of location and time (e.g.,
of these observational studies and their strengths and weak- prostate specific antigen testing for prostate cancer).
nesses follows. In order to maximize the contrast in exposure prevalence or
levels across geographic units, many ecologic studies compare
Ecologic Studies Ecologic studies contrast the incidence of disparate geographic populations for disease occurrence. For
disease across populations (or population subgroups) that differ example, most ecologic studies that have examined the
in terms of presence or degree of an exposure to an association between dietary fat and breast cancer incidence
environmental factor. The incidence of disease among different have compared national populations from around the world
population subgroups may be evaluated on the basis of, for where data on both diet and cancer incidence were available.
226 ADAMI ET AL.

This approach allowed for the inclusion of populations in Cohort Studies It often happens that the same chemicals to
which a variety of dietary intakes was present but made it which one or more communities are exposed also are
difficult to interpret whether the association seen (i.e., higher encountered in persons who work in the manufacture or
fat intake associated with higher breast cancer incidence) was distribution of these chemicals. Because these exposures tend
because of the dietary differences in fat intake or to differences to be higher than those received in a community at large, any
in one or more of the other characteristics of these populations. impact on disease risk from exposure to the agent is likely to be
The absence of a relationship between dietary fat and breast greater in magnitude in the exposed workforce and therefore
cancer incidence was suggested by the results of cohort studies easier to ascertain in an epidemiologic study.
where little or no association between dietary fat intake and Because it is often possible to identify workforce members
breast cancer risk was observed among individuals within and monitor their status through vital records and disease
certain populations (Hunter et al., 1996). These data argue that registers, epidemiologic studies based on the workers’

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
the strong positive association observed in ecologic studies was experience are feasible. The results of occupational cohort
in fact a reflection of the confounding influence of one or more studies can nevertheless be difficult to interpret because of the
characteristics that were associated with both diet and breast presence of multiple chemical exposures on the job and,
cancer risk (Colditz et al., 2006). particularly in retrospective cohort studies, difficulties in
accounting for prior work history and other disease-causing
exposures not ascertained in available records such as smoking
Case-Control Studies Case-control studies ascertain the history. Nonetheless, occupational cohort studies have contrib-
proportion of persons who previously experienced one or more uted a great deal to our understanding of health effects of
exposures among persons with a disease (cases) and a sample chemical exposures and, when available, can assume a prom-
of persons representative of the person-time from which the inent place in the evaluation of the safety of exposure to
cases were generated (controls). chemicals.
Exposure ascertainment is a potential source of bias in case-
control studies, notably in studies investigating environmental
exposures because they may be incompletely or inaccurately Applications of Causal Inference in Toxicology
reported or recorded and misclassification may vary by case- In toxicology, the test agent is given to the animal or in vitro
control status. Thus, the results can indicate either spuriously cellular system under clearly defined exposure conditions (e.g.,
high or spuriously low estimates of the magnitude of any oral, dermal, inhalation; gavage, diet; short term, long term,
association. Direct measurements of blood or tissue levels of etc.). The physiological status of each test group is compared
chemical exposures (or metabolites of these chemicals) with the untreated group. From this body of data, a toxicologist
obtained after diagnosis in the cases may not reflect earlier must then decide which responses are exposure related and
levels of exposure because the illness and its treatment may determine whether the responses observed are relevant to
have led to an alteration in these levels. Even if exposure levels humans. In the absence of evidence to the contrary, the
of cases were unaffected by disease status among cases, the toxicologist assumes that findings in animals are likely to be
levels measured at the time of the study may not be indicative relevant to human health.
of those present earlier in life when critical pathogenic events However, as our understanding of biological systems has
occurred. evolved, we realize that effects in animals may not be relevant
Unless a case-control study can overcome the difficulties of to humans. The need for this important distinction depends on
valid retrospective ascertainment of exposure status, it cannot either qualitative differences in biology or quantitative differ-
be confidently relied upon to provide a valid estimate of the ences between animals and humans in the kinetics of the
association between an exposure and a disease. Cross-sectional chemical or the dynamics of the response.
studies, in which current exposure levels are compared between A systematic way of drawing conclusions of human
persons with and without a given condition at the time of the relevance (causality) was first proposed by the International
assessment of the exposure (irrespective of when that condition Programme on Chemical Safety (Sonich-Mullin et al., 2001)
first developed), are particularly problematic in this regard. and later expanded substantially with the development of
In unusual circumstances—when the proportion of ill frameworks for evaluating the human relevance of mode of
individuals with a history of a given exposure far exceeds action (MoA) in experimental animals for carcinogens (Boobis
what might be expected—an association can be inferred et al., 2006; Cohen et al., 2003; Klaunig et al., 2003, Meek
without the need for a formal control group. For example, et al., 2003) and for noncancer effects (Boobis et al., 2008;
because all cases of a form of pneumonia in an area of Spain Seed et al., 2005). Julien et al. (2009) carried this concept one
during a relatively short period of time reported ingestion of step further and proposed the key events dose-response
adulterated rapeseed oil (Tabuenca, 1981), it was reasonable to framework (KEDRF). KEDRF is a step-wise decision-logic
infer a causal connection (and to take preventive action) prior process that provides a foundation for more rigorous and
to the enrollment of controls into this study. quantitative descriptions of dose-response.
 TOXICOLOGY AND EPIDEMIOLOGY 227

The essential form of MoA analysis asks three questions to extremely strong association seen in epidemiologic studies
establish the likelihood of a chemical’s potential effect on between aspirin use and Reye’s syndrome, combined with the
humans (Fig. 2): absence of any similar association with the use of other
analgesics (Halpin et al., 1982; Hurwitz and Schonberger,
1) Is there sufficient evidence in animal studies to establish
1987; Forsyth et al., 1989), served as a solid basis for
a MoA?
discouraging aspirin use in children, even without any precise
2) If so, is that mode of action operative in humans? and
knowledge at that time of how aspirin might have caused
3) If so—considering pharmacokinetic and dynamic
a child with flu or chicken pox to develop this illness.
characteristics—would the MoA be operative in humans?
In other instances, little or no evidence suggests causation,
such as in the published literature relating exposure to magnetic
If the answer is YES to all three questions, then the effect fields and the occurrence of cancer. In this case, it is likely that

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
seen in animals could plausibly occur in humans. Likewise, if most groups of experts would conclude that there is no
the MoA is considered to be not relevant to humans, then the evidence for an etiologic connection between exposure to mag-
biological plausibility of the effect being observed in humans netic fields and cancer in adults, that is, there is ‘‘no evidence
through the proposed MoA is highly unlikely. supporting causality.’’ Because it is not possible to rule out
a weak effect of exposure on disease incidence, it is not sur-
prising that some debate continues regarding the safety of
THE EPID-TOX FRAMEWORK exposure to magnetic fields.
These instances of varied information from toxicology and
In one of the initial essays that wrestled with the bases for epidemiology argue for a systematic approach that brings
inferences of the causes of disease, Hill (1965) concluded that comprehensive, disciplined thinking into a complete and
it is not possible to ‘‘lay down some hard-and-fast rules of rational evaluation of the evidence. Such a systematic treatment
evidence that must be obeyed before we accept cause and lays on the table the complete story and gives practitioners
effect.’’ In practice, tentative inference regarding the presence a way to point to specific gaps in knowledge or lapses in logic
or absence of a causal relation between exposure and disease is based on the totality of information.
made through a subjective process in which one considers Overall, the Epid-Tox Framework follows a series of steps
which of the indicated features are present and, in particular, that assesses an explicit effect such as a specific cancer,
the degree to which they are present. Occasionally, the process neurological disease, or any tissue or system-specific adverse
is straightforward—all the evidence supports a causal hypoth- effect. The following steps would be to:
esis—and nearly everyone who addresses the issue arrives at
the same conclusion, for example, that cigarette smoke is 1) collect all relevant studies (toxicology and epidemiology),
a cause of lung cancer. The evidence is considered ‘‘conclu- 2) assess the quality of each study and assign it to a quality
sive’’; causation is viewed as ‘‘definitely present.’’ category,
A similar conclusion occasionally can be reached when the 3) evaluate the epidemiological and toxicological weight of
epidemiologic data are overwhelming, even without supporting the evidence,
evidence from other medical disciplines. For example, the 4) assign a scalable conclusion to the biological plausibility
(toxicological) and epidemiological evidence, and
5) determine placement in a causal relationship grid.

Collect All Relevant Studies

This may be too obvious, but a serious source of bias is the
selective collection of studies. A comprehensive search for all
studies relevant to the end point in question should be
conducted and documented as part of the process. This step
is meant to be as inclusive as possible, bearing in mind that the
process begins with a specific question: does agent X cause
effect Y? All studies that offer data should be included at this
point. One problem that continues to plague both toxicology and
epidemiology is the nonpublication of ‘‘negative’’ studies wherein
no effects were seen and investigators and journals are reluctant to
publish such information. Nonetheless, no-effect studies are an
FIG. 2. Steps 1 and 2 of the Epid-Tox framework: study identification and important part of the total available data set and their absence
quality categorization. biases the overall judgment in favor of studies showing effects.
228 ADAMI ET AL.

Assess Quality and Categorize to be an acceptable study might be rejected by another. The
Both kinds of studies, in epidemiology and toxicology, may value of this step in the Epid-Tox Framework is to fully reveal
present the observer with a wide range of investigations carried and document not just the investigator’s quality categorization
out in variable ways, with differing entry or exclusion criteria, but the reason for drawing a particular conclusion.
variable ascertainment of effects, a range of exposures or Documentation of these evaluation and categorization
exposure estimations, and observational endpoints. No study decisions can be provided in narrative form for individual
should be excluded at this stage of consideration. Having studies. Study attributes to be considered include—but may not
collected all available studies, each study should be included or necessarily be limited to—the number of subjects, the range of
excluded by using a transparent rationale. Both disciplines exposure levels among these persons, study enrollment
have generally accepted criteria for assessing study quality. methodology, disease and exposure ascertainment methods,
range of exposure, potential information bias, identification and

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
Toxicology The U.S. Environmental Protection Agency measurement of potential confounders, and statistical method-
(USEPA) developed quality criteria that are typically applied in ology used to assess associations and to control for
the evaluation of studies submitted for regulatory purposes. confounders.
Various terms have been used to describe a study’s suitability, As more reports of epidemiology studies include a complete
relevance, conduct, and how well the study satisfies the intent description of the design and analytic methods used, the
of a particular guideline (USEPA, 1993), including ‘‘core information needed to perform a quality assessment will be
guideline,’’ ‘‘core minimum,’’ ‘‘core supplementary,’’ or ‘‘in- more readily available. A report checklist was developed by
valid.’’ Core guideline indicates an acceptable study, whereas von Elm et al. (2007) for observational studies, known as the
core minimum indicates that ‘‘while some things are missing, ‘‘Strobe statement,’’ and serves as a method to evaluate the
the study still fulfills the guideline requirements.’’ Core quality of reporting of a study. In a similar initiative, the London
supplementary has been used to identify studies with ‘‘. . . Principles for Epidemiology itemized the attributes that char-
a significant deficiency or that additional information is acterize well-conducted observational epidemiological studies
required.’’ Terms have changed over the years to ‘‘Acceptable’’ (Graham, 1995; London Principles, 1996).
and ‘‘Unacceptable,’’ with additional statements as to whether However, it might be measured, the quality of epidemio-
a study is upgradable to Acceptable status (USEPA, 2001). For logical studies will likely be distributed from useful to useless.
the purpose of the Epid-Tox framework, the extremes of For practical purposes, studies can be put into discrete
Acceptable and Unacceptable are useful categories. There are categories of quality similar to those used for toxicology
clearly going to be well done studies with verifiable studies. Studies that are well designed, relatively free of bias,
conclusions and on the other hand studies with inapplicable and have adequate control of known confounders are classified
methods, inappropriate data, or unsubstantiated conclusions. as Acceptable. Supplemental studies would have more serious
As well, an intermediate category is needed to categorize imperfections and be of lesser quality but still be useable.
studies that have deficiencies that render them less than fully Unacceptable studies would fail to meet several or all of the
acceptable, but have sufficient quality that they cannot be quality criteria and would not be used in subsequent steps in
regarded as unacceptable. Thus, the suggested categories the evaluation (Fig. 3).
include Acceptable, Supplemental, and Unacceptable (Fig. 3).
Evaluate the Weight of Evidence
Epidemiology Similar to all scientific investigations, no
Toxicology With all Acceptable and Supplemental studies
epidemiological study is perfect; all have limitations to some
at hand, the question is asked, ‘‘Is the effect of interest
extent. Nonetheless, experienced epidemiologists can evaluate
present.’’ If there is evidence for a specific effect in some or all
the strengths and weaknesses of individual studies and
animal studies, the next stage in the evaluation is to use MoA
categorize them as to whether they can be used to inform
analysis to determine human relevance (Fig. 2). If the answer to
a judgment regarding causality. ECETOC (2009) has an
all three MoA Framework questions is ‘‘yes,’’ then the effect is
excellent rendition of quality criteria that are based on elements
considered plausible to occur in humans. If the specific effect
of study design, exposure information, and health effects data.
of interest is absent from the animal studies or the effect is
However, no objective, numerical yardstick exists to grade the
present but judged by MoA analysis to be not relevant to
quality of epidemiology studies.
human health, then the effect is concluded to have low
Certainly, there is a subjective element in the categorization
biological plausibility to occur in humans.
process. However, it is better to take a study’s quality into
account, acknowledging the imperfection of the process, than Epidemiology Based on the evaluation of the complete set
to give each study an equal weight. How this is done might of studies categorized as Acceptable or Supplemental,
vary from investigator to investigator, but in any case, the a judgment is made as to whether or not there is an association
process of quality categorization needs to be transparently between an agent and a given disease in humans as well as the
documented in the evaluation. What one investigator may find strength of that association. This conclusion must be made
 TOXICOLOGY AND EPIDEMIOLOGY 229

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
FIG. 3. The human relevance mode of action framework.

from the totality of evidence and may require balancing a positive association will be identified when the reverse, a lack
conflicting studies to produce one encompassing statement of association, may also become increasingly plausible as
about the epidemiological evidence. scientific evidence accumulates.
Various approaches can be used to produce one encompass- Therefore, the scaling of strength for an epidemiological or
ing statement, including the systematic use of the Hill criteria toxicological evaluation begins at the center of the scale and,
(Hill, 1965). But the essence of evaluating the weight of depending on the presence or absence of the effect, the scaling
evidence relies upon several central concepts. These include, moves accordingly in the positive or negative direction. By
but are not limited to, an effect within and among the studies starting at the center of each scale (the middle of the grid),
that is found with strength, consistency, specificity, and evidence of absence can be distinguished from absence of
coherence (Cole, 1997; Lagiou et al., 2005). Whereas there evidence for an association. For example, with few sufficient
are currently no hard-and-fast systematic, numerical character- quality epidemiology studies, one may have to state that there
izations that capture this expert judgment process, most is an absence of evidence to conclude one way or another that
practitioners would acknowledge that faced with an array of there is a causal association. On the other hand, evidence for an
epidemiological studies, these concepts would guide their absence of an epidemiological relationship can take either of
judgment in deriving a reliable encompassing statement of two forms:
causal inference.
1) There may be a sufficient number of epidemiological
studies to conclude that an association does not exist. For
Assign a Scalable Conclusion example, relative risks are around 1.0 and no statistical
The ultimate value of the Epid-Tox Framework is to differences are seen within the studies. There is, therefore,
determine the degree of strength or likelihood of the effect of evidence for an absence of an effect. As a consequence, the
interest. Therefore, for both the epidemiological and toxico- scaling shifts toward the left, indicating that there is
logical findings, there needs to be a semiquantitative conclu- epidemiological evidence ‘‘against’’ a causal link. With the
sion that states the degree to which the studies indicate accumulation of more and more studies not showing a given
a positive, a negative, or no relationship. effect, the confidence for evidence against an association is
At the beginning of any epidemiological or toxicological strengthened.
evaluation, there has to be a starting point from which evidence 2) Along with data sets showing no association, there may
pushes a conclusion toward the existence or lack thereof of also be epidemiological data sets that actually indicate
causality. Starting at one end of a scale is not appropriate. Such a protective effect. In this case, relative risks would be less
a starting point implies that as studies are accumulated, than 1.0 and statistically significant.
230 ADAMI ET AL.

Determine Placement in a Causal Relationship Grid At this point, the evaluator can clearly see where the
Figure 4 shows the Epid-Tox graphical template for epidemiological and toxicological evidence intersects and,
establishing a causal relationship. Starting at the intersection based on that location on the graphic makes an overall
of the x- and y-axes (middle of graph), the degree of biological conclusion that starts with, ‘‘A causal relationship is . . .’’ and
plausibility (toxicology) is scaled on the y-axis and the degree completes the conclusion with words that describe the resultant
(weight) of epidemiological evidence on the x-axis. The area. Short, descriptive phrases are used here, but the
intersection of the toxicological and epidemiological scaling underlying data and weight of evidence should be well
leads to an appropriate, evidence-based conclusion regarding understood at this point. The categories are as given below.
causality.
The structure and appearance of the causal relationship Likely A causal relationship is ‘‘Likely’’ between the
environmental factor and the disease condition. This implies

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
graphic is fundamental to ensuing decisions about causality.
Several factors led to the development of its form, including the that consistent, reliable evidence from epidemiological and
impact of the degree of ‘‘positive’’ or ‘‘negative’’ data and the animal studies permits a causal inference to be made. Two
relative weighting of epidemiological studies versus toxico- examples of this outcome are asbestos as a cause of
logical studies. At the beginning of any analysis, there may be mesothelioma and tobacco smoke as a cause of lung cancer.
a dearth of either toxicological or epidemiological studies. In
such a case, where the scaling remains at or near the center Uncertain A causal relationship is ‘‘Uncertain’’ between
point, there is ‘‘insufficient information’’ to draw any the environmental factor and the disease condition. In this case,
conclusions. Note that the area of insufficient information is there may be epidemiological evidence that can reasonably be
oblong and extended for the biological plausibility axis. The interpreted as indicating a causal link. However, there may be
reason is that animal studies require a greater degree of little or no biological plausibility based on animal studies. Note
evidence relative to epidemiology. Animal studies are in the lower right-hand corner that the transition between
surrogates for actual human data and as such require higher Likely and Uncertain favors epidemiological evidence. That is,
levels of evidence. with a high degree of epidemiological evidence, significant,
In addition, as more studies and information become and compelling data for a lack of biological plausibility must
available, the scaling of either the toxicological plausibility exist to transition from Likely to Uncertain. This again stresses
or epidemiological evidence can change in a way that can be the primacy of epidemiological evidence.
easily illustrated with the two-dimensional graphic. For example, Kaposi’s sarcoma, a normally rare tumor in
humans, showed such a remarkable increased incidence
following HIV infection (Sarid et al., 2002) that epidemiolog-
ical criteria for a causal relationship were met (Fig. 5).
However, at the time, no laboratory studies had verified the
pathogen. Therefore, the association was categorized as likely
but of low biological plausibility. Later, extensive laboratory
investigations led to the discovery of a specific herpes virus
(HHV8 or KSHV) that strengthened the inference of causality
because of an increased knowledge regarding a likely patho-
genesis of Kaposi’s sarcoma.
Based on some early suggestive results, a number of
epidemiologic studies have been done on the possible relation
between exposure to electromagnetic fields (EMF) and the
occurrence of brain cancer. The results from occupational
studies—which typically involve higher levels of exposure than
residential studies—have been summarized (Kheifets et al., 2008).
The relative risk associated with EMF exposure was statistically
significant (RR ¼ 1.14, 95% CI ¼ 1.07–1.22). The authors of the
review, however, concluded that ‘‘the lack of a clear pattern of
EMF exposure and outcome risk does not support a hypothesis that
these exposures were responsible for the excess risk.’’ Biological
plausibility is low in this example because ‘‘in vitro, in vivo, or
mechanistic evidence has not provided clues’’ as to a basis for an
association between exposure to EMF and the development of
FIG. 4. The causal inference grid: how strong is the evidence for or against brain cancer (Kheifets et al., 2009). As shown in Figure 5, the
a causal relationship in humans? initial analysis of epidemiological studies showed some evidence;
 TOXICOLOGY AND EPIDEMIOLOGY 231

however, in combination with low plausibility, the causal epidemiological evidence, would be considered Uncertain
relationship would be considered Uncertain. With time, more but plausible (Fig. 6). Subsequent investigations showed that
recent studies—generally with relatively better exposure ascer- D-limonene–induced kidney toxicity is not relevant to
tainment—tended to observe an even smaller association than did humans (Swenberg and Lehman-McKeeman, 1999; Meek
earlier studies. The updated evaluation would move the et al., 2003), which moves the conclusion of a causal
categorization from Uncertain to Unlikely. relationship to the Unlikely category.
Another example, not shown on the grid, is phenobarbital.
Uncertain A causal relationship is ‘‘Uncertain’’ but plausible
Phenobarbital increased the incidence of liver tumors in long-
between the environmental factor and the disease condition. In this
term rodent bioassays (Whysner et al., 1996) by a mode of
instance, the weight of evidence analysis of epidemiological studies
action that would be plausible in humans. Biological plausibility
shows little or no evidence of any effect although toxicological
would be considered high for phenobarbital. However, epide-

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
studies may indicate the plausibility of an effect in humans.
miological studies have found no evidence of liver tumors in
For example, as shown in Figure 6, melamine bladder and
patients on lifetime anti-epilepsy treatment with phenobarbital
kidney toxicity seen in animal studies was considered relevant to
(IARC, 2001; Whysner et al., 1996). Without epidemiological
human health, albeit only at very high exposures. But no
evidence, the categorization would be Likely or Uncertain, but
epidemiological evidence supported a causal relationship. An
with sufficient epidemiological evidence for an absence of an
initial evaluation placed melamine in this category (Uncertain
effect in humans, the categorization would be Unlikely.
but plausible). However, the unfortunate incidents in China after
As mentioned previously, a checkbox approach to charac-
the adulteration of milk with melamine and resultant rise in the
terize the nature of the evidence that would lead an expert team
number of children with melamine crystals detected in the
of epidemiologists and toxicologists to reach a weight of
urinary bladder and death because of kidney damage confirmed
evidence decision is not practical. However, the Epid-Tox
that the mode of action understood from animal models is
Framework described here and shown schematically in Figure
relevant to humans at high levels of exposure (World Health
7 provides by structure and example a way of systematically
Organization, 2009). This additional epidemiological evidence
working through all evidence and reaching a conclusion that
moved the conclusion of causality from Uncertain to Likely.
can be tracked, debated, and modified with further data.
Unlikely A causal relationship is ‘‘Unlikely’’ between the
environmental factor and the disease condition. Both epidemi-
FUTURE NEEDS AND DIRECTIONS
ological and toxicological evidence is compatible with the
absence of effect. The proposed new framework represents a concerted effort
For example, because D-limonene causes kidney toxicity to bridge the fields of epidemiology and toxicology in a way
in male rats, the biological plausibility was high and, without that can impact, and hopefully improve, human risk

FIG. 5. Applications of the Epid-Tox framework: HIV/Kaposi’s sarcoma FIG. 6. Applications of the Epid-Tox framework: melamine and
and EMF and brain tumors. d-limonene.
232 ADAMI ET AL.

assessment. It will benefit from application and critique and for quality are less well defined and are often the product of where
will undoubtedly require some modification. This formalized the work was done and where it was published.
set of steps, in and of itself, provides a structure for challenging Another area that will need debate and refinement is the
both disciplines and how they can and should be brought degree of detail one needs to complete the two-dimensional
together. Each step of the process invites improvement, grid proposed in the Epid-Tox Framework. It may be
including the availability of studies, the determination of sufficient to declare general degrees of confidence in the
quality, the proper metric for assigning the degree or strength scaling of the two axes. However, with more precise scaling,
of evidence, and the appearance and utility of the two- one could imagine dividing the grid into four quadrants,
dimensional grid. with four quadrants within each quadrant, thereby subdivid-
Availability of evidence will always be an Achilles Heel in any ing and giving greater granularity to the overall conclu-
evaluation that seeks scope and completeness. Unpublished data sions. This may add greater detail to the analysis but may

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
that languish in the drawer or is only contained in official also spark fruitless debates about precisely where the
submissions to regulatory agencies is an unfortunate omission that biological plausibility or epidemiological point should be
currently is unavoidable. Furthermore, for both disciplines, the on each axis.
lack of ‘‘negative’’ studies (showing no effects) are usually judged Nonetheless, a framework can provide the logic and
to be of lesser value either by the investigator seeking a new disciplined thinking that promotes open discourse and leads
finding or a journal editor requiring impactful research. Such to evidenced-based decisions. Furthermore, decisions about
evidence rarely appears in the literature. what epidemiological or toxicological study should be done
One area that continues to plague both toxicology and can be facilitated by using the framework. For example,
epidemiology is measurement of quality. Whereas poor and clear indications from animal studies for a particular effect
excellent studies can often be identified and categorized, there is should inform the data collected in an epidemiology study.
no consistent and agreed method for those that fall in between Likewise, epidemiological findings should spur the design of
poor and excellent. For example, the Strobe statement (von Elm in silico, in vitro, or in vivo studies that could corroborate
et al., 2007) details criteria for judging the quality and reliability observations in human populations. Important decisions
of epidemiology studies, but the method is not routinely used or about human safety should rely on the cohesive appreciation
cited with studies or reviews. In experimental biology, the criteria of both epidemiology and toxicology and the synergistic

FIG. 7. Schematic representation of the framework for causal inference based upon weight of evidence of animal and epidemiological data.
 TOXICOLOGY AND EPIDEMIOLOGY 233

value that their combination brings to a comprehensive Graham, S., Howe, G. R., et al. (1996). Cohort studies of fat intake and
evaluation. the risk of breast cancer—a pooled analysis. N. Engl. J. Med. 334, 356–361.
The refinement of any method occurs by working Hurwitz, E. S., and Schonberger, L. B. (1987). Public Health Services study of
Reye’s syndrome and medication. Report of the main study. J. Am. Med.
examples through it In order to take that first step toward
Assoc. 257, 1905–1911.
refinement, Simpkins et al. (2011) provides a case study that
IARC, (2001). Phenobarbital and Its Sodium Salts. IARC Monographs on the
utilizes the framework to collect, evaluate, and integrate Evaluation of Carcinogenic Risks to Humans. Vol. 79, 161–288. IARC
epidemiological and toxicological evidence for causal Press, Lyon, France.
inference. Hopefully, more environmental agents will be Julien, E., Boobis, A., and Olin, S. (2009). The key events dose-response
worked through the Epid-Tox Framework to test and framework: a cross-disciplinary mode-of-action based approach to
improve its utility. examining dose-response and thresholds. Crit. Rev. Food Sci. Nutr. 49,
682–689.

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
Kheifets, L., Bowman, J. D., Checkoway, H., Feychting, M., Harrington, J. M.,
REFERENCES Kavet, R., Marsh, G., Mezei, G., Renew, D. C., and van Wijngaarden, E.
(2009). Future needs of occupational epidemiology of extremely low
Bayne-Jones, S., Burdette, W., Cochran, W., Farber, E., Fieser, L., Furth, J., frequency electric and magnetic fields: review and recommendations. Occup.
Hickam, J., LeMaistre, C., Schuman, L., and Seevers, M. (1964). In Smoking Environ. Med 66, 72–80.
and Health: Report of the Advisory Committee to the Surgeon General of the Kheifets, L., Monroe, J., Vergara, X., Mezei, G., and Abdelmonem, A. (2008).
Public Health Service. Department of Health, Education, and Welfare Public Occupational electromagnetic fields and leukemia and brain cancer: an
Health Service. Public Health Service Publication No. 1103 Superintendent update to two meta-analyses. J Occup Environ Med 50, 677–688.
of Documents. U.S. Government Printing Office,Washington, DC. Klaunig, J. E., Babich, M. A., Baetcke, K. P., Cook, J. C., Corton, J. C.,
Boobis, A. R., Cohen, S. M., Dellarco, V., McGregor, D., Meek, M. E., David, R. M., DeLuca, J. G., Lai, D. Y., McKee, R. H., Peters, J. M., et al.
Vickers, C., Willcocks, D. and Farland, W. (2006). IPCS framework for (2003). PPAR alpha agonist-induced rodent tumors: modes of action and
analyzing the relevance of a cancer mode of action for humans. Crit. Rev. human relevance. Crit. Rev. Toxicol. 33, 655–780.
Toxicol. 36, 781–792. Koch, R. (1884). Die Aetiologie der Tuberkulose. Mitt Kaiser Gesundh 2:1–88.
Boobis, A. R., Doe, J. E., Heinrich-Hirsch, B., Meek, M. E., Munn, S., Koch, R. (1893). Über den augenblicklichen Stand der bakteriologischen
Ruchirawat, M., Schlatter, J., Seed, J., and Vickers, C. (2008). IPCS Choleradiagnose (in German). Zeitschrift für Hygiene und
framework for analyzing the relevance of a noncancer mode of action for Infectionskrankheiten 14, 319–333.
humans. Crit. Rev. Toxicol. 38, 87–96. Kundi, J. M. (2007). Causality and the interpretation of epidemiological
Cohen, S. M., Meek, M. E., Klaunig, J. E., Patton, D. E., and Fenner- evidence. Environ. Health Perspect. 114, 969–974.
Crisp, P. A. (2003). The human relevance of information on carcinogenic Lagiou, P., Adami, H., and Trichopoulos, D. (2005). Causality in cancer
modes of action: overview. Crit. Rev. Toxicol. 33, 581–589. epidemiology. Eur. J. Epidemiol. 20, 565–574.
Colditz, G. A., Baer, H. J., and Tamimi, R. M. (2006). Breast cancer. In Cancer London Principles. (1996). The London Principles for Evaluating
Epidemiology and Prevention (D. Schottenfeld, FraumeniJ. F.Jr.), 3rd ed. Epidemiologic Data in Regulatory Risk Assessment. Available at: http://
Oxford University Press, New York. www.fedfocus.org/science/london.html.
Cole, P. (1997). Causality in epidemiology, health policy and law. Environ. Meek, M. E., Bucher, J. R., Cohen, S. M., Dellarco, V., Hill, R. N., Lehman-
Law Rep. 27, 10279–10285. McKeeman, L. D., Longfellow, D. G., Pastoor, T., Seed, J., and Patton, D. E.
ECETOC. (2009). Framework for the Integration of Human and Animal Data (2003). A framework for human relevance analysis of information on
in Chemical Risk Assessment. Technical Report No. 104 ISSN-0773-8072- carcinogenic modes of action. Crit. Rev. Toxicol. 33, 591–653.
104. European Centre for Ecotoxicology and Toxicology of Chemicals, Pastoor, T., and Stevens, J. (2005). Historical perspective of the cancer
Brussels, Belgium. bioassay. Scand. J. Work Environ. Health 31(Suppl. 1), 129–140.
Forsyth, B. W., Horwitz, R. I., Acampora, D., Shapiro, E. D., Viscoli, C. M., Phillips, C. V., and Goodman, K. J. (2004). The missed lessons of Sir Austin
Feinstein, A. R., Henner, R., Holabird, N. B., Jones, B. A., Karabelas, A. D. E., Bradford Hill. Epidemiol. Perspect. Innov. 1, 3.
et al. (1989). New epidemiologic evidence confirming that bias does not explain Popper, K. (1959). In The Logic of Scientific Discovery. First printed in English
the aspirin/Reye’s syndrome association. J. Am. Med. Assoc. 261, 2517–24. by Hutchinson & Co.; Republished by Ruteledge, 2006, New York, NY.
Graham, J. D. (1995). The role of epidemiology in regulatory risk assessment. Potter, M. (1962). Percivall Pott’s contribution to cancer research. NCI
In Proceedings of the Conference on the Proper Role of Epidemiology in Monograph. 10, 1–13.
Risk Analysis, 13–14 October 1994, Boston, MA. Elsevier Science Ltd, New Qian, G. S., Ross, R. K., Yu, M. C., Yuan, J. M., Gao, Y. T., Henderson, B. E.,
York, NY. Wogan, G. N., and Groopman, J. D. (1994). A follow-up study of urinary
Guzelian, P. S., Victoroff, M. S., Halmes, N. C., James, R. C., and markers of aflatoxin exposure and liver cancer risk in Shanghai, People’s
Guzelian, C. P. (2005). Evidence-based toxicology: a comprehensive frame- Republic of China. Cancer Epidemiol. Biomarkers Prev. 3, 3–10.
work for causation. Hum. Exp. Toxicol. 24, 161–201. Rothman, K. J. (1976). Causes. Am. J. Epidemiol. 104, 578–592.
Halpin, T. J., Holtzhauer, F. J., Campbell, R. J., Hall, L. J., Correa- Rothman, K. J., and Greenland, S. (2005). Causation and causal inference in
Villasenor, A., Lanese, R., Rice, J., and Hurwitz, E. S. (1982). Reye’s epidemiology. Am. J. Public Health 95, S144–S150.
syndrome and medication use. J. Am. Med. Assoc., 248, 687–691. Sarid, R., Klepfish, A., and Schattner, A. (2002). Virology, pathogenic
Hill, A. B. (1965). The environment and disease: association or causation. mechanisms and associated diseases of Kaposi sarcoma-associated herpes-
Proc. R. Soc. Med. 58, 295–300. virus (Human herpesvirus 8). Mayo Clin. Proc. 77, 941–949.
Hunter, D. J., Spiegelman, D., Adami, H.-O., Beeson, L., van den Seed, J., Carney, E., Corley, R., Crofton, K., DeSesso, J., Foster, P.,
Brandt, P. A., Folsom, A. R., Graser, G. E., Goldbohm, R. A., Kavlock, R., Kimmel, G., Klaunig, J., Meek, E., et al. (2005). Overview:
234 ADAMI ET AL.

using mode of action and life stage information to evaluate the human U.S. Environmental Protection Agencty (USEPA) (2001). HED Standard
relevance of animal toxicity data. Crit. Rev. Toxicol. 35, 663–672. Operating Procedure: Executive Summaries for Toxicology Data Evaluation
Simpkins, J. W., Swenberg, J. S., Weiss, N. S., Brusick, D., Eldridge, J. C., Records (DERs) SOP 2001.02, p. 7.
Stevens, J. T., Handa, R. G., Hovey, R. C., Plant, T. M., Pastoor, et al. von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., and
(2011). Atrazine and breast cancer: a framework assessment of the Vandenbroucke, J. P. (2007). The strengthening of the reporting of
toxicological and epidemiological evidence. Tox. Sci. Advance Access
observational studies in epidemiology (STROBE) statement: guidelines for
published on July 18, 2011; doi: doi:10.1093/toxsci/kfr176.
reporting observational studies. PLoS Med 4, 1623–1627.
Sonich-Mullin, C., Fielder, R., Wiltse, J., Baetcke, K., Dempsey, J.,
Fenner-Crisp, P., Grant, D., Hartley, M., Knaap, A., Kroese, D., et al. Wang, L. Y., Hatch, M., Chen, C. J., Levin, B., You, S. L., Lu, S. N.,
(2001). IPCS conceptual framework for evaluating a mode of action for Wu, M.-H., Wu, W.-P., Wang, L.-W., Wang, Q., et al. (1996). Aflatoxin
chemical carcinogenesis. Regul. Toxicol. Pharmacol. 34, 146–152. exposure and risk of hepatocellular carcinoma in Taiwan. Int. J. Cancer 67,
Susser, M. (1986). The logic of Sir Karl Popper and the practice of 620–625.

Downloaded from http://toxsci.oxfordjournals.org/ at University of Hawaii, PBRC, Kewalo Marine Lab. on February 24, 2015
epidemiology. Am. J. Epidemiol. 124, 711–718. Weed, D. L. (2005). Weight of evidence: a review of concepts and methods.
Swenberg, J. A., and Lehman-McKeeman, L. D. (1999). a2 Urinary-globulin- Risk Anal. 25, 1545–1557.
associated nephropathy as a mechanism of renal tubule cell carcinogenesis in Whysner, J., Ross, P. M., and Williams, G. M. (1996). Phenobarbital mechanistic
male rats. In Species Differences in Thyroid Kidney and Urinary Bladder data and risk assessment: enzyme induction, enhanced cell proliferation and
Carcinogenesis. IARC Scientific Publications No. 147 (C. Capen, E. Dybing, tumor promotion. Pharmacol. Ther. 71, 153–191.
J. Rice, and J. Wilbourne, Eds.). Lyon, France, pp. 95–118. WHO Press, Geneva,
Wilbourn, J., Haroun, L., Vainio, H., and Montesano, R. (1984). Identification
Switzerland.
of chemicals carcinogenic to man. Toxicol. Pathol. 12, 397–399.
Tabuenca, J. M. (1981). Toxic-allergic syndrome caused by ingestion of
rapeseed oil denatured with aniline. Lancet 2, 567–568. World Health Organization. (2009). Report of a WHO expert meeting
Tomatis, L., Aitio, A., Wilbourn, J., and Shuker, L. (1989). Human carcinogens in collaboration with FAO supported by Heath Canada.
so far identified. Jpn J Cancer Res 80, 795–807. Toxicological and Health Aspects of Melamine and Cyanuric Acid. WHO
U.S. Environmental Protection Agencty (USEPA) (1993). Pesticide Reregis- Press, Geneva, Switzerland.
tration Rejection Rate Analysis—Toxicology National Service Center for Yamagiwa, K., and Ichikawa, K. (1918). Experimental study of the
Environmental Publications 738R93004, p. 22. pathogenesis of carcinoma. J. Cancer Res. 3, 1–29.