A Guide for Monitoring Child Development in Low- and Middle-Income

Countries
Ilgi O. Ertem, Derya G. Dogan, Canan G. Gok, Sevim U. Kizilates, Ayliz Caliskan,
Gulsum Atay, Nilgun Vatandas, Tugba Karaaslan, Sevgi G. Baskan and Domenic V.
Cicchetti
Pediatrics 2008;121;e581
DOI: 10.1542/peds.2007-1771

The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://pediatrics.aappublications.org/content/121/3/e581.full.html

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

publication, it has been published continuously since 1948. PEDIATRICS is owned,
published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point
Boulevard, Elk Grove Village, Illinois, 60007. Copyright 2008 by the American Academy
of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014

ARTICLE

A Guide for Monitoring Child Development in Low-

and Middle-Income Countries
Ilgi O. Ertem, MDa, Derya G. Dogan, MDa, Canan G. Gok, MSca, Sevim U. Kizilates, MDa, Ayliz Caliskan, MDa, Gulsum Atay, MDb,
Nilgun Vatandas, MDb, Tugba Karaaslan, MSca, Sevgi G. Baskan, MDa, Domenic V. Cicchetti, PhDc

aDevelopmental-Behavioral Pediatrics Unit, Department of Pediatrics, Ankara University School of Medicine, Ankara, Turkey; bDepartment of Pediatrics,
Baskent University School of Medicine, Ankara, Turkey; cChild Study Center and Departments of Psychiatry, Epidemiology, and Public Health in Biometry,
Yale University School of Medicine, New Haven, Connecticut

The authors have indicated they have no nancial relationships relevant to this article to disclose.

ABSTRACT
OBJECTIVE. In low- and middle-income countries, methods for clinicians to address
difficulties in language, social-emotional, cognitive, behavioral, or neuromotor de-
velopment during early childhood are lacking. To fill this gap, we designed, in www.pediatrics.org/cgi/doi/10.1542/
peds.2007-1771
Turkey, the Guide for Monitoring Child Development, which aims to aid clinicians in
monitoring and supporting child development and the early detection and manage- doi:10.1542/peds.2007-1771
ment of developmental difficulties. The Guide for Monitoring Child Development Key Words
child development, developing countries,
monitoring component is a practical, open-ended interview that catalyzes commu- surveillance and monitoring,
nication between clinicians and caregivers and obtains a portrayal of the childs developmental screening, screening tools
development. We report on the development and psychometric properties of the Abbreviations
Guide for Monitoring Child Development monitoring component for children aged LAMIlow and middle income
0 to 24 months. GMCDGuide for Monitoring Child
Development
METHODS. We examined the ages of attainment of Guide for Monitoring Child Devel- WHOWorld Health Organization
UNICEFUnited Nations International
opment milestones and internal consistency in a cross-sectional study of healthy Childrens Education Fund
children receiving well-child care (study 1). In 2 clinical samples, we studied the Accepted for publication Jul 31, 2007
interrater reliability between medical students and a child development specialist Address correspondence to Ilgi O. Ertem, MD,
administering the guide (study 2), as well as the concurrent validity of the guide Developmental-Behavioral Pediatrics Unit,
administered during a health visit and a comprehensive developmental assessment Department of Pediatrics, Ankara University
School of Medicine, Cebeci, Ankara, 06100
(study 3). Turkey. E-mail: ertemilgi@yahoo.com

RESULTS. In study 1 (N 510), item-total scale correlations ranged from 0.28 to 0.91. PEDIATRICS (ISSN Numbers: Print, 0031-4005;
Online, 1098-4275). Copyright 2008 by the
An age-dependent attainment pattern was seen in all of the milestones. In study 2 American Academy of Pediatrics
(N 92), interrater reliability between medical-student pairs and between the child
development specialist and students was high (kappa scores were 0.83 0.88). In
study 3 (N 79), the sensitivity, specificity, and positive and negative predictive values were 0.88, 0.93, 0.84, and
0.94, respectively.

CONCLUSIONS. The Guide for Monitoring Child Development is an innovative method for monitoring child development
that is designed specifically for use by health care providers in low- and middle-income countries. Studies in Turkey
provide preliminary evidence for its reliability and validity.

I N LOW- AND middle-income (LAMI) countries,1 as childhood mortality has decreased, developmental difficulties,
including disabilities, disorders, or delays in cognitive, language, social-emotional, behavioral, or neuromotor
development that begin during early childhood are increasingly recognized as important contributors to morbidity
across the life span.26 In high-income countries, an important strategy for the early detection and management of
developmental difficulties has been the integration of developmental monitoring of children (ie, standardized
screening and surveillance) into health care.712 In LAMI countries, health services are often the only professional
services available to young children and offer, particularly during the first 2 years of life, important opportunities to
address child development.3,4,13 Interventions to enhance the development of young children are increasingly
becoming available in developing countries and include low-cost strategies, such as addressing malnutrition and iron

PEDIATRICS Volume 121, Number 3, March 2008 e581

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
deficiency, improving caregiver-child relationships, in- Several differences between high-income and LAMI
creasing psychosocial stimulation, and establishing com- countries need to be addressed, however, when adapting
munity-based rehabilitation.1317 To date, however, or developing instruments that can be used in LAMI
methods designed specifically for developmental moni- countries. First, low caregiver literacy limits the use of
toring of young children by health care providers in written questionnaires and checklists. Second, in popu-
LAMI countries are lacking.26,13,14,18 lations where developmental difficulties are prevalent,
Research in Western countries has shown that chil- caregivers may not have an appropriate reference as to
dren and their caregivers benefit from developmental how children should typically develop. Although iden-
monitoring during health visits in a number of ways: (1) tifying caregivers concerns is of key importance in de-
if the child is developing typically, clinicians can provide velopmental monitoring, reliance on this method alone
reassurance, support parenting competence, and provide for early identification of developmental difficulties in
anticipatory guidance; (2) if the child is at developmen- LAMI countries requires further examination.27,31 Third,
tal risk or has an established or emerging delay or diffi- the use of structured questions or checklists about mile-
culty, this can be detected early and addressed; and (3) stones may also be problematic. Caregivers may not
in both situations, caregivers can be supported and in- readily admit that their child has not reached a mile-
formed about how to enhance their childs develop- stone and may provide socially desirable answers, par-
ment.712 At a population level, developmental monitor- ticularly if they do not receive health care from the same
ing can inform policy about rates of developmental trusted clinician at each visit, do not believe that inter-
difficulties so that existing interventions can be appro- ventions exist, or are concerned about stigma related to
priately allocated, their effect can be monitored, and the developmental difficulties. Fourth, the alternative reli-
need for further interventions can be determined.2,56,13,17 ance on child testing methods is neither practical nor
In the United States, the implementation of develop- desirable. Caregivers do know their children best, and
mental monitoring and the early detection of develop- even more so than in Western countries, they are the
mental difficulties have made only limited progress key resource to support childrens development. There-
without the use of standardized instruments and proto- fore, family centered methods for monitoring child de-
cols; therefore, the American Academy of Pediatrics cur- velopment that have evolved in the West should be the
rently recommends that standardized instruments be methods of choice for developing countries as well. Test-
used.7,19,20 Such instruments have evolved in 2 areas in ing of the child most often leaves the caregiver watch-
the past 40 years.2124 First, language, social-emotional, ing rather than participating in the evaluation, and,
cognitive, and behavioral development, as well as func- therefore, this approach does not capitalize on the part-
tional capacity, have become essential constructs that nership of clinicians with caregivers. Furthermore, it is
are incorporated into instruments. Second, the impor- difficult to elicit the optimal developmental functioning
tance of caregiver-clinician communication and partner- of young children during health care visits. When ob-
ship has been reflected in the methods used for devel- jects are needed to elicit childrens skills, the cleanliness
opmental monitoring. Based on the family centered care and maintenance of such objects may be a concern in
initiative in pediatrics and advances in early interven- LAMI countries. Fifth, instruments that are to be used
tion, models in which a parent watches while a clinician across LAMI countries need to include universal and not
tests the child have evolved to models in which a culture-specific concepts in child development. Sixth,
caregiver and clinician use instruments to talk about monitoring child development is a new concept for
the childs development. Many instruments that ask health care providers in LAMI countries, and methods
caregivers about their concerns regarding their childs that are to facilitate monitoring should build on existing
development and/or whether their child has achieved protocols (such as growth monitoring and immuniza-
certain developmental milestones have been shown to tions) and clinicians skills (such as clinician-patient
have appropriate psychometric properties as screening communication).
tools and are now recommended and widely used in A number of instruments that were designed to in-
Western countries.2224 volve caregivers in the monitoring process and were
Studies from LAMI countries suggest that caregiv- developed in Western countries3236 have the potential to
ers2527 and health care providers2830 may not be well be adapted for use in developing countries. None of
equipped with knowledge about early childhood devel- these instruments, however, has been designed specifi-
opment, and, therefore, the need for instruments in the cally to address concerns in developing countries. There-
monitoring process may be even more important than in fore, the Guide for Monitoring Child Development
high-income countries. Ideally, in both high-income and (GMCD) was developed for use by health care providers
LAMI countries, methods used for developmental mon- in LAMI countries to monitor the development of chil-
itoring in health systems should (1) be based on well- dren 0.0 3.5 years of age. The GMCD has 3 components:
supported current theories and conceptualizations of (1) the monitoring development component, which is
child development, (2) be reliable and valid, (3) be reported here; (2) the supporting development compo-
linked directly to frameworks for supporting develop- nent, which is an expanded version of the World Health
ment and managing developmental problems when they Organization (WHO)/United Nations International Chil-
are detected, (4) be brief, easy to learn, and easy to drens Education Fund (UNICEF) Care for Development
administer, and (5) require minimal cost, equipment, Intervention15,37 and has been incorporated by the WHO
and paperwork. into the newly launched International Growth Stan-

e582 ERTEM et al
Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
 TABLE 1 GMCD Questions and Examples of Milestones
Developmental Domains and 67 mo 810 mo
Interview Questions
1. Concerns32: By development I mean her
learning, understanding, communicating,
relationships, her behavior and emotions,
how she uses her ngers and hands, legs
and body, her hearing and vision. Do you
have any concerns about Ayses
development in any of these areas?
2. Expressive language and communication: Makes ga, gu, da, ba, sounds (joins Repeats syllables like da-da
Tell me about how she communicates. vowels and consonants) Uses gestures like shaking head
How does Ayse let you know when she in protest
wants something?
3. Receptive language: Tell me examples of When caregiver speaks listens, looks at Understands repeated simple
what she can understand when you talk her mouth words like mummy no
to her? Recognizes and prefers caregivers
voice
Responds with sounds when talked to
4. Fine and gross motor: What does Ayse Reaches with hands Transfers from hand to hand
do with her hands and ngers and with Holds onto toys or objects Picks up small objects like raisins
her legs and body? Sits with support Turns front to back to front
Bears weight on legs Sits without support
5. Relationship (social-emotional): Tell me Recognizes caregivers, reaches to them, Reacts when mother leaves
about Ayses relationships with people smiles, inspects their faces May turn away from strangers in
she knows. What about strangers, how anxiety, caution, shyness or
does she relate to them? fear
6. Play (social-emotional, cognitive): Id like Regards hands Inspects toys with curiosity
to learn about her play, can you give me Shakes objects Throws, bangs toys, objects
examples? Responds to peek-a-boo Looks for objects
Plays peek-a-boo
7. Self-help skills: What kinds of things can
Ayse do for herself now, like eating or
dressing?

dards training package38; and (3) the managing develop- well as 2 well-known and widely used instruments, the
mental difficulties component, which has been adopted Parents Evaluation of Developmental Status32 and the
by the Turkish Ministry of Health and UNICEF-Turkey to Vineland Scales of Adaptive Behavior.33 The open-ended
be used in a nationwide training program on child de- interview technique was chosen as the administration
velopment for primary health care providers. We report method for the GMCD. This technique is based on the
here a description of the GMCD monitoring develop- fundamental and universal principles of human commu-
ment component, referred to as the GMCD, and the nication and recall of information,44 builds on patient-
results of research in Turkey on its development, inter- centered communication techniques,45,46 avoids assump-
rater reliability, and validity for children aged 0 to 24 tions about what the child should be doing, lessens the
months. possibility that caregivers will provide socially desirable
answers, and is designed to be culturally neutral.
METHODS The GMCD is a brief, open-ended, precoded interview
Description of the GMCD with the primary caregiver. The interview is adminis-
The GMCD aims to provide a method for developmental tered in the following standard way. Caregivers are first
monitoring and early detection of developmental diffi- provided with an explanation of the reason for the in-
culties in LAMI countries and is rooted in the ecological terview, and caregiver interest and cooperation are elic-
and transactional conceptualizations of child develop- ited. Table 1 shows examples of the questions of the
ment,39,40 family centered care,41 and relationship- and GMCD and the way it is structured. The first question is
strengths-based developmental assessment.21 To develop adapted from the Parents Evaluation of Developmental
the GMCD, a comprehensive search of the literature was Status32 and relates to identifying parental concerns. If
conducted, and consultations were held with experts on the caregiver expresses concerns, these are explored fur-
child development in both Western and developing ther before the interview is continued. Next, the clini-
countries. Core ideas were adapted from 3 seminal mod- cian explains the importance of obtaining a portrayal of
els: New Visions for Developmental Assessment of the childs typical functioning and asks the 6 open-ended
Young Children,21 Bright Future Guidelines,42 and the questions shown in Table 1. The questions (27) relate to
WHO International Classification of Functioning,43 as the following developmental domains: (question 2) ex-

PEDIATRICS Volume 121, Number 3, March 2008 e583

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
pressive language and communication, (question 3) re- to child health and development (such as malnutrition,
ceptive language, (question 4) gross and fine motor, low birth weight, chronic infections including HIV/
(question 5) relationship (social-emotional), (question AIDS, parasitic infestations, iron deficiency anemia, and
6) play, and (question 7) self-help skills (for children perinatal complications), references for monitoring
older than 12 months). For each of the 6 questions, growth and development should be based on what the
there are specific precoded milestones. Caregivers spon- WHO refers to as a prescriptive sample of healthy,
taneous responses to the open-ended questions are ap- thriving children without these risks rather than geo-
plied to the milestones whenever possible. Additional graphic, whole-population-based references.50,51 The
questions are used when necessary to prompt responses participants were children who, from birth onward, had
to specific milestones. The GMCD does not include ques- received preventive health care at 2 university-affiliated
tions for the cognitive domain, because for young chil- community well-child care clinics in Ankara (Ankara
dren it is difficult for a caregiver to narrate aspects of and Baskent Universities). The intended sample size was
cognitive development separately from language, relat- 30 children of each gender in 8 age ranges under 25
ing, and play skills. Cognitive development is addressed months, resulting in a total of 480 children. All of the
in the domains and in the first question, specifically eligible children were enrolled and participated in the
asking if the caregiver is concerned about the childs study. The pediatricians in the clinics who conducted
cognitive development, using explanations such as the well-child visit examination completed the health
thinking, using his mind, and intelligence. record. At the end of the visit, a developmental-behav-
The GMCD form, which is used to ascertain whether ioral pediatrician reviewed the health records and ap-
a child is developing typically and to document the plied the following inclusion criteria to identify the final
monitoring, is composed of 2 tables provided on each sample that was used in the analyses. Children were
side of a single sheet. The questions are placed in rows, included in the sample if they were born as healthy
the 8 age ranges (13, 4 5, 6 7, 8 10, 1113, 14 16, singletons with birth weight 2500 g and gestational age
1719, and 20 24 months) are placed in columns, and 37 weeks; their growth had been between fifth to 95th
the precoded developmental milestones are in the cells. percentiles since birth; they had received and complied
The age ranges were selected based on the schedules for with the free iron prophylaxis available to children in
monitoring growth and providing immunizations rec- Turkey after 6 months of age or their hemoglobin
ommended by the WHO. The milestones were con- screens had been normal within 1 month of the study;
structed by the following method. We pooled all of the and the pediatric evaluation and chart review concluded
developmental milestones from 5 previously standard- that they were healthy, growing normally, and had not
ized and validated developmental screening or assess- had any health-related problems since birth apart from
ment instruments: Denver II,47 Vineland,33 Brigance acute minor illnesses.
Screening Test,48 Ages and Stages Questionnaire,34 and
Bayley Scales of Infant Development Second Edition
Procedures
(Bayley II).49 Three experts in child development se-
Four researchers were trained in the administration of
lected milestones that could be easily observed and re-
the GMCD and reached 95% interrater agreement on 25
ported by caregivers in response to the GMCD questions
consecutive administrations. At both sites, eligible care-
and that were thought to be universal (eg, interactive
givers were invited to participate in the study while they
play such as peek-a-boo) and not culture specific (eg,
were waiting for their health visit or after the visit.
playing with a mirror). A pilot study was conducted to
Caregivers were informed that this was a study to obtain
assess the appropriateness and comprehension of the
normative information about child development. The
questions and milestones. This pilot study included 150
standard open-ended method of administration was
caregivers of children referred to Ankara University from
used; 1 modification was made for the study. To avoid
many regions of Turkey. Some of the GMCD milestones
asking caregivers about milestones normally occurring
were subsequently revised based on caregivers narra-
much earlier or later than the age of the child, mile-
tions of their childs development.
stones were temporarily placed in age rows based on the
Three studies were conducted in Turkey on the con-
following process. The 5 instruments33,34,4749 used to con-
struction and psychometric properties of the GMCD, for
struct the GMCD were examined, and the milestone was
children aged 0 24 months. Study 1 aimed to determine
placed in the age column in which 90% of children in
the ages of attainment of the GMCD milestones, study 2
1 of the instruments achieved this milestone. After the
sought to examine ease of administration and interrater
caregivers spontaneous responses were recorded, they
reliability, and study 3 served to examine concurrent
were told that there may be other things that the child
validity of the GMCD with a comprehensive develop-
may or may not be doing. Milestones 1 below and 1
mental assessment.
above the childs age range were read to the caregiver
who was then asked whether the child had attained
Study 1: Ages of Attainment of the GMCD Milestones these milestones.
Participants
We used the WHO recommendations to select the study Data Analysis
sample. The WHO recommends that, in populations Internal consistency of the GMCD was examined by
with a high prevalence of conditions that are hazardous computing item-total score correlation for each mile-

e584 ERTEM et al
Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
stone and Cronbachs for each domain and the total TABLE 2 Sociodemographic Characteristics of participants in
instrument. Item-total scale correlations of .25 and Studies 1, 2, and 3
Cronbachs of .70 were considered appropriate.52 The
Sociodemographic Study 1 Study 2 Study 3
presence of an age-dependent developmental pattern Characteristics (N 510), % (N 92), % (N 79), %
was examined using Pearson correlation coefficients for
Gender of child, male 50.6 52.2 46.8
each domain score and age. The 2 test was used to
Age of child, mo
examine whether significant differences existed between 16 29.8 30.4 5.1
girls and boys in the ages of attainment of developmental 712 28.8 27.2 29.1
milestones. We computed the age at which 90% of the 1324 41.4 43.4 65.8
study sample performed each milestone, which was then Caregiver education
placed in the corresponding age column. Illiterate 0.0 13.0 0.0
Elementary school 2.4 51.1 26.6
Secondary or high school 28.2 20.7 30.4
Study 2: Ease of Administration and Interrater Reliability University 69.4 15.2 43.0
In study 2, 1118 fifth-year medical students at Ankara Caregiver employment
University School of Medicine were trained in the use of Employed 69.4 21.7 43.0
the GMCD; the training involved a 1.0-hour seminar and a Unemployed 30.6 78.3 57.0
1.5-hour practicum. Of these, 184 randomly selected fifth-
year students were paired as partners. The paired students
administered the GMCD to caregivers of 92 children aged
0 24 months who received health care at Ankara Univer- Index was 2 SDs below the mean of the US reference
sity School of Medicine Department of Pediatrics. While 1 (score 70). , sensitivity, and specificity were used to
student administered the GMCD, the other observed, and examine concurrent validity.53
both completed the GMCD independently; administration For all 3 of the studies, data were analyzed using SPSS
time was recorded. A child development specialist with 11.0 (SPSS Inc, Chicago, IL).54 Written, informed con-
extensive experience on the GMCD administered the sent from caregivers was obtained for all 3 of the studies,
GMCD to the same caregivers within 48 hours of the which were approved by Ankara University School of
medical students. The students completed a questionnaire Medicine Ethics Committee.
related to ease of administration, and the child develop-
ment specialist asked caregivers questions about the ease of RESULTS
responding to the GMCD. Percentage of agreement and
Study 1: Ages of Attainment of the GMCD Milestones
were used to examine interrater reliability for the overall
Of the 546 families invited to participate in the study, 18
GMCD result. We used as our criterion for interrater reli-
were excluded because of iron deficiency anemia (n
ability a value of 0.60, defining a level of good
12), chronic illness (n 2), or history of perinatal as-
chance-corrected interrater agreement.52
phyxia (n 4). Of the 528 families who met study
inclusion criteria, 18 declined, resulting in a 97% re-
Study 3: Concurrent Validity sponse rate. The sociodemographic characteristics of the
Preliminary criteria for interpreting the GMCD were de- remaining sample of 510 children are shown in Table 2.
veloped to examine concurrent validity. If the child was The sample was 51% male, and 59% were 12 months
reported to exhibit all of the milestones at age level, the of age. Most mothers had at least a secondary school
GMCD interpretation was classified as appropriate for education (97.6%) and worked (69.4%). Most were
age. If the child did not demonstrate 1 of the age- nuclear families (86.7%), and in 58.1%, the mother was
appropriate milestones, the GMCD interpretation was the childs full-time caretaker.
classified as requires follow-up evaluation with or with- Item-total scale correlations ranged from 0.29 to 0.91;
out intervention. In a cross-sectional study, we exam- 84% of the 89 milestones had item-total correlations
ined the concurrent validity of the GMCD. Infants aged 0.40. Internal consistency measured by the Cronbachs
124 months who had been born with birth weight was high,50 ranging from .80 to .96 for each of the 6
1500 g, treated in the NICU at Ankara University, and domains and was .95 for the total GMCD. An age-de-
who came for health visits to the follow-up clinic were pendent developmental pattern was seen in all of the
enrolled consecutively over a 6-month period. A pedia- milestones, as reflected by Pearson correlation coeffi-
trician with no specific training in child development but cients for each domain score and age in months ranging
trained in the use of the GMCD administered the GMCD from 0.88 to 0.96 (P .001). No statistically significant
at the time of a clinic visit. An experienced developmen- differences were found between the mean ages of attain-
tal-behavioral pediatrician, blinded to the GMCD re- ment for girls and boys on any of the milestones or
sults, conducted a comprehensive developmental assess- domains. Therefore, 1 reference table was constructed
ment within 48 hours. This evaluation included a for both genders. Table 3 shows an example of the
developmental history, play observations, neurologic ex- developmental progression of the ages of attainment of 3
amination, and the Bayley II.49 The comprehensive eval- selected milestones. A similar age-dependent develop-
uation was considered positive if the clinician decided mental pattern was seen in all of the milestones. The
that the child needed developmental interventions or milestones were placed in the age ranges based on the
the Bayley Mental Index or Psychomotor Development 90% cutoff; the final GMCD had between 9 and 15

PEDIATRICS Volume 121, Number 3, March 2008 e585

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
 TABLE 3 Examples of Age-Dependent Attainment of Selected positive accuracy value of 0.84 and a predicted negative
Milestones accuracy value of 0.94.
Age, n N (%)
mo
DISCUSSION
Laughs Makes Sounds Joining Repeats Syllables The GMCD is a novel method with the ultimate goal of
Aloud Vowels and Consonants Like da-da facilitating developmental monitoring and the early detec-
(eg, ga or da)
tion of developmental difficulties of children in LAMI
02 26 1 (5.3) 0 (0.0) 0 (0.0) countries. The theoretical conceptualization of the GMCD
3 29 9 (31.0) 1 (3.4) 0 (0.0) is anchored in family centered, relationship, and strengths-
4 30 20 (66.7) 7 (23.3) 0 (0.0)
based developmental assessment theory,21,24,3941 and the
5 34 32 (94.1)a 13 (38.2) 0 (0.0)
6 27 27 (100.0) 17 (63.0) 4 (14.8)
techniques used are adapted from currently used strategies
7 42 42 (100.0) 41 (97.6)a 18 (42.9) in screening and surveillance.32,33,42 The GMCD advances
8 18 18 (100.0) 18 (100.0) 14 (77.8) the field conceptually by introducing a new and practical
9 17 17 (100.0) 17 (100.0) 16 (94.1)a method of using an open-ended interviewing technique to
10 37 37 (100.0) 37 (100.0) 37 (100.0) obtain comprehensive information about the childs devel-
a Data show the age after which 90% of sample attained the milestone. opmental functioning. Brevity of the training, administra-
tion, and scoring and the 1-sheet instrument offer a prac-
tical approach to developmental monitoring. The 3 studies
in Turkey have provided promising results on the psycho-
milestones, respectively, for each of the 8 age ranges metric properties of the GMCD. Internal consistency was
from 0 to 24 months. high, as measured by item-total score correlations and
Cronbachs for the domains and total score. The sample
Study 2: Ease of Administration and Interrater Reliability size of 64 children in each of the 8 age ranges was
The sociodemographic characteristics of the sample in adequate to demonstrate a stable, age-dependent attain-
study 2 are shown in Table 2. Most caregivers in this ment of the milestones. The 90th percentile cutoff point
sample had 5 years of education (64%), and 78% were was used for placing milestones in age ranges; the percen-
unemployed mothers. After the standard training, 96% tile method has been used previously in other instruments,
of medical students stated that the GMCD was easy to such as the Denver II.47 Medical students with no previous
administer and that within the brief encounter they background on child development were able to learn the
achieved good communication with caregivers. The GMCD after a brief training and administer it reliably.
mean administration time by the students was 7 2.3 Sensitivity and specificity were in the range that is consid-
minutes. Almost all of the caregivers (98%) stated that ered good to excellent for the diagnostic accuracy of devel-
the questions of the GMCD were easy to understand and opmental screening tests.53
answer. For the total sample, interrater reliability be- In high-income countries, different instruments have
tween the student pairs (agreement: 93.4%; 0.84; been standardized and validated for developmental
P .001) and between the child development specialist monitoring and the early detection of developmental
and the students (student 1: agreement: 94.5%; difficulties.7,22,23 In LAMI countries, research on child
0.88; P .001; student 2: agreement: 92.3%; 0.83; development is extremely limited.2,5,6,13,14 Instruments
P .001) were high.52 Between student pairs, was 0.79 such as the Ten Questions Questionnaire,5557 Access
(P .001) for caregivers with a primary school educa- Portfolio,58 and Disability Screening Schedule59 have
tion or less and 0.93 (P .001) for caregivers with at been developed but are designed to question caregivers
least a secondary school education. about whether a child has a severe disability and do not
provide a framework for monitoring the development of
Study 3: Concurrent Validity young children. The Denver test47 has been used in
The sociodemographic characteristics of the sample in many countries, but decades after large standardization
study 3 are shown in Table 2. Most caregivers had a high and restandardization studies in many different LAMI
school education or less (57%) and were unemployed countries, research on the use of the Denver test in
(57%) mothers. The development of the former very promoting developmental monitoring and child devel-
low birth weight children was found to be appropriate opment is lacking. The Denver test relies on child test-
for age in 68.4% with the GMCD and in 69.6% with ing and structured questions, both of which are not
the comprehensive evaluation. The average agreement ideal in LAMI countries. The training does not stress the
on positive (developmental delay) and negative (no de- importance of developing partnerships with caregivers
lay) cases was 21 (87.5%) of 24 and 51 (92.7%) of 55. or of promoting child development and does not have a
The weighted sum of these 2 indices of average agree- component that can be used for planning interventions.
ment produced the overall agreement of 72 (91.1%) of Furthermore, the Denver II test is less commonly used in
79 and a of 0.79 (P .001). The sensitivity and the West than it was previously because of research
specificity rates were 21 of 24 (0.88; 95% confidence demonstrating its inadequate screening accuracy.60
interval: 0.69 0.96) and 51 of 55 (0.93; 95% confidence Each of the 3 studies has limitations in sampling and
interval: 0.83 0.97), respectively, which represent methodology. The study on the ages of attainment of
good and excellent levels of diagnostic accuracy.53 GMCD milestones (study 1) is only generalizable to
Based on this clinic sample, we obtained a predicted healthy children receiving well-child care in Ankara. In

e586 ERTEM et al
Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
Turkey, 4 risk factors that are associated with develop- curacy only pertains to the clinical sample on which it is
mental difficulties in young children are prevalent: mal- based. Future research is required to test whether similar
nutrition, iron deficiency anemia, low birth weight, and results will be obtained for a population-based sample.
chronic illness.61 In study 1, therefore, we used the pre- The sample size was not adequate to determine whether
scriptive sample approach recommended by the validity was appropriate for all of the age ranges; the
WHO.50,51 In the United States, most instruments on sample size for the 0- to 6-month range was particularly
cognitive development have standards based on general small. Until further evidence is available on the validity
populations where no attempts are made to exclude of the scoring criteria in large, population-based samples
children with health conditions that pose risks to devel- and different cultures, the GMCD should be used to
opment. However, because children in developing coun- guide clinicians in monitoring and supporting child de-
tries have much higher rates of such health-related velopment and should not be used as a cross-sectional
problems that increase the likelihood of developmental screening test. Furthermore, in study 3, the validity of
problems, the WHO recommends using prescriptive the GMCD was examined using a cutoff point for the
samples to construct standards. This approach was ap- Bayley II of 2 SD and a 90% cutoff point for the
plied by the WHO in the construction of the newly milestones. In future studies, so as to avoid missing
launched WHO International Growth Standards and the children with mild delays, it will be important to deter-
WHO Motor Development Study.50,51 These studies have mine which cutoff points for the milestones are accurate
shown that, when child health is homogeneous and in comparison with a 1.5-SD cutoff point for the gold
optimal, child growth and motor development are sim- standard assessment.
ilar across diverse countries. The prescriptive sample The GMCD training program developed by the au-
approach enables LAMI countries to have standards for thors consists of written materials, slides, and demon-
child development that are independent of major stration videos and has been adopted by the Turkish
health-related risk factors for child development, more Ministry of Health and UNICEF-Turkey to be used in a
comparable between countries, and similar to those of nationwide training program on child development for
Western children. This approach is now being applied in primary health care providers. The training involves 1
developing countries as exemplified by the population- day for each of the 3 GMCD components. The training
based standardization study for developmental mile- includes interpreting the result of the GMCD together
stones in Argentina.62 As suggested by our study and that with all of the other existing clinical information, giving
from Argentina, when healthy subjects are recruited, the feedback starting with the childs specific strengths and
sample may be skewed toward children whose caregiv- using the following components to develop a plan with
ers have higher educational levels than national aver- the caregiver to support the childs development and to
ages. Population-based studies are needed to examine manage developmental difficulties if they are detected.
the diagnostic accuracy of instruments, such as the In future research it will be important to: examine psy-
GMCD, that have used healthy samples to construct chometric properties in population-based samples in di-
standard references. Particular attention should be di- verse countries and to examine the efficacy and effec-
rected to whether this approach leads to high false- tiveness of the GMCD training program in the early
positive rates or whether it enables the identification of detection and management of developmental difficulties
children who have delayed development but would be within health care systems.
considered to be developing normally if population- This study also raises an important question related to
based standard references were used. international research on measures for monitoring child
The findings of the reliability study (study 2) were development in LAMI countries: if, as suggested by the
limited to medical students in a research setting. This WHO, healthy prescriptive samples are used to de-
study has provided evidence that medical students who velop standard references for instruments such as the
have minimal clinical experience can be rapidly trained GMCD, are the ages of attainment of key developmental
to administer and score the GMCD. Future studies must milestones of healthy children similar across countries?
address whether the reliable administration of the Comparisons of our data with other studies62,63 suggest
GMCD can be sustained in real-life clinical practice. In that, during the early ages, children from different pop-
this study, the sociodemographic characteristics of the ulations may attain developmental milestones at similar
sample were similar to national census data: 64% of the ages. If milestones with similar ages of attainment across
caregivers had 5 years of education, and 12% were populations can be included in instruments for develop-
illiterate. Those caregivers with low education could mental monitoring, such as the GMCD, this may have
communicate their childs development in the GMCD important implications for eliminating the need to stan-
process and provide information for the GMCD. was dardize and validate instruments in each country.
lower but still within the excellent range for caregivers
with primary school education or less.52 Future popula- CONCLUSIONS
tion-based studies in real-life settings are needed to de- As an effective strategy to enhance child development,
termine the reliability and validity of the GMCD for many developed countries have redefined health care
caregivers with different levels of education. and have incorporated efforts to monitor and support
The study on validity (study 3) did not use a popula- the development of young children. As childhood mor-
tion-based sample, and, therefore, the information on tality continues to fall in LAMI countries, there is an
positive predictive accuracy and negative predictive ac- emerging interest in introducing models to optimize

PEDIATRICS Volume 121, Number 3, March 2008 e587

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
child development and to reduce the burden of devel- mal child development from birth to age 3 years: review of the
opmental difficulties beginning in early childhood. The literature. Arch Pediatr Adolesc Med. 2001;155(12):13111322
GMCD developed specifically to address some of the 11. Davis H, Tsiantis J. Promoting childrens mental health: the
challenges in LAMI countries offers a practical new European Early Promotion Project (EEPP). Int J Mental Health
Promot. 2005;7:4 16
method for developmental monitoring and the early
12. Dworkin PH. British and American recommendations for de-
detection of developmental difficulties within health sys- velopmental monitoring: the role of surveillance. Pediatrics.
tems in such countries. 1989;84(6):1000 1010
13. Engle PL, Black MM, Behrman JR, et al. International Child
Development Steering Group. Strategies to avoid the loss of
ACKNOWLEDGMENTS developmental potential in more than 200 million children in
This study was supported by a grant from the National the developing world. Lancet. 2007;369(9557):229 242
Institutes of Health and Fogarty International Center 14. Walker SP, Wachs TD, Gardner JM, et al. International Child
titled Brain Disorders Across the Lifespan (NIH R21 Development Steering Group. Child development: risk factors
TW006678-01; CFDA No. 93.9899 Promoting Child De- for adverse outcomes in developing countries. Lancet. 2007;
velopment: Yale-Ankara Collaboration). 369(9556):145157
We deeply appreciate the contribution of Drs John M. 15. Ertem IO, Atay G, Bingoler BE, Dogan DG, Bayhan A, Sarica D.
Promoting child development at sick-child visits: a controlled
Leventhal and Brian Forsyth for mentoring on the stud-
trial. Pediatrics. 2006;118(1). Available at: www.pediatrics.org/
ies and preparation of the article; Drs Frances Page Glas- cgi/content/full/118/1/e124
coe, Sirel Karakas, and Sedat Isikli for suggestions on the 16. Mitchell R. Community-based rehabilitation: the generalized
data analysis; Drs Meena Cabral, Mercedes de Onis, and model. Disabil Rehabil. 1999;21(10 11):522528
Jose Martines of the World Health Organization for sug- 17. Walker SP, Chang SM, Powell CA, Grantham-McGregor SM.
gestions and support; William Storandt for editing; and Effects of early childhood psychosocial stimulation and nutri-
Nermin Sezer for data entry. tional supplementation on cognition and education in growth-
stunted Jamaican children: prospective cohort study. Lancet.
2005;366(9499):1804 1807
REFERENCES 18. Mungala-Odera V, Newton CR. Identifying children with neu-
1. The World Bank. Data country classification. Available at: http:// rological impairment and disability in resource-poor countries.
web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/ Child Care Health Dev. 2007;33(3):249 256
0,contentMDK:20421402pagePK:64133150piPK:64133175 19. Sand N, Silverstein M, Glascoe FP, Gupta VB, Tonniges TP,
theSitePK:239419,00.html. AccessedJuly 21, 2007 OConnor KG. Pediatricians reported practices regarding de-
2. Grantham-McGregor S, Cheung YB, Cueto S, et al. Develop- velopmental screening: do guidelines work? Do they help?
mental potential in the first 5 years for children in developing Pediatrics. 2005;116(6):174 179
countries. Lancet. 2007;369(9555):60 70 20. Bethell C, Reuland CH, Halfon N, Schor EL. Measuring the
3. Durkin MS, Schneider H, Pathania VS, et al. Learning and quality of preventive and developmental services for young
developmental disabilities. In: Jamison DT, Breman JG, children: national estimates and patterns of clinicians perfor-
Measham AR, et al, eds. Disease Control Priorities in Developing mance. Pediatrics. 2004;113(suppl 6):S1973S1983
Countries. 2nd ed. New York, NY: Oxford University Press; 21. Meisels SJ, Fenichel ES, eds. New Visions for the Developmental
2006:933951 Assessment of Infants and Young Children. Washington, DC: Zero
4. Committee on Nervous System Disorders in Developing Coun- to Three/National Center for Infants, Toddlers, and Families
tries, Board on Global Health. Neurological, Psychiatric, and De- Publications; 1996
velopmental Disorders: Meeting the Challenge in the Developing 22. Gilliam WS, Meisels SJ, Mayes LC. Screening and surveillance
World. Washington, DC: Institute of Medicine, National Acad- in early intervention systems. In: Guralnick MJ, ed. The Devel-
emy Press; 2001 opmental Systems Approach to Early Intervention. Baltimore, MD:
5. Maulik PK, Darmstadt GL. Childhood disability in low- and Paul H. Brookes; 2005:7398
middle-income countries: overview of screening, prevention, 23. Glascoe FP. Screening for developmental and behavioral prob-
services, legislation, and epidemiology. Pediatrics. 2007; lems. Ment Retard Dev Disabil Res Rev. 2005;11(3):173179
120(suppl 1):S1S55 24. Msall ME. Measuring functional skills in preschool children at
6. Msall ME, Hogan DP. Counting children with disability in risk for neurodevelopmental disabilities. Ment Retard Dev Disabil
low-income countries: enhancing prevention, promoting child Res Rev. 2005;11(3):263273
development, and investing in economic well-being. Pediatrics. 25. Ertem IO, Atay G, Dogan DG, et al. Mothers knowledge of
2007;120(1):182185 young child development in a developing country. Child Care
7. Council on Children With Disabilities, Section on Developmen- Health Dev. 2007;33:728 737
tal Behavioral Pediatrics, Bright Futures Steering Committee, 26. Li Y, Liu J, Liu F, Guo G, Anme T, Ushijima H. Maternal
and Medical Home Initiatives for Children With Special Needs child-rearing behaviors and correlates in rural minority areas
Project Advisory Committee. Identifying infants and young of Yunnan, China. J Dev Behav Pediatr. 2000;21(2):114 122
children with developmental disorders in the medical home: an 27. de Lourdes Drachler M, de Castro Aerts DG, de Souza RM, de
algorithm for developmental surveillance and screening. Pedi- Carvalho Leite JC, Giugliani EJ, Marshall T. Social inequalities
atrics. 2006;118(1):405 420 in maternal opinion of child development in southern Brazil.
8. Kuo AA, Inkelas M, Lotstein DS, Samson KM, Schor EL, Halfon Acta Paediatr. 2005;94(8):1006 1008
N. Rethinking well-child care in the United States: an interna- 28. Kalra V, Seth R, Sapra S. Autism-experiences in a tertiary care
tional comparison. Pediatrics. 2006;118(4):16921702 hospital. Indian J Pediatr. 2005;72(3):227230
9. Blair M, Hall D. From health surveillance to health promotion: 29. Lian WB, Ho SK, Yeo CL, Ho LY. General practitioners knowl-
the changing focus in preventive childrens services. Arch Dis edge on childhood developmental and behavioural disorders.
Child. 2006;91(9):730 735 Singapore Med J. 2003;44(8):397 403
10. Regalado M, Halfon N. Primary care services promoting opti- 30. Figueiras AC, Puccini RF, da Silva EM, Pedromonico MR. Eval-

e588 ERTEM et al
Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
 uation of practices and knowledge among primary health care 47. Frankenburg WK, Dodds J, Archer P, Shapiro H, Bresnick B.
professionals in relation to child development surveillance [in The Denver II: a major revision and restandardization of the
Portuguese]. Cad Saude Publica. 2003;19(6):16911699 Denver Developmental Screening Test. Pediatrics. 1992;89(1):
31. Theeranate K, Chuengchitraks S. Parents Evaluation of Devel- 9197
opmental Status (PEDS) detects developmental problems com- 48. Glascoe FP. Technical Manual for the Brigance Screening Tests.
pared to Denver II. J Med Assoc Thai. 2005;88(suppl 3): North Billerica, MA: Curriculum Associates, Inc; 1996
S188 S192. 49. Bayley, N. Bayley Scales of Infant Development. 2nd ed. San An-
32. Glascoe FP. Collaborating With Parents: Using Parents Evaluation tonio, TX: Psychological Corporation; 1993
of Developmental Status (PEDS) to Detect and Address Developmental 50. de Onis M, Onyango AW, Borghi E, Garza C, Yang H. World
and Behavioral Problems. Nashville, TN: Ellsworth & Vander- Health Organization Multicentre Growth Reference Study
meer Press LLC; 2002 Group. Comparison of the World Health Organization (WHO)
33. Sparrow SS, Balla DA, Cicchetti DV. Vineland Adaptive Behavior Child Growth Standards and the National Center for Health
Scales. Circle Pines, MN: American Guidance Service; 1984 Statistics/WHO international growth reference: implications
34. Bricker D, Squires J, Mounts L, Potter L, Nickel R, Farrell J. for child health programmes. Public Health Nutr. 2006;9(7):
Ages & Stages Questionnaires (ASQ):A Parent-Completed, Child- 942947
Monitoring System. 2nd ed. Baltimore, MD: Paul H. Brookes; 51. Wijnhoven TM, de Onis M, Onyango AW, et al. Assessment of
1995 gross motor development in the WHO Multicentre Growth
35. Ireton H, Glascoe FP. Assessing childrens development using Reference Study. Food Nutr Bull. 2004;25(suppl 1):S37S45
parents reports. The Child Development Inventory. Clin Pedi- 52. Cicchetti DV. Guidelines, criteria, and rules of thumb for eval-
atr (Phila). 1995;34:248 255 uating normed and standardized assessment instruments in
36. Ireton H. The child development review: monitoring childrens psychology. Psychol Assess. 1994;6:284 290
development using parents and pediatricians observations. 53. Cicchetti DV. The precision of reliability and validity estimates
Infants Young Child. 1996;9:4252 re-visited: distinguishing between clinical and statistical signi-
37. World Health Organization. Integrated Management of Childhood fication of sample size requirements. J Clin Exp Neuropsychol.
Illness. Counsel the Mother on Feeding, Care for Development, When 2001;23(5):695700
to Return and the Mothers Health. Geneva, Switzerland: World 54. SPSS for Windows [computer program]. Version 11.0. 1. Chi-
Health Organization, Department of Child and Adolescent cago, IL: SPSS, Inc; 2001
Health; 2001 55. Thorburn M, Desai P, Paul TJ, Malcolm L, Durkin M, Davidson
38. World Health Organization. Training Course on Child Growth L. Identification of childhood disability in Jamaica: the ten
Assessment. Version 1, November 2006. Geneva, Switzerland: questions screen. Int J Rehabil Res. 1992;15(2):115127
World Health Organization; 2006:60 66 56. Durkin MS, Hasan ZM, Hasan KZ. The ten questions screen for
39. Bronfenbrenner U, Ceci SJ. Nature-nurture reconceptualized childhood disabilities: its uses and limitations in Pakistan. J
in developmental perspective: a bioecological model. Psychol Epidemiol Community Health. 1995;49(4):431 436
Rev. 1994;101(4):568 586 57. Mungala-Odera V, Meehan R, Njuguna P, et al. Validity and
40. Sameroff AJ, Fiese BH. Transactional regulation: the develop- reliability of the Ten Questions questionnaire for detecting
mental ecology of early intervention. In: Shonkoff JP, Meisels moderate to severe neurological impairment in children aged
SJ, eds. Handbook of Early Childhood Intervention. New York, NY: 6 9 years in rural Kenya. Neuroepidemiology. 2004;23(12):
Cambridge University Press; 2000:135160 6772
41. Committee on Hospital Care. American Academy of Pediatrics. 58. Wirz S, Edwards K, Flower J, Yousafzai A. Field testing of the
Family-centered care and the pediatricians role. Pediatrics. ACCESS materials: a portfolio of materials to assist health
2003;112(3 Pt 1):691 697 workers to identify children with disabilities and offer simple
42. Green M, Palfrey J, eds. Bright Futures: Guidelines for the Health advice to mothers. Int J Rehabil Res. 2005;28(4):293302
Supervision of Infants, Children, Adolescents. 2nd ed. Arlington, 59. Chopra G, Verma IC, Seetharaman P. Development and assess-
VA: National Center for Education in Maternal and Child ment of a screening test for detecting childhood disabilities.
Health; 2000 Indian J Pediatr. 1999;66(3):331335
43. International Classification of Functioning, Disability and 60. Glascoe FP, Byrne KE, Ashford LG, Johnson KL, Chang B,
Health, World Health Organization. International Classification of Strickland B. Accuracy of the Denver-II in developmental
Impairments, Disabilities, and Handicaps. Geneva, Switzerland: screening. Pediatrics. 1992;89(6 pt 2):12211225
World Health Organization; 1980 61. United Nations Childrens Fund. State of the Worlds Children.
44. Fisher RP, McCauley ML. Information retrieval: interviewing New York, NY: UNICEF Publications; 2007
witnesses. In: Brewer N, Wilson C, eds. Psychology and Policing. 62. Lejarraga H, Pascucci MC, Krupitzky S, et al. Psychomotor
Hillsdale, NJ: Erlbaum; 1995:8199 development in Argentinean children aged 0 5 years. Paediatr
45. Boyle D, Dwinnell B, Platt F. Invite, listen, and summarize: a Perinat Epidemiol. 2002;16(1):47 60
patient-centered communication technique. Acad Med. 2005; 63. Lansdown RG, Goldstein H, Shah PM, et al. Culturally appro-
80(1):29 32 priate measures for monitoring child development at family
46. Teutsch C. Patient-doctor communication. Med Clin North Am. and community level: a WHO collaborative study. Bull World
2003;87(5):11151145 Health Organ. 1996;74(3):283290

PEDIATRICS Volume 121, Number 3, March 2008 e589

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014
 A Guide for Monitoring Child Development in Low- and Middle-Income
Countries
Ilgi O. Ertem, Derya G. Dogan, Canan G. Gok, Sevim U. Kizilates, Ayliz Caliskan,
Gulsum Atay, Nilgun Vatandas, Tugba Karaaslan, Sevgi G. Baskan and Domenic V.
Cicchetti
Pediatrics 2008;121;e581
DOI: 10.1542/peds.2007-1771
Updated Information & including high resolution figures, can be found at:
Services http://pediatrics.aappublications.org/content/121/3/e581.full.h
tml
References This article cites 44 articles, 12 of which can be accessed free
at:
http://pediatrics.aappublications.org/content/121/3/e581.full.h
tml#ref-list-1
Citations This article has been cited by 1 HighWire-hosted articles:
http://pediatrics.aappublications.org/content/121/3/e581.full.h
tml#related-urls
Subspecialty Collections This article, along with others on similar topics, appears in
the following collection(s):
Development/Behavioral Issues
http://pediatrics.aappublications.org/cgi/collection/developme
nt:behavioral_issues_sub
Permissions & Licensing Information about reproducing this article in parts (figures,
tables) or in its entirety can be found online at:
http://pediatrics.aappublications.org/site/misc/Permissions.xht
ml
Reprints Information about ordering reprints can be found online:
http://pediatrics.aappublications.org/site/misc/reprints.xhtml

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

publication, it has been published continuously since 1948. PEDIATRICS is owned, published,
and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk
Grove Village, Illinois, 60007. Copyright 2008 by the American Academy of Pediatrics. All
rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org by guest on June 5, 2014