Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Ijerph 19 09583

Download as pdf or txt
Download as pdf or txt
You are on page 1of 13

International Journal of

Environmental Research
and Public Health

Article
Transactions between Problem Behaviors and Academic
Performance in Early Childhood
Chin-Chih Chen 1, * , Sheng-Lun Cheng 2 , Yaoying Xu 1 , Kathleen Rudasill 3 , Reed Senter 1 , Fa Zhang 4 ,
Melissa Washington-Nortey 5 and Nikki Adams 6

1 Department of Counseling and Special Education, Virginia Commonwealth University,


Richmond, VA 23284, USA
2 Department of Library Science and Technology, Sam Houston State University, Huntsville, TX 77341, USA
3 School of Education, Virginia Commonwealth University, Richmond, VA 23284, USA
4 Department of Education, Open University of China, Beijing 100039, China
5 Department of Psychology, King’s College, London WC2R 2LS, UK
6 Ph.D. Program in Special Education, University of Illinois at Urbana-Champaign, Champaign, IL 61820, USA
* Correspondence: ccchen@vcu.edu

Abstract: This study aimed to further the understanding of transactional relationships that exist
between problem behaviors and academic performance in early childhood. Early academic and
behavior difficulties increase the risk of school disengagement, academic failure, and dropout.
Although children’s academic and behavioral difficulties have been shown to be intercorrelated, little
research has focused on how the relationship reciprocates and progresses in early childhood. This
study investigated how problem behaviors (i.e., externalizing and internalizing) influence and are
influenced by academic performance (i.e., poor reading and math) from kindergarten to third grade.
Participants included 18,135 students (51.22% boys) derived from a nationally representative sample
Citation: Chen, C.-C.; Cheng, S.-L.;
in the Early Childhood Longitudinal Study, Kindergarten Class of 2011 (ECLS-K: 2011). Teacher
Xu, Y.; Rudasill, K.; Senter, R.; Zhang,
F.; Washington-Nortey, M.; Adams, N.
ratings of children’s internalizing (low self-esteem, anxiety, loneliness, or sadness) and externalizing
Transactions between Problem (fighting, arguing, showing anger, impulsively acting, and disruptive behaviors) problem behaviors,
Behaviors and Academic as well as direct assessments of children’s academic performance (reading and math), were collected
Performance in Early Childhood. Int. yearly. Cross-lagged panel modeling (CLPM) was employed to examine reciprocal relationships
J. Environ. Res. Public Health 2022, 19, between problem behaviors and academic performance over time from kindergarten to third grade.
9583. https://doi.org/10.3390/ The results supported the transactional relationships in early childhood, with higher externalizing as
ijerph19159583 well as internalizing problem behaviors predicting lower academic performance and lower academic
Academic Editor: Paul B. performance predicting higher externalizing and internalizing problem behaviors. The implications
Tchounwou for research, prevention, and early intervention regarding the progression of academic and behavioral
problems are discussed.
Received: 2 June 2022
Accepted: 29 July 2022
Keywords: problem behaviors; academic performance; transactional relationship; early childhood
Published: 4 August 2022

Publisher’s Note: MDPI stays neutral


with regard to jurisdictional claims in
published maps and institutional affil- 1. Introduction
iations.
Early academic and behavioral struggles place children at a heightened risk for school
disengagement, academic failure, and dropout, and they can be expected to lead to chal-
lenges during adulthood [1–4]. Emerging research suggests that behavioral and academic
Copyright: © 2022 by the authors.
problems do not function unidirectionally but instead work reciprocally to predict later
Licensee MDPI, Basel, Switzerland. school maladjustment and failure [5]. However, findings on the reciprocal associations
This article is an open access article between problem behaviors and academic achievement for children are less conclusive. A
distributed under the terms and nuanced understanding of the complex mechanisms between behavioral and academic
conditions of the Creative Commons problems could strengthen our capacity to assess and identify students’ barriers to learning
Attribution (CC BY) license (https:// and better inform the development of evidence-based intervention efforts.
creativecommons.org/licenses/by/ Children’s problem behaviors and academic challenges likely exacerbate one another
4.0/). and place children at greater risk for maladaptive adjustment, with serious consequences

Int. J. Environ. Res. Public Health 2022, 19, 9583. https://doi.org/10.3390/ijerph19159583 https://www.mdpi.com/journal/ijerph
Int. J. Environ. Res. Public Health 2022, 19, 9583 2 of 13

for society [6,7]. Research has demonstrated a negative association between problem
behaviors and academic problems [8–10]. Academic difficulties are predictive of behavior
difficulties, including internalization (e.g., depression) and externalization (e.g., disruptive
behavior). Similarly, students with internalizing and externalizing problem behaviors
are at increased risk for lower academic performance (e.g., [11–13]). While some studies
have clearly established unidirectional relationships between academic and behavioral
problems, other research has postulated transactional relationships, which suggest that
behavior and academic difficulties work reciprocally to predict later school maladjustment
and failure [5,11].
However, there is a paucity of research examining the developmental progression
of behavioral and academic problems and how these problems potentially reciprocate
during early childhood, the period of a child’s most rapid growth/development. It is
important to clarify the developmental processes underlying behavior (e.g., externalizing
and internalizing) problems and academic performance for those facing early adversity
and identify potential points of intervention to mitigate maladaptive adjustment over time.
The proposed study investigates how internalizing and externalizing problem behaviors
influence and are influenced by academic performance and vice versa from kindergarten to
third grade in the hope of elucidating these transactional relationships in early childhood.

1.1. Transactions between Behavioral and Academic Problems


According to developmental and social–ecological theory, variations in individual
functioning can trigger a sequence of events [14,15]. According to dynamic systems theory,
human development involves a cascade of changes influenced by transactions within
individuals or between individuals and their social contexts [16]. Poor behavior could
lead to lower levels of academic functioning, and, in turn, academic failure could lead
to vulnerability to future behavior issues. The link between the behavior and academic
domains suggests the possibility of developmental cascades [5,11].
Several models have been proposed to explain the relationships between problem
behaviors and academic performance, including the adjustment erosion model, academic
incompetence model, bidirectional model, and shared risk model [11]. The adjustment
erosion model posits that the behavioral construct at an earlier time point has a significant
negative correlation with academic success at a later time point, while the academic in-
competence model postulates that poor academic competence and performance predict
later externalizing and internalizing problems. To demonstrate bidirectional associations
between behavior and academic achievement, the behavioral construct must predict aca-
demic achievement at a later time point and vice versa. The shared risk model postulates
an alternative hypothesis, which states that behavioral problems and academic struggles
share a common predictive factor.

1.2. Externalizing Problem Behaviors and Academic Achievement


Considerable stability of problem behaviors has been identified beginning in early
childhood. Externalizing problems, such as aggressive/disruptive behavior, can interfere
with students’ classroom learning and affect classroom dynamics. Such linkage is evident
even before school begins [17]. Recent longitudinal studies have aimed to identify trans-
actions between academic achievement and externalizing problems and have revealed
underlying processes across multiple developmental periods. With the use of cross-lagged
modeling, these longitudinal studies have tested such transactional relationships by con-
trolling for interindividual stability over time and within-time covariation of externalizing
problems and academic performance. However, the findings on transactional effects are
inconclusive.
Few studies have shown bidirectional relationships that suggest that externalizing
behaviors have a negative effect on academic performance and vice versa from late child-
hood to adolescence. Moilanen and colleagues [11] studied cross-lagged effects of at-risk
U.S. boys and found that higher externalizing problems in sixth and eighth grade were
Int. J. Environ. Res. Public Health 2022, 19, 9583 3 of 13

indicative of lower academic competence in eighth and tenth grade. Lower academic com-
petence in 10th and 11th grade was predicative of higher externalizing problems in 11th
and 12th grade. Zhang and colleagues [13] identified bidirectional associations between
externalizing behaviors and academic performance (final exam scores) across fifth to ninth
grade for Chinese children and noted that the predictive strength from academic failure to
externalizing behaviors increased, whereas the magnitude of the adjustment erosion path-
way decreased during the transition into adolescence. Zimmermann and colleagues [18]
identified similar bidirectional relationships between externalizing problems and academic
achievement (grades and standardized assessments) of German children from fifth to ninth
grade. Externalizing problems had a more notable impact on grades than they had on
standard achievement. Similarly, Okano et al. [19] found bidirectional effects between
externalizing behaviors and school achievement from fifth to ninth grade for U.S. children.
A longitudinal study of Canadian children found negative cross-lagged effects between
externalizing behaviors and academic competence from age 8 to 9 to age 14 to 15 [20].
Studies also demonstrated bidirectional relationships from early childhood to mid-
childhood. Spanning kindergarten to fourth grade, Metsäpelto and colleagues [21] identi-
fied bidirectional associations suggesting that high externalizing problems in earlier grades
were linked to low academic performance in later grades and vice versa for Finnish children.
Van der Ende and colleagues [22] examined both internalizing and externalizing behaviors
and academic difficulties in a multiple-cohort longitudinal study spanning eight years of
children in the Netherlands aged six and ten. The results indicated that externalizing behav-
iors predicted academic difficulties, and academic difficulties predicted both externalizing
and internalizing problem behaviors.
Different from prior studies that have reported bidirectional pathways, many studies
have only identified unidirectional relationships between externalizing problem behav-
iors and academic achievement. For example, several studies have strongly supported
the adjustment erosion model, suggesting that externalizing behaviors predict academic
difficulties from childhood through adolescence and into adulthood [5,23–27]. By contrast,
Vaillancourt et al.’s [28] findings supported the academic incompetence model, suggest-
ing that aggression predicted academic competence/performance during early/middle
childhood (standardized assessment and GPA).

1.3. Internalizing Problem Behaviors and Academic Achievement


Findings on the transactions between internalizing symptoms and academic achieve-
ment are also mixed in the following studies, which used cross-lagged modeling. Some
studies identified bidirectional linkages between depression, anxiety, and general internal-
izing symptoms and academic achievement over time. Keles and colleagues [29] identified
reciprocal relations between general internalizing symptoms and academic ratings in a
longitudinal cohort of Norwegian students from fourth to seventh grade. Verboom and
colleagues [30] examined the associations between depressive symptoms and academic per-
formance in a longitudinal study of Dutch adolescents between the ages of 10 and 18 years.
However, they only found bidirectional associations in girls. Zhang and colleagues [13]
found bidirectional associations between depression and academic achievement among
Chinese students from grades five to nine, but the results were only observed in the nar-
row transition into middle school. Weidman and colleagues [31] also found bidirectional
relationships between depression and anxiety and academic achievement (GPA) among
U.S. children between sixth and tenth grade. Okano and colleagues [12] found significant
bidirectional pathways between internalizing behaviors and academic grades among girls
from third to ninth grade.
Most other longitudinal studies have identified unidirectional pathways between
internalizing behaviors and academic achievement. Some studies have only supported
adjustment erosion pathways, suggesting that internalizing behaviors predict academic
achievement. For example, Hall and colleagues [32] found social withdrawal in kinder-
garten and first grade predicted reading achievement in second grade. There are many
Int. J. Environ. Res. Public Health 2022, 19, 9583 4 of 13

studies that only support the academic incompetence model and strongly suggest that
academic achievement predicts later internalizing behaviors [5,11,22,24,26,33]; such associ-
ations were found in later years through adolescence [11,24,26], and into adulthood [22].
The pathway appears stronger in girls also [26].

1.4. Current Study


Early childhood is a time full of developmental opportunities and vulnerability. While
the transactions between problem behaviors and academic performance have been docu-
mented [5,11,34], there is a paucity of longitudinal research examining the transactional
associations in early childhood from a developmental perspective. Mixed findings exist in
the current literature. Some studies have shown unidirectional links and others have estab-
lished bidirectional links between externalizing and internalizing problem behaviors and
academic performance across different developmental stages [5,11,18,22,25,30]. However,
little research has specifically addressed the transactions between behavior problems and
academic performance in early childhood.
For interventions to be successful in preventing the progression of academic and be-
havior problems, it is necessary to understand transactional relationships between behavior
and academic issues over time [5,8,9,18,22]. This study investigated how internalizing
and externalizing problems predict and are predicted by academic difficulties (i.e., poor
academic achievement and classroom problem behavior) over time. We addressed the
following research questions to unpack the developmental mechanism for students in early
childhood: (1) Do externalizing problem behaviors predict children’s academic perfor-
mance in K–3 over time and vice versa? (2) Do internalizing problem behaviors predict
children’s academic performance in K–3 over time and vice versa? We hypothesized
that there are transactional relationships between externalizing and internalizing problem
behavior and academic performance.

2. Method
2.1. Participants and Research Design
This study consisted of 18,135 children derived from a nationally representative sample
in the United States participating in the Early Childhood Longitudinal Study, Kindergarten
Class of 2011 (ECLS-K: 2011) [35]. About 51.22% of the children were male (n = 9288) and
48.78% were female (n = 8847). Most of the children were White (46.81%; n = 8488), and
most had two biological/adoptive parents (69.03%; n = 9249) with a high school diploma
(22.14%; n = 3543) or some college education (26.50%; n = 4242). More than half of the
children (52.27%; n = 7070) lived in a household at or above 200 percent of the poverty
threshold. The average age of children was 73.44 months (SD = 4.47) (note: percentages
and numbers listed above were unweighted).
On the basis of a multistage probability sample design, the ECLS-K: 2011 drew a
nationally representative sample of U.S. children enrolled in kindergarten during the
2010–2011 school year and longitudinally followed the same sample through fifth grade
(201–2016) [35]. The multistage probability sample design involved a three-stage process,
starting with dividing the U.S. and sampling primary sample units (PSUs), then selecting
both public and private schools within each PSU, and finally sampling a fixed number
of kindergarten children within each sampled school. The complete dataset included
970 schools within 90 PSUs [35]. It should be noted that because of the design of the ECLS-
K: 2011, teachers were included in the study by virtue of their connection to the sampled
children. Therefore, there were no specific procedures implemented to recruit teachers.
Employing multiple sources and methods, the ECLS-K: 2011 provides researchers and
policymakers with information on children’s early education experiences, development,
and learning over time collected from parent interviews, teacher ratings/surveys, school
records, and direct child assessments. In the current study, we used publicly available data
collected each spring semester during kindergarten and the first-, second-, and third-grade
years to analyze the longitudinal relationships between children’s problem behaviors and
Int. J. Environ. Res. Public Health 2022, 19, 9583 5 of 13

academic performance. We used direct child assessments of reading and mathematics and
teacher questionnaires of children’s internalizing and externalizing problem behaviors.

2.2. Measures
Internalizing and Externalizing Problem Behaviors. Teachers were asked to rate how
frequently children exhibited problem behaviors using two subscales from the modified
version of the Social Skills Rating System (SSRS) [36] in the ECLS-K: 2011 study. Children’s
behaviors were rated on a 4-point Likert scale, ranging from never to very often. Teachers
rated 4 items related to the child’s internalizing behaviors (low self-esteem, anxiety, lone-
liness, and sadness) and 5 items for externalizing behaviors (fighting, arguing, showing
anger, impulsively acting, and disrupting class activities). Average scores were created for
each of the scales of internalizing and externalizing problem behaviors. The higher the
scores, the more likely the child was to exhibit internalizing and externalizing problem
behaviors. The Cronbach’s alpha coefficients for internalizing and externalizing problem
behaviors were 0.89 and 0.78.
Academic Performance. Academic performance assessments that focused on chil-
dren’s reading and mathematics skills were administered using a two-stage process. The
ECLS-K used Item Response Theory (IRT) scale scores for the children’s assessments in
reading and mathematics [35]. IRT scores were determined by the number of correct an-
swers and the probability of the child answering the questions that were not administered
correctly. Using the IRT, children’s scores can be compared within and across grades. The
reading assessment measured children’s basic language skills, vocabulary knowledge, and
reading comprehension. In the two-stage process, reading was first assessed with the mea-
sure of English basic reading skills (EBRS), which includes 18 reading items and 2 screener
items from the Preschool Language Assessment Scale [37]. The scores on the EBRS deter-
mined the difficulty of the questions (low, middle, or high) that the child would receive in
the second stage. Children who spoke Spanish and did not meet the criteria for the EBRS
were evaluated by the measure of Spanish early reading skills (SERS), which encompassed
31 items translated from the low and middle second-stage English reading test.
The mathematics assessment measured children’s conceptual knowledge, procedural
knowledge, and problem-solving. Items consisted of questions on number sense, properties,
and operations; measurement; geometry and spatial sense; data analysis, statistics, and
probability; and patterns, algebra, and functions. The first stage of the assessment included
18 items that were used to determine the difficulty of the questions (low, middle, or high)
in the second-stage assessment [35]. Children whose home language was Spanish and
who did not pass the preLAS were administered the entire mathematics assessment in
Spanish. According to the user’s manual, the Cronbach’s alpha coefficients for reading and
mathematics assessments were 0.95 and 0.94.
Demographics. We controlled for gender and ethnicity as covariates in this study.
Gender was binary, with males as the reference group. Races/ethnicities were categorized
as non-Hispanic White, black/African American, Hispanic, Asian, and “other”, with the
non-Hispanic White group as the reference group.

2.3. Data Analysis


Prior to the main analyses, descriptive statistics and bivariate correlations were ex-
amined. Cross-lagged panel modeling (CLPM) was then employed to examine bidirec-
tional relationships between problem behaviors and academic achievement across four
timepoints—kindergarten to third grade—while controlling for gender and race/ethnicity.
One of the primary goals of CLPM is to identify causal predominance by assessing the
magnitude of the lagged effects while accounting for contemporaneous effects (i.e., autore-
gressive effects) and variance across time [38]. We built two cross-lagged panel models
(CLPM) to examine the reciprocal relationships between internalizing/externalizing prob-
lem behaviors and academic achievement. Gender and race/ethnicity were included in all
the CLPMs as covariates. In addition to the cross-lagged and autoregressive paths between
Int. J. Environ. Res. Public Health 2022, 19, 9583 6 of 13

adjacent time points, second-order autoregressive paths (AR2, see [39]) between the first
and third time point measurements and between the second and fourth time point measure-
ments were specified. These additional autoregressive pathways are reasonable for CLPMs
with at least three measurement points because influences observed in the early stage may
be carried over beyond the steady change between adjacent measurements [40,41]. The
overall goodness of model fit was assessed by global fit statistics, including the comparative
fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation
(RMSEA), and the standardized root mean square residual (SRMR). A satisfactory model
fit was defined by the following criteria: CFI ≥ 0.95, TLI ≥ 0.95, RMSEA ≤ 0.06, and
SRMR ≤ 0.08 [42].
Full information maximum likelihood (FIML) implemented in Mplus 8.4 [43] was
used to handle missing data in endogenous variables because it is recommended as
an ideal approach to address missingness in longitudinal panel models [44,45]. The
TYPE = COMPLEX option in Mplus 8.4 was used to account for the complex sampling de-
sign employed by the ECLS-K: 2011. To obtain correct point estimates and produce accurate
variance estimation, the child-level sampling weight (W7C27P_2T270) and 80 correspond-
ing replicate weights (W7C27P_2T271-W7C27P_2T2780) were entered into the analytic
models. The paired jackknife replication method (JK2) was selected, as recommended by
the user’s manual for the ECLS-K: 2011 [35].

3. Results
3.1. Descriptive Statistics and Correlations
Table 1 presents descriptive statistics for problem behaviors and academic perfor-
mance variables. The absolute values of skewness and kurtosis were all less than 1.5,
suggesting that the assumption of normality was met [46] (Kline, 2016). Table 2 presents
zero-order correlations. The directions of the coefficients aligned with the theory. Children’s
earlier internalizing problem behaviors were strongly related to later internalizing problem
behaviors (r ≥ 0.26, p < 0.001). Earlier externalizing problem behaviors were strongly
related to later externalizing problem behaviors (r ≥ 0.53, p < 0.001). Similarly, earlier
academic achievement was strongly related to later achievement (r ≥ 0.75). Internalizing
and externalizing problem behaviors had positive associations with one another, with rs
ranging from 0.08 to 0.33 (p < 0.001). Academic achievement had negative associations
with internalizing and externalizing problem behaviors, with rs ranging from 0.14 to 0.23
(p < 0.001).

Table 1. Descriptive statistics of problem behaviors and academic performance variables.

Variable M SD Skewness Kurtosis Min Max


Internalizing problem behaviors T1 1.50 0.49 1.36 2.49 1 4
Internalizing problem behaviors T2 1.54 0.50 1.41 2.78 1 4
Internalizing problem behaviors T3 1.59 0.53 1.28 2.10 1 4
Internalizing problem behaviors T4 1.60 0.54 1.29 1.97 1 4
Externalizing problem behaviors T1 1.62 0.62 1.21 1.40 1 4
Externalizing problem behaviors T2 1.72 0.61 1.10 1.06 1 4
Externalizing problem behaviors T3 1.72 0.62 1.07 0.81 1 4
Externalizing problem behaviors T4 1.69 0.62 1.14 1.06 1 4
Academic achievement T1 59.73 12.68 0.65 1.22 22.14 122.71
Academic achievement T2 83.83 15.29 −0.19 0.15 22.50 136.68
Academic achievement T3 101.13 15.92 −0.39 0.05 32.95 142.37
Academic achievement T4 112.31 15.27 −0.47 −0.22 44.22 152.13
Note. T1: kindergarten; T2: 1st grade; T3: 2nd grade; T4: 3rd grade.
Int. J. Environ. Res. Public Health 2022, 19, 9583 7 of 13

Table 2. Bivariate correlations.

1 2 3 4 5 6 7 8 9 10 11 12
1. Internalizing problem behaviors T1 —
2. Internalizing problem behaviors T2 0.30 —
3. Internalizing problem behaviors T3 0.26 0.30 —
4. Internalizing problem behaviors T4 0.26 0.29 0.35 —
5. Externalizing problem behaviors T1 0.27 0.18 0.19 0.19 —
6. Externalizing problem behaviors T2 0.11 0.30 0.20 0.20 0.60 —
7. Externalizing problem behaviors T3 0.11 0.14 0.33 0.19 0.56 0.60 —
8. Externalizing problem behaviors T4 0.08 0.14 0.17 0.31 0.53 0.58 0.61 —
9. Academic achievement T1 −0.17 −0.17 −0.20 −0.19 −0.18 −0.17 −0.16 −0.14 —
10. Academic achievement T2 −0.19 −0.21 −0.23 −0.21 −0.20 −0.21 −0.19 −0.17 0.85 —
11. Academic achievement T3 −0.19 −0.20 −0.23 −0.20 −0.19 −0.19 −0.19 −0.17 0.79 0.90 —
12. Academic achievement T4 −0.18 −0.20 −0.22 −0.20 −0.18 −0.19 −0.19 −0.18 0.75 0.85 0.91 —
Note. All the correlation coefficients are statistically significant at p < 0.001. Transactional relationships between
internalizing problem behaviors and academic performance.

The first CLPM examined the longitudinal relationships between internalizing problem
behaviors and academic achievement while controlling for gender and race/ethnicity. The
model fit the data well (χ2(18) = 679.739, p < 0.001; CFI = 0.980, TLI = 0.937, RMSEA = 0.046,
SRMR = 0.057). As shown in Figure 1, all the autoregressive paths were statistically
significant. The autoregressive coefficients of internalizing problem behaviors between
adjacent time points were stable over time (β = 0.273, K to first grade; β = 0.212, first to
second grade; β = 0.262, second to third grade; p < 0.001). The autoregressive coefficients
of academic achievement between adjacent time points were particularly large, which
showed substantial stability over time (β = 0.833, K to first grade; β = 0.785, first to
second grade; β = 0.733, second to third grade; p < 0.001). In addition, all the second-
order autoregressive (AR2) pathways were also statistically significant. A consistent AR2
Int. J. Environ. Res. Public Health 2022,emerged
19, x for internalizing problem behaviors (β = 0.161, K to second grade; β = 0.185, first
8 of 13
to third grade; p < 0.001) and academic achievement (β = 0.119, K to second grade; β = 0.179,
first to third grade; p < 0.001).

Figure 1.
Figure 1. Transactions
Transactions between
between internalizing
internalizing problem
problem behaviors
behaviors and
andacademic
academicachievement.
achievement.

3.2. Transactional
In terms of theRelationships between
cross-lagged Externalizing
relationships, Problem Behaviors
internalizing problemand Academicand aca-
behaviors
Performance
demic achievement consistently predicted one another over time, even after controlling for
autoregressive paths, AR2, gender, and race/ethnicity. Internalizing problem behaviors
The second CLPM examined the transactional relationships between externalizing
negatively predicted academic achievement in reading and math from kindergarten to first
problem behaviors and academic achievement longitudinally while controlling for gender
grade (β = −0.052, p < 0.001), from first grade to second grade (β = −0.014, p < 0.001), and
and race/ethnicity. The model also fit the data well (𝜒( ) = 1106.618, p < 0.001; CFI = 0.973,
from second grade to third grade (β = −0.013, p < 0.001). The magnitude of the transactional
TLI = 0.913,
effects RMSEA = 0.059,
of internalizing SRMR
problem = 0.068).
behaviors onAs shown achievement
academic in Figure 2, all the autoregressive
appeared significant
but slightly weakened over time. Academic achievement also negatively predicted prob-
paths were statistically significant. The autoregressive coefficients of externalizing inter-
lem behaviors between adjacent time points were stable over time (β = 0.567, K to first
grade; β = 0.399, first to second grade; β = 0.393, second to third grade; p < 0.001). The
autoregressive coefficients of academic achievement between adjacent time points were,
as with the internalizing model, particularly large (β = 0.833, K to first grade; β = 0.786,
first to second grade; β = 0.732, second to third grade; p < 0.001). In addition, all the second-
Int. J. Environ. Res. Public Health 2022, 19, 9583 8 of 13

nalizing problem behaviors from kindergarten to first grade (β = −0.134, p < 0.001), from
first grade to second grade (β = −0.170, p < 0.001), and from second grade to third grade
(β = −0.113, p < 0.001), with the largest path from first grade to second grade.

3.2. Transactional Relationships between Externalizing Problem Behaviors and Academic


Performance
The second CLPM examined the transactional relationships between externalizing
problem behaviors and academic achievement longitudinally while controlling for gender
and race/ethnicity. The model also fit the data well (χ2(18) = 1106.618, p < 0.001; CFI = 0.973,
TLI = 0.913, RMSEA = 0.059, SRMR = 0.068). As shown in Figure 2, all the autoregressive
paths were statistically significant. The autoregressive coefficients of externalizing problem
behaviors between adjacent time points were stable over time (β = 0.567, K to first grade;
β = 0.399, first to second grade; β = 0.393, second to third grade; p < 0.001). The autore-
gressive coefficients of academic achievement between adjacent time points were, as with
the internalizing model, particularly large (β = 0.833, K to first grade; β = 0.786, first to
second grade; β = 0.732, second to third grade; p < 0.001). In addition, all the second-order
autoregressive pathways were also statistically significant. A consistent AR2 emerged for
externalizing problem behavior (β = 0.294, K to second grade; β = 0.309, first to third grade;
Int. J. Environ. Res. Public Health 2022, 19, x 9 of 13
p < 0.001) and academic achievement (β = 0.118, K to second grade; β = 0.181, first to third
grade; p < 0.001).

Figure 2.
Figure 2. Transactions
Transactions between
between externalizing
externalizingproblem
problembehaviors
behaviorsand
andacademic
academicachievement
achievement(dotted
(dot-
ted lines
lines indicate
indicate nonsignificant
nonsignificant paths).
paths).

4. Discussion
In terms of the cross-lagged relationships, there were transactional effects between
externalizing problem behaviors and academic performance over time. Externalizing
Children’s academic and behavioral problems have been shown to be intercorrelated
problem behaviors significantly predicted academic achievement. Higher externalizing
and reciprocally affected. This study is one of the first to investigate the transactional re-
problem behaviors predicted lower academic achievement from kindergarten to first grade
lationships between externalizing and internalizing problem behaviors and academic
(β = −0.053, p < 0.001) and second to third grade (β = −0.015, p < 0.001) but not first
achievement in early childhood using a nationally representative sample. Using cross-
to second grade. Academic achievement predicted externalizing problem behaviors at
lagged modeling and controlling for within-individual stabilities over time and covari-
all time points. Specifically, greater academic achievement consistently predicted lower
ance, we identified K to third grade as the time when developmental cascading occurs in
externalizing problem behaviors (β = −0.070, K to first grade; β = −0.045, first to second
the transactional
grade; β = −0.028,process. Our
second to discussion
third grade; p focuses
< 0.001).on the understanding of the develop-
mental mechanisms in terms of timing and transactional process and on leveraging this
information
4. Discussionin early intervention efforts to facilitate positive development.
Children’s academic and behavioral problems have been shown to be intercorrelated
4.1. Transactions
and reciprocallyinaffected.
Early Childhood
This study is one of the first to investigate the transactional
Prior research
relationships hasexternalizing
between suggested that externalizing
and andproblem
internalizing internalizing problem
behaviors and behaviors
academic
and poor academic performance function reciprocally and contribute to later maladjust-
ment and school failure (e.g., [11,13]). This study further identified bidirectional relation-
ships between externalizing and internalizing problem behaviors and academic achieve-
ment in early childhood; these relationships were identified earlier than those observed
in most prior longitudinal studies, which have identified bidirectional relationships be-
Int. J. Environ. Res. Public Health 2022, 19, 9583 9 of 13

achievement in early childhood using a nationally representative sample. Using cross-


lagged modeling and controlling for within-individual stabilities over time and covariance,
we identified K to third grade as the time when developmental cascading occurs in the
transactional process. Our discussion focuses on the understanding of the developmental
mechanisms in terms of timing and transactional process and on leveraging this information
in early intervention efforts to facilitate positive development.

4.1. Transactions in Early Childhood


Prior research has suggested that externalizing and internalizing problem behaviors
and poor academic performance function reciprocally and contribute to later maladjustment
and school failure (e.g., [11,13]). This study further identified bidirectional relationships
between externalizing and internalizing problem behaviors and academic achievement
in early childhood; these relationships were identified earlier than those observed in
most prior longitudinal studies, which have identified bidirectional relationships between
academic performance and externalizing problem behaviors (e.g., [11,13,18,19]) and inter-
nalizing problem behaviors (e.g., [13,29–31]) from middle childhood through adolescence.
Additionally, our findings are consistent with and build on findings from two kinder-
garten cohort studies [21,22], which showed transactional relationships between external-
izing problem behaviors and academic achievement from kindergarten to third grade, as
well as prior longitudinal studies, which also indicated bidirectional relationships between
internalizing problem behaviors (e.g., [13,29–31]) and academic performance from middle
childhood through adolescence. What is novel in the current study is the confirmation of
these bidirectional relationships with a nationally representative sample of children.
This study contributes to the existing literature on transactional relationships between
problem behaviors and academic performance by identifying such relationships in early
childhood. The findings highlight the importance of considering nuanced information
in intervention efforts to manage students’ academic and behavior problems as early as
kindergarten. Further attention should be paid to addressing the transactional nature of
internalizing problem behaviors and academic performance.

4.2. Adjustment Erosion, Academic Incompetence, or Bidirectional


While most studies support adjustment erosion as the primary model in the investiga-
tion of externalizing problems and academic performance (e.g., Chen et al., 2010; Deighton
et al., 2018; Zhou et al., 2010), this study provides evidence of a bidirectional, reciprocal link-
age, suggesting that externalizing problem behaviors are related to change in academic fail-
ures and vice versa. As noted in prior research, few studies have found bidirectional path-
ways between externalizing behaviors and academic performance (e.g., [11,13,18,19,47,48]).
Additional studies are needed that further explore the link between externalizing problem
behavior and academic achievement in early childhood. Notably, and contrary to our
findings from kindergarten to first grade and second grade to third grade, externalizing
problem behaviors in first grade did not predict academic achievement in second grade.
Future studies can also examine whether other factors covary with externalizing problem
behaviors and affect academic achievement.
Research on the associations between academic performance and internalizing prob-
lem behaviors is less conclusive. The academic incompetence model, which posits that
poor academic competence predicts problem externalizing and internalizing behaviors, has
been validated from middle childhood to late adolescence by many researchers. Consistent
with prior studies (e.g., [29–31]), the linkage between internalizing problem behaviors and
academic performance has also been found to be bidirectional.

4.3. Implications in Research and Practice


This study delineated the timing and processes between problem behaviors and
academic performance, which has important implications for guiding the targets and
timing of intervention efforts. A nuanced understanding of the complex mechanisms
Int. J. Environ. Res. Public Health 2022, 19, 9583 10 of 13

between behavioral and academic problems could strengthen our capacity to assess and
identify students’ barriers to learning and help inform the development of evidence-based
intervention efforts. The investigation of transaction relationships is crucial not only for
basic research but also for prevention/intervention efforts. It will help practitioners target
domains of adjustment that will likely contribute to problems in other domains. Armed
with the knowledge that these problem behaviors are more likely to interact with academic
achievement, educators may be alert to the presentation of externalizing/internalizing
symptoms and provide the support needed to those who are struggling with behavioral
challenges and prevent more pervasive difficulties in other domains.
Educators should be mindful of these predictive associations and should strive to
provide holistic support for students’ externalizing behaviors and internalizing symptoms
alongside their academic challenges. Intervening across domains may forestall the “spread”
of children’s difficulties; for instance, if a child presents with externalizing behavior prob-
lems, intervention in both behavior and academic performance may mitigate the risk of
later academic problems. Due to the complex and significant associations between external-
izing behaviors, internalizing symptoms, and academic challenges, educators may benefit
from the implementation of a robust social-emotional curriculum as a complement to their
academic instruction and interventions.

4.4. Limitations and Further Research


Future research can provide further clarity to the directionality between academic
performance and behavior with the consideration of the investigation of variables that
moderate or mediate these associations. It is important to explore the salient student char-
acteristics that may moderate the association between problem behaviors and academic
achievement. This notion ties back to the shared risk theory, which presents the possibility
that the association between academic performance and behavior may be explained by
another variable (e.g., gender, age, race, socioeconomic status, or family characteristics). Fu-
ture research can identify salient student characteristics, such as gender and socioeconomic
status, which likely moderate the transactional relationships. The examination of gender
differences is warranted in the timing and nature of progressions, as prior longitudinal
studies have indicated gender-specific transactional relationships (e.g., [19,26,30]) in differ-
ent developmental stages. Practitioners may benefit from learning about whether these
are gender-specific pathways between behavior and academic performance, particularly
regarding internalizing symptoms in girls. Further research can look into how varying so-
cioeconomic status affects the association between behavior and academic performance. In
addition, racial/ethnic differences are likely to be at play, given the deeply rooted structural
racism in social institutions in the US, such as the education system [49].
Future research can also identify mediators in the bidirectional models to unpack
the developmental dynamics in the early school years. Developmental variables, such
as interpersonal social skills and peer competence, have been strongly implicated in the
association between symptoms and adaptive behavior [5,50]. In addition, Metsäpelto
et al. [21] also indicated the mediating role of task-avoidant behavior on the pathways
between externalizing problems and academic performance.
We limited the scope of work and focused on two domains of problem behaviors
and one major academic performance measure (composite of reading and math). There
was a notable degree of difference between the conceptualization of behavioral constructs.
Different types of internalizing symptoms may yield different associations with academic
outcomes. For instance, while depression and anxiety both fall under the umbrella term “in-
ternalizing disorders,” studies that examined these in isolation were more likely to produce
findings that diverged from the trends identified in studies that examined internalizing
symptoms as a singular construct. Future research can examine the relationships between
specific internalizing behavior, such as depression and anxiety, and specific domains of
academic performance (reading and math). In addition, children’s problem behaviors were
solely based on the ratings reported by teachers, which might introduce assessment bias.
Int. J. Environ. Res. Public Health 2022, 19, 9583 11 of 13

Future research can collect multiple sources of data to triangulate the findings on children’s
problem behaviors.

5. Conclusions
This longitudinal study indicated bidirectional relationships between behavioral and
academic problems, with externalizing and internalizing problem behavior predicting
academic performance and vice versa in a nationally representative student sample in
early childhood. The findings of the current study help enhance the understanding of how
academic and behavior challenges co-interact over time and suggest that, early on, teachers
can identify and support students who experience academic and behavioral difficulties.

Author Contributions: Conceptualization, C.-C.C.; methodology, C.-C.C., F.Z. and N.A.; formal
analysis, S.-L.C.; investigation, C.-C.C. and Y.X.; resources, R.S. and M.W.-N.; writing—original draft
preparation, C.-C.C.; writing—review and editing, Y.X. and K.R.; visualization, S.-L.C. All authors
have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Cairns, R.B.; Cairns, B.D. Lifelines and Risks: Pathways of Youth in Our Time; Harvester Wheatsheaf: New York, NY, USA, 1994.
2. Chen, C.-C.; Culhane, D.P.; Metraux, S.; Park, J.M.; Venable, J.C. The heterogeneity of truancy among urban middle school
students: A latent class growth analysis. J. Child Fam. Stud. 2016, 25, 1066–1075. [CrossRef]
3. Olivier, E.; Archambault, I.; Dupere, V. Boys’ and girls’ latent profiles of behavior and social adjustment in school: Longitudinal
links with later student behavioral engagement and academic achievement? J. Sch. Psychol. 2018, 69, 28–44. [CrossRef] [PubMed]
4. Roeser, R.W.; Peck, S.C. Patterns and pathways of educational achievement across adolescence: A holistic-developmental
perspective. New Dir. Child Adolesc. Dev. 2003, 101, 39–62. [CrossRef] [PubMed]
5. Masten, A.S.; Roisman, G.I.; Long, J.D.; Burt, K.B.; Obradovic, J.; Riley, J.R.; Boelcke-Stennes, K.; Tellegen, A. Developmental
Cascades: Linking Academic Achievement and Externalizing and Internalizing Symptoms Over 20 Years. Dev. Psychol. 2005, 41,
733–746. [CrossRef]
6. Grant, K.E.; Katz, B.N.; Thomas, K.J.; O’Koon, J.H.; Meza, C.M.; DiPasquale, A.-M.; Rodriguez, V.O.; Bergen, C. Psychological
Symptoms Affecting Low-Income Urban Youth. J. Adolesc. Res. 2004, 19, 613–634. [CrossRef]
7. Yoshikawa, H.; Aber, J.L.; Beardslee, W.R. The effects of poverty on the mental, emotional, and behavioral health of children and
youth: Implications for prevention. Am. Psychol. 2012, 67, 272–284. [CrossRef] [PubMed]
8. Hinshaw, S.P. Externalizing behavior problems and academic underachievement in childhood and adolescence: Causal relation-
ships and underlying mechanisms. Psychol. Bull. 1992, 111, 127–155. [CrossRef] [PubMed]
9. Roeser, R.W.; Eccles, J.S.; Strobel, K.R. Linking the study of schooling and mental health: Selected issues and empirical illustrations
at the level of the individual. Educ. Psychol. 1998, 33, 153–176. [CrossRef]
10. Valdez, C.; Lambert, S.; Ialongo, N. Identifying patterns of early risk for mental health and academic problems in adolescence: A
longitudinal study of urban youth. Child Psychiatry Hum. Dev. 2011, 42, 521–538. [CrossRef]
11. Moilanen, K.L.; Shaw, D.S.; Maxwell, K.L. Developmental cascades: Externalizing, internalizing, and academic competence from
middle childhood to early adolescence. Dev. Psychopathol. 2010, 22, 635–653. [CrossRef]
12. Okano, L.; Jeon, L.; Crandall, A.; Riley, A. Differential effects of internalizing behaviors on academic functioning for girls versus
boys: An analysis of developmental cascades from elementary to high school. Dev. Psychopathol. 2020, 32, 751–764. [CrossRef]
13. Zhang, W.; Zhang, L.; Chen, L.; Ji, L.; Deater-Deckard, K. Developmental changes in longitudinal associations between academic
achievement and psychopathological symptoms from late childhood to middle adolescence. J. Child Psychol. Psychiatry Allied
Discip. 2019, 60, 178–188. [CrossRef] [PubMed]
14. Bronfenbrenner, U. Ecology of the family as a context for human development: Research perspectives. Dev. Psychol. 1986, 22, 723.
[CrossRef]
15. Sameroff, A.J. Developmental systems and psychopathology. Dev. Psychopathol. 2000, 12, 297–312. [CrossRef] [PubMed]
16. Cox, M.J.; Mills-Koonce, R.; Propper, C.; Gariépy, J.-L. Systems theory and cascades in developmental psychopathology. Dev.
Psychopathol. 2010, 22, 497–506. [CrossRef]
Int. J. Environ. Res. Public Health 2022, 19, 9583 12 of 13

17. Hinshaw, S.P.; Anderson, C.A. Conduct and oppositional defiant disorders. In Child Psychopathology; Mash, E.J., Barkley, R.A.,
Eds.; Guilford Press: New York, NY, USA, 1996; pp. 113–149.
18. Zimmermann, F.; Schütte, K.; Taskinen, P.; Köller, O. Reciprocal effects between adolescent externalizing problems and measures
of achievement. J. Educ. Psychol. 2013, 105, 747–761. [CrossRef]
19. Okano, L.; Jeon, L.; Crandall, A.; Powell, T.; Riley, A. The Cascading Effects of Externalizing Behaviors and Academic Achievement
across Developmental Transitions: Implications for Prevention and Intervention. Prev. Sci. Off. J. Soc. Prev. Res. 2020, 21, 211–221.
[CrossRef] [PubMed]
20. Weeks, M.; Ploubidis, G.B.; Cairney, J.; Wild, T.C.; Naicker, K.; Colman, I. Developmental pathways linking childhood and
adolescent internalizing, externalizing, academic competence, and adolescent depression. J. Adolesc. 2016, 51, 30–40. [CrossRef]
21. Metsäpelto, R.-L.; Pakarinen, E.; Kiuru, N.; Poikkeus, A.-M.; Lerkkanen, M.-K.; Nurmi, J.-E. Developmental dynamics between
children’s externalizing problems, task-avoidant behavior, and academic performance in early school years: A 4-year follow-up.
J. Educ. Psychol. 2015, 107, 246–257. [CrossRef]
22. Van der Ende, J.; Verhulst, F.C.; Tiemeier, H. The bidirectional pathways between internalizing and externalizing problems and
academic performance from 6 to 18 years. Dev. Psychopathol. 2016, 28, 855–867. [CrossRef]
23. Chen, X.; Huang, X.; Chang, L.; Wang, L.; Li, D. Aggression, social competence, and academic achievement in Chinese children:
A 5-year longitudinal study. Dev. Psychopathol. 2010, 22, 583–592. [CrossRef] [PubMed]
24. Deighton, J.; Humphrey, N.; Belsky, J.; Boehnke, J.; Vostanis, P.; Patalay, P. Longitudinal pathways between mental health
difficulties and academic performance during middle childhood and early adolescence. Br. J. Dev. Psychol. 2018, 36, 110–126.
[CrossRef] [PubMed]
25. Obradović, J.; Burt, K.; Masten, A. Testing a Dual Cascade Model Linking Competence and Symptoms Over 20 Years from
Childhood to Adulthood. J. Clin. Child Adolesc. Psychol. 2010, 39, 90–102. [CrossRef] [PubMed]
26. Panayiotou, M.; Humphrey, N. Mental health difficulties and academic attainment: Evidence for gender-specific developmental
cascades in middle childhood. Dev. Psychopathol. 2018, 30, 523–538. [CrossRef] [PubMed]
27. Zhou, Q.; Main, A.; Wang, Y. The relations of temperamental effortful control and anger/frustration to Chinese children’s
academic achievement and social adjustment: A longitudinal study. J. Educ. Psychol. 2010, 102, 180–196. [CrossRef]
28. Vaillancourt, T.; Brittain, H.L.; McDougall, P.; Duku, E. Longitudinal Links Between Childhood Peer Victimization, Internalizing
and Externalizing Problems, and Academic Functioning: Developmental Cascades. J. Abnorm. Child Psychol. 2013, 41, 1203–1215.
[CrossRef]
29. Keles, S.; Olseth, A.R.; Idsøe, T.; Sørlie, M.A. The longitudinal association between internalizing symptoms and academic
achievement among immigrant and non-immigrant children in Norway. Scand. J. Psychol. 2018, 59, 392–406. [CrossRef] [PubMed]
30. Verboom, C.E.; Sijtsema, J.J.; Verhulst, F.C.; Penninx, B.W.J.H.; Ormel, J. Longitudinal associations between depressive problems,
academic performance, and social functioning in adolescent boys and girls. Dev. Psychol. 2014, 50, 247–257. [CrossRef] [PubMed]
31. Weidman, A.C.; Augustine, A.A.; Murayama, K.; Elliot, A.J. Internalizing symptomatology and academic achievement: Bi-
directional prospective relations in adolescence. J. Res. Personal. 2015, 58, 106–114. [CrossRef]
32. Hall, C.M.; Welsh, J.A.; Bierman, K.L.; Nix, R. Kindergarten social withdrawal and reading achievement: A cross-lagged path
model for at-risk learners. Psychol. Sch. 2016, 53, 751–759. [CrossRef]
33. Davis, J.P.; Dumas, T.M.; Merrin, G.J.; Espelage, D.L.; Tan, K.; Madden, D.; Hong, J.S. Examining the pathways between bully
victimization, depression, academic achievement, and problematic drinking in adolescence. Psychol. Addict. Behav. 2018, 32,
605–616. [CrossRef] [PubMed]
34. Wang, M.T.; Eccles, J.S. Adolescent behavioral, emotional, and cognitive engagement trajectories in school and their differential
relations to educational success. J. Res. Adolesc. 2012, 22, 31–39. [CrossRef]
35. Tourangeau, K.; Nord, C.; Lê, T.; Sorongon, A.G.; Hagedorn, M.C.; Daly, P.; Najarian, M. Early Childhood Longitudinal Study,
Kindergarten Class of 2010–11 (ECLS-K:2011), User’s Manual for the ECLS-K:2011 Kindergarten Data File and Electronic Codebook, Public
Version; (NCES 2015-074); U.S. Department of Education, National Center for Education Statistics: Washington, DC, USA, 2015.
36. Gresham, F.M.; Elliott, S.N.; Vance, M.J.; Cook, C.R. Comparability of the Social Skills Rating System to the Social Skills
Improvement System: Content and psychometric comparisons across elementary and secondary age levels. Sch. Psychol. Q. 2011,
26, 27–44. [CrossRef]
37. Duncan, S.E.; De Avila, A. preLAS 2000 Cue Picture Book English Form C; CTB/McGraw-Hill Companies, Inc.: Monterey, CA, USA,
1998.
38. Kearney, M.W. Cross lagged panel analysis. In The SAGE Encylopedia of Communication Research Methods; Allen, M.R., Ed.; SAGE
Publications Ltd.: New York, NY, USA, 2017; pp. 313–314.
39. Little, T.D. Longitudinal Structural Equation Modeling; The Guilford Press: New York, NY, USA, 2013.
40. Latvala, A.; Rose, R.J.; Pulkkinen, L.; Dick, D.M.; Korhonen, T.; Kaprio, J. Drinking, smoking, and educational achievement:
Cross-lagged associations from adolescence to adulthood. Drug Alcohol Depend. 2014, 137, 106–113. [CrossRef] [PubMed]
41. Tucker, J.S.; Miles, J.N.V.; D’Amico, E.J.; Zhou, A.J.; Green, H.D.; Shih, R.A. Temporal associations of popularity and alcohol use
among middle school students. J. Adolesc. Health 2013, 52, 108–115. [CrossRef] [PubMed]
42. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives.
Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [CrossRef]
43. Muthén, L.K.; Muthén, B.O. Mplus User’s Guide, 8th ed.; Mplus: Los Angeles, CA, USA, 2017.
Int. J. Environ. Res. Public Health 2022, 19, 9583 13 of 13

44. Enders, C.K. Applied Missing Data Analysis; Guilford Press: New York, NY, USA, 2010.
45. Lee, D.Y.; Harring, J.R.; Stapleton, L.M. Comparing methods for addressing missingness in longitudinal modeling of panel data. J.
Exp. Educ. 2019, 87, 596–615. [CrossRef]
46. Kline, R.B. Structural Equation Modeling, 4th ed.; Guilford Press: New York, NY, USA, 2016.
47. Palmu, I.R.; Närhi, V.M.; Savolainen, H.K. Externalizing behaviour and academic performance—The cross-lagged relationship
during school transition. Emot. Behav. Diffic. 2018, 23, 111–126. [CrossRef]
48. Wigelsworth, M.; Qualter, P.; Humphrey, N. Emotional self-efficacy, conduct problems, and academic attainment: Developmental
cascade effects in early adolescence. Eur. J. Dev. Psychol. 2017, 14, 172–189. [CrossRef]
49. Roberts, S.O.; Rizzo, M.T. The psychology of American racism. Am. Psychol. 2021, 76, 475–487. [CrossRef]
50. Masten, A.S.; Curtis, W.J. Integrating competence and psychopathology: Pathways toward a comprehensive science of adaptation
in development. Dev. Psychopathol. 2000, 12, 529–550. [CrossRef] [PubMed]

You might also like