International Journal of Learning and Teaching Vol. 2, No.

2, December 2016

Measuring Coach Effects in Archery Sport with

Hierarchical Linear Modeling
Kai-Li Wang
Graduate Institute of International Sport Affairs, National Taiwan Sport University, Taoyuan, Taiwan
Email: elverz@ms8.hinet.net

Yun-Ci Ye
Department of Somatic and Sports Leisure Industry, National Taitung University, Taitung, Taiwan
Email: scanker1981@nttu.edu.tw

Abstract—Little quantitative research has been done on coaching results [3], [4]. Some other studies using the
coaching effects. This is because scoring in sports games is players’ satisfaction as the criterion for evaluating the
often associated with teammates, adversaries, and referees. coach’s coaching behavior suggest that various facets of
This paper attempts to initiate a probe into archery, whose the coach’s coaching behavior are associated with the
scoring is relatively objective, to measure the coach effects
players’ satisfaction [5], [6]. For the coaching behavior,
in training. The score records, totally 6,044 pieces of data
from 2270 players and 146 coaches, about standard outdoor the Multi-dimension Model of Leadership (MML) –
target archery rounds in national archery competitions held which is established using the Leadership Scale for
between late 1997 and early 2007 in Taiwan were collected. Sports (LSS) designed by Chelladurai and Saleh [7] – is
The authors of this study managed to collect supplementary commonly used as the independent variable to assess the
data of these players and coaches, including gender, coach’s coaching behavior. Results show that: the
seniority and educational background, etc. HLM dimensions of ‘training and instruction’, ‘democratic
(hierarchical linear modeling) analysis was performed by behavior’, ‘social support’, & ‘positive feedback’ have
including the player-related data in the individual level of the positive relationship with the players’ satisfaction,
independent variables, and the coach-related data in the
while ‘autocratic behavior’ has the negative relationship.
group level of independent variables with the 30-m archery
game scores as the dependent variable. Results show that There are some other studies focusing on the structural
the coaching effect is non-ignorable because the intra-class relationship: “coach’s behavior — player’s perception
correlation coefficient is high enough. We performed a and memory — player’s response evaluation” [8].
further linear regression analysis of both individual level Especially highlighted is the confusion over the coach’s
and group level. Results show that the coach effect is more objective behavior and the player’s subjective perception
important in the early-stage training. The training quality as resulted from the players’ self-assessment of coach’s
(measured by coach-player ratio) also seems considerable to behavior in past studies. These studies observed, recorded
training performance. The study result can serve as and coded the coach’s actual behavior by utilizing the
references for sport and physical education policy.
Coaching Behavior Assessment System (CBAS). A post-
Index Terms—archery, sports performance, coach effect,
game questionnaire survey on the players’ perception and
coach-player ratio, hierarchical linear modeling attitude was also conducted to analyze the influences
among them. Results show that the player-perceived
coach behavior explained most of the variances; in some
I. INTRODUCTION studies, it even mediated the coach’s actual behavior to a
full extent [9], [10]. These studies shed light on how
Most of previous coaching effects studies on the psychological and sociological studies can be limited by a
influence of sportsmen’s performance focus on questionnaire-based approach to measure coaching
psychology and sociology, showing the tendency to behaviors and coaching effects. The limitations include
indirectly evaluate sportsmen by questionnaire survey [1]. less objectivity and an overestimation of the association
Some of these studies go through the association between between coaching behaviors/effects.
the coach’s coaching behavior and the player’s scores. Comparatively objective criteria for assessing coaching
Garland and Barry use the play time as the criteria (i.e. effects may include coaching time, number of
starter/lineup, dugout/bench player, backup player) for instructions given to players, or other coach-specific
evaluating sports performance, finding out that the variables (e.g. the coach’s seniority and gender; the
coach’s coaching behavior has outstanding predictive coaching stage to the player). In the past, few studies are
power on the sports performance [2]. However, more available concerning the correlation of these variables
researchers use the sports performance perceived by with the actual sports performance. These easily-
players as the dependent variable to predict the coach’s observable manifest variables used to be neglected as a
moderator in the fields of psychology and sociology.
Manuscript received July 1, 2015; revised December 23, 2015. Reasons may include the following situations: (1)

© 2016 International Journal of Learning and Teaching 156

doi: 10.18178/ijlt.2.2.156-160
 International Journal of Learning and Teaching Vol. 2, No. 2, December 2016

Concerns over issues may vary. Observing only manifest where:

variables (such as gender, seniority) is considered Yij = player i’s 30-m archery game scores, on coach j,
insufficient as the psychology and sociology focus on Genderij = player i’s male or female, on coach j,
interpersonal interaction, teaching scenario, etc. (2) Better Educationij = player i’s elementary school, junior high,
understanding of implicit variables such as attitude and senior high, or university, on coach j,
motive, feeling requires a questionnaire survey to be Seniorityij = player i’s years of seniority, on coach j,
conducted. The collection of questionnaire survey is quite Timesij = player i’s times of participation in games, on
costly, especially for longitudinal researches. Failure to coach j,
address technical barriers or cost problems may cause γij = stochastic error term, γij ~ N(0, σ2) is assumed.
doubts about the validity and reliability of data collected.
Under such a circumstance, analyzing only the manifest Equation (1) is referred to as the ‘level one model’ ,
variables may cause doubts about the accuracy of providing a specified model for 30-m archery game
explained variances (especially when involved in cross- scores at the player level.
period data of coaches and players) as compared with A hierarchical linear modeling further assumes that the
typical psychology and sociology statistical analysis. (3) coefficient β0j to β4j can be modeled as functions of
Failure to assess sports performance (the dependent coach-level effects, for example:
variable) in an objective manner may also undermine the
researcher’s determination to collect data. The collection β0j = γ00 + γ01 Genderj + γ02 Primaryj + γ03
of large amounts of data on game scores for long periods Juniorj+ γ04 Seniorj+γ05 CP1997j+ γ06
of time, after all, is ever easy. The inclusion of unwanted CP1998j+ γ07 CP1999j + γ08 CP2000j+γ09
information from evaluators (e.g. the influence by the CP2001j+ γ010 CP2002j+ γ011 CP2003j+
judge) will undermine the possibility of such research. γ012 CP2004j+ γ013 CP2005j+ γ014 CP2006j
To overcome the above difficulties, this paper will +υ0j (2)
collect cross-period data from Taiwanese archery players β1j = γ10 + υ1j (3)
by using Hierarchical Linear Modeling (HLM) for
β2j = γ20 + υ2j (4)
empirical study. The field of archery is chosen as the
subject for its less sensitivity to the judge and the β3j = γ30 + υ3j (5)
teammate, which can leave the objectivity of dependent β4j = γ40 + υ4j (6)
variable to be neglected. Regarding independent variables,
we collected as many as objective variables related to where:
coach or player, including gender, seniority, identity and Genderj = male or female for coach j,
coach-player ratio. While this information may not be as Primaryj=the dummy variable for primary school
microscopic as what is collected via questionnaire survey, coach j
an HLM-based analysis can serve to separate coach- Juniorj = the dummy variable for junior school coach j
specific variance form player-specific variance, thereby Seniorj=the dummy variable for primary school coach j
elevating the model’s explanatory power. CP1997j ~ CP2006j = coach-player ratio of the teams
instructed by the coach j in
II. RESEARCH METHOD academic years 1997-2006
υ0j, υ1j, υ2j, υ3j, υ4j = stochastic error term which are
A. Statistical Method: Hierarchical Linear Modeling assumed to be normally distributed.
(HLM)
Equations (2), (3), (4), (5) and (6), referred to as the
In many cases, individual observations on dependent ‘level two model’, are designed to capture variations at
variables come from ‘nested’ environments, for example, the coach level. By modeling the coefficients β0j, β1j, β2j,
research in education, where HLM models have found β3j andβ4j as shown above, total variation in 30-m archery
their greatest application typically focus on the individual game scores can be divided into that variation which
student [11]. The student, however, is part of a class, occurs at level one (player attributes) and the variation
which is nested within a school belonging to a school which occurs at level two (coach attributes). While other
district. This nesting of observations suggests that methods, such as a one-level dummy variable approach,
individual performance may be a result of both student- may provide a similar analysis, they may be less efficient
level attributes as well as, say, school-level qualities. as they consume a greater number of degrees of freedom.
Analogously, the game scores that archery players
receive may be determined by player-level attributes as B. Objects and Data Description
well as coach-level attributes. That is, players with the In this paper we are able to collect archery
same level of skill may in fact have differing game scores competitions data on 2270 players and 146 coaches for
if they follow different coaches, due to coach-specific the years 1997 through 2007 in Taiwan (i.e. academic
effects. In order to separate out the player- and coach- years 1997-2006 of Taiwan). The kinds of archery
level effects, the following econometric model for player competition game in each year included President Cup,
game score can be considered: Youth Cup, and Master Cup, all of which are national
Yij = β0j + β1j Genderij + β2j Educationij + events held annually. However, in the player-level, the
β3j Seniorityij + β4j Timesij + γij (1) independent variables were gender, education

© 2016 International Journal of Learning and Teaching 157

 International Journal of Learning and Teaching Vol. 2, No. 2, December 2016

background, years of seniority, and times of participation the random coefficient regression models allowed us to
in games of players; in the coach-level, the independent use intercepts-as-outcomes models.
variables were gender, the coach-instructed school level A sequence of intercepts-as-outcomes models was
and the coach-player ratio of coach-instructed team in performed to test for the hypothesized relationships.
each academic year. Finally, the dependent variable was Parameter estimates of Level 2 intercepts-as-outcomes
30-m archery game scores in HLM model. models are reported in Table I. The results revealed that
the junior, senior, and CP2006 have significant impact on
III. RESULT 30-m archery game scores while the coach gender,
primary, and CP1997~CP2005 don’t have the effect.
First of all, Intra-class correlation coefficients, ICC(1),
were calculated to examine between-group variance. The
IV. CONCLUSION AND SUGGESTION
ICC(1) for 30-m archery game scores (30-m AGS)
was .30, (F(144) = 12.86, p< .001), indicating that 30% of
A. Conclusion
the variability in player ratings of 30-m AGS was related
to coach membership. However, ICC(1) values were Results of the above analysis indicate that it is not only
higher than a median value of .12 in the organizational conceptually meaningful but also statistically supports the
literature [12] and represent moderate to moderately high categorization of factors influencing archery game
ICC(1) values [13]. players’ scores into coach-level factor and player-level
Moreover, we ran null models to examine whether factor. In terms of the scores of archery game players in
there was systematic between-group variance in the Taiwan between academic years 1997-2006, different
dependent variables. Significant chi squares for 30-m coaches have statistical meaning to the performances of
AGS, (χ2(144) = 1423.36, p< .001), indicated that this players, accounting for 30% of variance in scores. The
prerequisite was met for each variable. About 30% of the remaining 70% of variance is the very factor that comes
variance in 30-m AGS was between-group variance. from players themselves.
This paper further demonstrates that factors —gender,
TABLE I. HLM SUMMARY education, and times— urge players to have influence on
Parameter
variance in scores. Among them, the parameter estimate
Variable SE df T of gender is positive, indicating that male archers had
estimate
Random coefficient regression model significantly better performance than female players did
Interceptγ00 217.26 7.68 144 26.68* because male archers could draw a bow better with their
Gender γ10 6.41 1.45 144 4.41* more physically-advantaged strengths and shoot an arrow
Education γ20 15.42 2.36 144 6.52*
Seniority γ30 -1.46 0.83 144 -1.77
at a quicker speed, making the flying arrow be less
Times γ40 6.67 0.67 144 9.89* affected by external force (such as wind force). Hence,
Intercepts-as-outcomes model male archers have higher shooting average. The
Gender γ01 -1.46 2.94 139 -0.49 parameter estimate of education is positive, indicating
Primary γ02 1.33 5.30 139 0.25 that: higher education background of playing archers
Junior γ03 23.95 4.61 139 5.19*
Senior γ04 14.04 4.48 139 3.13* means higher scores. With higher education background
CP1997 γ05 1.87 4.17 139 0.45 and resultant higher mental and physical maturity, players
CP1998 γ06 6.85 4.69 139 1.46 may improve their sports performance. The parameter
CP1999 γ07 2.13 4.04 139 0.53 estimate of times is positive, showing that more times of
CP2000 γ08 0.28 5.38 139 0.05
CP2001 γ09 6.34 5.75 139 1.10 participation lead to higher scores, which may be due to
CP2002 γ010 2.32 4.20 139 0.55 an increase in playing experiences. The effect of seniority
CP2003 γ011 -2.85 5.24 139 -0.55 is not significant probably because that the variance that
CP2004 γ012 -4.15 5.73 139 -0.72
CP2005 γ013 -9.34 5.05 139 -1.85
the seniority could measure is almost measured by
CP2006 γ014 24.52 7.87 139 3.12* players’ education background and times. With
“education” and “times” taken into consideration, the
The effects of player gender, education, seniority, and statistical model in this paper suggests that the seniority
times on dependent variables were controlled prior to has no statistical significance.
hypotheses testing. Random coefficients regression The statistical analysis of coach-level variables sees a
models were run to estimate whether player gender, more interesting result. The gender of the coach did not
education, seniority, and times were associated with the influence game scores, demonstrating equally effective
outcomes of interest. As shown in Table I, the results teaching performances between male coaches and female
indicated the player gender (γ10 = 6.41, t(144) = 4.41, coaches. The conclusion is intuitive. Out of curiosity
p< .05), education (γ20 = 15.42, t(144) = 6.52, p< .05), about the influence of coaches in different school levels
and times (γ40 = 6.67, t(144) = 9.89, p< .05) significantly on players’ scores, this paper takes the university coach
effect on 30-m archery game scores while the seniority as the reference, and includes the primary school, junior
doesn’t (γ30 = -1.46, t(144) = -1.77, p> .05). Chi-square high school and senior school coaches as three dummy
tests provided evidence that systematic variance in the variables (primary, junior, senior) in the statistical model
intercepts across coaches was found for 30-m archery (Note: A player delivers a better performance in a school
game scores (χ2(144) = 1287.31, p< .001). The significant of different level, while the coach does not. So the values
between-coaches variance in the player-level intercepts of of the variable of players’ education background can be

© 2016 International Journal of Learning and Teaching 158

 International Journal of Learning and Teaching Vol. 2, No. 2, December 2016

set as ordinal variable 1-4, with the coach-level variables CP2006 is significant, with all other CP values
set as dummy variable). Results of the analysis show that insignificant, as shown in Table I. This paper analyzes the
the primary variable is insignificant, but junior and senior reason and concludes that the few number of archery
variables are significant and the parameter estimates of coaches before 2005 accounts for the fierce
junior and senior are both positive. It indicates that the competitiveness over studying at those few schools with
influence of junior high school coach and senior high archery teams. This led to few samples of coach-player
school coach on archer’s scores is superior to that of ratio and low variance, meaning statistical insignificance.
university coach. In addition, the parameter estimate of Since 2004 when Taiwan won one silver medal and one
junior (=23.95) is greater than the parameter estimate of copper medal in Athens Olympic Games, the government
senior (=14.04), showing that the influence of junior high jumped on the bandwagon by investing more resources in
school coach on archer’s scores is superior to that of schools of difference levels to establish more archery
senior high school coach. As for the insignificance of teams. This is why CP2006 is significant because the
primary, it cannot be used to conclude that primary sample size of coach-player ratio and variance are
school coach is inferior because a number of micro included for year 2005 and later years. Positive parameter
variables concerning the coach and the player are still not estimate of CP2006 represents that the greater the coach-
separated out, leaving no statistical result that supports player ratio, the higher the player’s scores. This result
the primary school coach. We can assume that this is supports that coaching quality affects the player’s score,
also because that sports performance is not highly as perceived by the public.
demanded in primary schools, where the trainees are
B. Suggestion
supposed to only develop an interest in archery.
Therefore, the statistical model here does not reveal how It can be inferred from the above model that the factors
a primary school coach could actually affect a player’s of both coach and player have statistical significance on
scoring results. the scores. For players, their playing experiences, mental
and physical maturity, all of which contributed to their
TABLE II. SUMMARY OF FACTOR EFFECT focus on practices over long periods of time, may account
Level Factor Effect Explanation/Possible
for that significance. Therefore, we consider it
reason to not sig. strategically important to create a well-established
competitive sports platform, where players are willing to
Player Gender positive physically-advantaged
Level strengths train themselves as long-term investments.
Education positive age, physical, mental As for the factor of coach, the data on junior high
maturity school coaches, senior high school coaches and university
Seniority not sig. we have already considered coaches (except for primary school coach) seems to
the “experience” variable
support the argument “coaches at basic levels have
Times positive experience
Coach Gender not sig. no difference in coaching
greater influence on the player’s scores.” The competent
Level performance authorities are to take into consideration the importance
Primary not sig. no difference in coaching of coaches at basic levels for players. The government is
performance or data advised to invest more resources in basic training of
problem?
students at junior high schools (or at a lower level). The
Junior positive better than university coach
Senior positive better than university coach
goal is to lay a sturdy foundation for long-term
CP2006 positive more quality lead to better development of competitive sports.
performance Although a further verification is required to analyze
CP1997 not sig. no influence on the effect of coach-player ratio, so far we can conclude
~CP2005 performance or data that: The quality that coach-player ratio stands for may be
problem?
an important factor, too, that influences the player’s
scores. This reminds competent authorities of the
Some may argue that the coach of a school of different importance of investment in software/hardware resources
levels have different influence on the player’s score and implementation in order to have better competitive
because of the difference in software/hardware resources sports performance.
rather than the coach’s individual factors. It is generally
believed that the school’s resources (hardware, software, ACKNOWLEDGMENT
etc.) deal much with the quality of training, which plays a
vital role in a player’s scoring performances. Due to the Some data, analyses, and contents of this paper are
cost constraint, this paper fails to initiate investigation revised and improved from the manuscript of “A HLM
into the software/hardware resources owned by each analysis of scores of archers in Taiwan: Considering the
school. Instead, this paper organizes the data on coach- factors of coach, player, and coach-player ratio”, which
player ratio (=1/number of players) of the team instructed was oral presented in the 2010 ECSS (European College
by each coach in each academic year, which may be used of Sport Science) conference at Turkey, Antalya, 2010
as a quality index to reflect the software/hardware June 26. Part of paper is also revised from the incomplete
resources owned by each school. When the data on and developing manuscript of “Who Dominates Sports
coach-player ratios of the ten academic years were Performances? The Player or the Coach?: A HLM
included in the statistical model, it reveals that only Analysis of Scores of Archers in Taiwan”, which was

© 2016 International Journal of Learning and Teaching 159

 International Journal of Learning and Teaching Vol. 2, No. 2, December 2016

oral presented in the 2009 SMAANZ (Sport Management [10] R. M. Baron and D. A. Kenny, “The moderator-mediator variable
in psychological research: Conceptual, strategic, and statistical
Association of Australia and New Zealand) conference at
considerations,” Journal of Personality and Social Psychology,
Bond University, Gold Coast, Australia, 2009 November vol. 51, pp. 1173-1182, 1986.
27. However, the above 2 manuscripts were not officially [11] R. W. Rumberger and S. L. Thomas, “The economic returns to
published yet. Thanks for suggestions from those college major, quality and performance: A multilevel analysis of
recent graduates,” Economics of Education Review, vol. 12, no. 1,
colleagues in the conference. They make this paper
pp. 1-19, 1993.
complete. [12] P. D. Bliese, “Within-group agreement, non-independence, and
reliability: Implications for data aggregation and analysis,” in
REFERENCES Multilevel Theory, Research, and Methods in Organizations:
Foundations, Extensions, and New Directions, K. J. Klein and S.
[1] S. Jowett and D. Lavallee, Social Psychology in Sport, Champaign, W. J. Kozlowski, Eds, San Francisco: Jossey-Bass, 2000, pp. 349
IL: Human Kinetics, 2007, ch. 1, pp. 3-14. – 381.
[2] D. J. Garland and J. R. Barry, “The effects of personality and [13] P. D. Bliese and P. J. Hanges, “Being too liberal and too
perceived leader behavior on performance in collegiate football,” conservative: The perils of treating grouped data as though they
Psychological Record, vol. 38, pp. 237-247, 1988. were independent,” Organizational Research Methods, vol.7, pp.
[3] M. R. Weiss and W. D. Friedrich, “The influence of leader 400–417, 2004.
behaviors, coach attributes, and institutional variables on
performance and satisfaction of collegiate basketball teams,” Kai-Li Wang works as an assistant professor
Journal of Sport Psychology, vol. 8, pp. 332-346, 1986. in Graduate Institute of International Sport
[4] S. Serpa, V. Pataco, and F. Santos, “Leadership patterns in Affairs, National Taiwan Sport University. His
handball international competition,” International Journal of Sport area of interest is the law issue and public
Psychology, vol. 22, pp. 78-89, 1991. policy in leisure. entertainment and sport field.
[5] P. Chelladurai, H. Imamura, Y. Yamaguchi, Y. Oinuma, and Y. Recently, he did many researches about the
Miyauchi, “Sport leadership in a cross-national setting: The case image right, patent, public policy and
of Japanese and Canadian university athletes,” Journal of Sport & legislation regarding the sport industry based
Exercise Psychology, vol. 10, pp. 374-389, 1988. on the approach of economics of law.
[6] J. M. Dwyer and D. G. Fischer, “Wrestler’s perceptions of coachs’
leadership as predictors of satisfaction with leadership,”
Perceptual & Motor Skills, vol. 71, pp. 511-517, 1990. Yun-Ci Ye works as an assistant professor in
[7] P. Chelladurai and S. D. Saleh, “Dimensions of leader behavior in Department of Somatic and Sports Leisure
sports: Development of a leadership scale,” Journal of Sport Industry, National Taitung University. His
Psychology, vol. 2, pp. 34-45, 1980. area of interest is sport management, sport
[8] F. L. Smoll and R. E. Smith, “Leadership behaviors in sport: A consumer behavior, statistics, etc.. Recently,
theoretical model and research paradigm,” Journal of Applied he did many researches about life satisfaction,
Social Psychology, vol. 19, pp. 1522-1551, 1989. buying behavior, experience marking based on
[9] F. L. Smoll, R. E. Smith, B. Curtis, and E. Hunt, “Toward a the mothed of psychology statistics.
meditational model of coach-player relationships,” Research
Quarterly, vol. 49, pp. 528-541, 1978.

© 2016 International Journal of Learning and Teaching 160