Vandelanotte et al. BMC Public Health
(2019) 19:407
https://doi.org/10.1186/s12889-019-6717-1
RESEARCH ARTICLE
Open Access
Validity and responsiveness to change of
the Active Australia Survey according to
gender, age, BMI, education, and physical
activity level and awareness
Corneel Vandelanotte1* , Mitch J. Duncan2, Rob Stanton3, Richard R. Rosenkranz4, Cristina M. Caperchione5,
Amanda L. Rebar1, Trevor N. Savage6, W. Kerry Mummery7 and Gregory S. Kolt6
Abstract
Background: This study aimed to investigate the validity of the Active Australia Survey across different subgroups
and its responsiveness to change, as few previous studies have examined this.
Methods: The Active Australia Survey was validated against the ActiGraph as an objective measure of physical
activity. Participants (n = 465) wore the ActiGraph for 7 days and subsequently completed the Active Australia
Survey. Moderate activity, vigorous activity and total moderate and vigorous physical activity were compared using
Spearman rank-order correlations. Changes in physical activity between baseline and 3-month assessments were
correlated to examine responsiveness to change. The data were stratified to assess outcomes according to different
subgroups (e.g., gender, age, weight, activity levels).
Results: With regards to the validity, a significant correlation of ρ = 0.19 was found for moderate physical activity, ρ = 0.33
for vigorous physical activity and ρ = 0.23 for moderate and vigorous physical activity combined. For vigorous physical
activity correlations were higher than 0.3 for most subgroups, whereas they were only higher than 0.3 in those with a
healthy weight for the other activity outcomes. With regards to responsiveness to change, a correlation of ρ = 0.32 was
found for moderate physical activity, ρ = 0.19 for vigorous physical activity and ρ = 0.35 for moderate and vigorous
physical activity combined. For moderate and vigorous activity combined correlations were higher than 0.4 for several
subgroups, but never for vigorous physical activity.
Conclusions: Little evidence for the validity of Active Australia Survey was found, although the responsiveness to change
was acceptable for several subgroups. Findings from studies using the Active Australia Survey should be interpreted with
caution.
Trial registration: World Health Organisation Universal Trial Number: U111–1119-1755. Australian New Zealand Clinical
Trials Registry, ACTRN12611000157976. Registration date: 8 March 2011.
Keywords: Measurement, Self-report, Accelerometer, Socio-demographics, Surveillance, Subgroups
* Correspondence: c.vandelanotte@cqu.edu.au
1
Physical Activity Research Group, School of Human, Health and Social
Sciences, Central Queensland University, Rockhampton, QLD, Australia
Full list of author information is available at the end of the article
© The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Vandelanotte et al. BMC Public Health
Page 2 of 11
(2019) 19:407
Background
Regular physical activity reduces the risk for developing
chronic diseases, yet large proportions of the population
are inactive which leads to an increased burden of disease in Australia [1, 2]. As such, robust physical activity
measures are important for epidemiology, surveillance
and evaluation of interventions. The most used,
cost-effective and feasible method of assessing physical
activity in large populations is through using self-report
questionnaires [3]. Although limitations associated with
self-report measurements are well known [4, 5], and the
use of accelerometer-based physical activity monitoring
is becoming increasingly feasible [6], self-reported measurement still represents an efficient way to collect data
on physical activity in population health research.
The accurate collection of physical activity using
self-reported methods is not easy, as it depends on a
number of factors. Accuracy relies on participants’ ability
to correctly recall physical activity performed in the past,
whether participants’ interpretation of physical activity
intensity aligns with established definitions for activity
intensity, as well as whether survey questions are able to
capture these interpretations of intensity [7]. For example, women or older adults may perceive that specific
activities of moderate intensity require greater effort
than what men and younger adults perceive of the same
activities, and therefore rather classify them as being of
vigorous intensity [8, 9]. Furthermore, physical activity
questionnaires validated for use in one population are
often used in different populations or settings in which
they have not been validated. It is therefore important to
investigate the extent to which the validity of a
self-report instrument varies across different populations
[5]. If validity differs by population group, then this has
important implications for physical activity surveillance.
Few studies have examined the accuracy of self-report
questionnaires according to socio-demographic factors.
While some studies demonstrate that self-reports can be
accurate for women and older age adults [9, 10], they
did not simultaneously assess validity in men and young
adults. Nevertheless, some studies have compared
groups and indicated that self-report accuracy decreases
when BMI increases and when activity levels increase [7,
11]. More research is however needed to confirm these
observations.
During the past 15 years, the Active Australia Survey
[12] has been widely used to measure physical activity in
Australian and international surveillance studies and
large cohort studies [13–16]. The Active Australia Survey assesses frequency (sessions) and duration (minutes)
of physical activity in the past week [12]. While correlation coefficients of 0.3 have often been reported as
demonstrating acceptable evidence of validity in physical
activity research [17–22], a systematic review on the
validity of physical activity questionnaires by Helmerhortst et al. (2012) categorised validity as poor when correlations were below 0.4 [23]. The measurement
properties of the Active Australia Survey have been
assessed, and with correlation coefficients for total physical activity ranging from 0.42 to 0.61 [10, 11], they have
been deemed acceptable [9, 10]. Few studies, however,
have examined how the validity differs across different
subgroups [7, 11, 23]. Furthermore, although the Active
Australia Survey was developed for physical activity surveillance [12]; it is nevertheless often used in intervention
research with small study groups and detecting change in
small groups requires greater measurement sensitivity
[24]. Two studies have examined responsiveness to change
using the Active Australia survey [25, 26], but only one of
these studies examined this in comparison to an objective
measure of physical activity [26]. This study found acceptable but lower responsiveness of the Active Australia Survey compared to using accelerometry [26]. Therefore the
aims of this study were: 1) to investigate the validity of the
Active Australia Survey in different population subgroups
from a sample of Australian adults who participated in a
randomised controlled trial; and 2) to investigate the responsiveness to change of the Active Australia Survey
relative to objective accelerometer assessments.
Methods
The Edinburgh Validity and Reliability Framework was
used to specify what types of validity our study assessed
[27]. Specifically, when we refer to ‘validity’ we mean
‘criterion validity’ and when we refer to ‘responsiveness
to change’ we mean ‘behavioural reliability’ (i.e., assessment of stability accounting for behavioural changes).
Participants
All participants in this study were part of the Walk 2.0
trial [28, 29], a three-group randomised controlled trial
that assessed the effectiveness of a traditional physical
activity promotion website (www.10000steps.org.au), a
social networked physical activity promotion website
(www.walk.org.au), and a print-based control group. Details of the study methods and procedures of Walk 2.0
have been published previously [28]. A total of 504 participants were recruited via random selection from Australian electoral roll, local print media, and email lists.
Eligible participants were inactive English speaking
adults (+ 18 years) with Internet access who lived in
Western Sydney or Rockhampton. All participants in the
Walk 2.0 trial who were randomised into a group were
included in this study. A single item physical activity
measure was used to screen participant activity levels
prior to randomisation [30]. While the aim of the Walk
2.0 study was to only recruit inactive participants, 42.9%
met the physical activity recommendations at baseline
Vandelanotte et al. BMC Public Health
(2019) 19:407
[29]. We have reported more details on the screening
procedure and its limitations elsewhere [31]. The issues
with recruiting an inactive sample suggest that many of
those recruited were motivated to become more active,
and as such they may have been different from the Australian population at large.
Procedures
Eligible participants were invited to the university, fitted
with an ActiGraph activity monitor and instructed to
wear it for 7 days. Participants were asked to record wear
time and reasons for removing the ActiGraph during the
day (e.g. water sports) using a paper-based log. Eight
days later participants returned to the university and
completed the Active Australia Survey. Before completing the Active Australia Survey, ActiGraph data were
inspected; if individual data were invalid, participants
were asked to wear the ActiGraph again, until valid data
were obtained. The Walk 2.0 trial measured participants
using this protocol at 4 time points (0, 3, 12 and 18
months), however the present paper only reports outcomes for baseline (validity) and 3-month (sensitivity to
change) time points. Only baseline data were used to assess validity, as the subsequent intervention would have
intentionally influenced physical activity at later time
points. Only baseline and 3-month time points were
used to assess responsiveness to change, as actual physical activity change due to the intervention will have
been the greatest immediately after completing the intervention and also because drop-out increased over subsequent time points which may introduce selection bias.
Measures
The Active Australia Survey: this survey comprises eight
items to assess the frequency (number of sessions) and
duration (minutes per week) of walking, moderate and
vigorous leisure physical activities and vigorous gardening (in at least 10-min bouts) over the preceding 7 days.
Acceptable 5-day test-retest reliability has been reported
for the Active Australia Survey with reliability coefficients (spearman’s ρ) ranging between 0.43 and 0.80 and
agreement scores (Kappa statistics) ranging between
0.40 and 0.83 [11]. Consistent with Active Australia Survey data treatment recommendations, when participants
reported spending time in vigorous gardening, these data
were not included in any calculations of total and vigorous physical activity [12]. Duration (minutes per week)
for walking, moderate and vigorous physical activities
were truncated at 840 min [12]. Total minutes for moderate physical activity (which includes walking minutes),
vigorous physical activity and total moderate and vigorous physical activity were calculated. Total minutes for
moderate physical activity and walking were combined,
as several studies have demonstrated that healthy adults’
Page 3 of 11
self-selected walking speed usually corresponds with
moderate intensity physical activity [32, 33].
The ActiGraph activity monitor: The ActiGraph (model
GT3x; ActiGraph LLC., Florida) was used to objectively
measure physical activity. Although accelerometers do not
provide a gold-standard measure, they are not subject to
the same sources of error as self-reporting, and are
well-accepted for providing evidence for the validity of
self-report measures [9, 34, 35]. The validity and reliability
of the ActiGraph accelerometer has previously been demonstrated in laboratory testing, and compared to other
commercially available activity monitors [36, 37]. For example, ActiGraph counts per minute were highly correlated with oxygen uptake (VO2) during treadmill running
at various speeds (r = .88) [38]. During the induction session, participants were instructed on how to wear the
ActiGraph, which was worn over their right hip and fastened using an elastic belt.
Demographics: Age, gender and education level were
assessed as part of the survey measure, however height
and weight were measured by project staff when participants visited the university using Seca 700 balance scales
and a Seca 220 measuring rod (Seca GmbH, Hamburg).
Participants removed shoes and heavy personal items.
The average of 3 consecutive measurements was recorded.
Using BMI (kg/m2) participants were classified as healthy
weight (BMI = 18.5–24.9), overweight (BMI = 25.0–29.9)
or obese (BMI ≥ 30) [39]. Educational level was initially
assessed in 6 categories, but was collapsed into 3 categories for this study (school education, vocational and technical education, higher education). Educational level may
influence the validity and responsiveness of survey instruments. A higher education level may result in better
knowledge about physical activity, a better understanding
of being active at different intensities and having a better
recall of past activities [40]. This is why we examined validity and responsiveness to change according to educational level.
Physical Activity Awareness: Awareness was assessed
using the five items that accompany the assessment of
the Active Australia Survey [12]. When participants answered 4 or more questions correctly they were categorised as having ‘high physical activity awareness’; if
fewer than 4 questions were answered correctly participants were categorised as having ‘Low physical activity
awareness’. (In) accurate awareness of one’s own physical
activity level (e.g., in relation to meeting physical activity
recommendations) may result in socially desirable responses to the self-report physical activity surveys, and
this may undermine the validity of the measure [41].
Overlap: The time frame of the Active Australia Survey (last 7 days) overlapped as much as possible with the
time when participants were wearing the ActiGraph;
however, it was not possible to always have a perfect
Vandelanotte et al. BMC Public Health
(2019) 19:407
overlap. As such, ‘optimal overlap’ was considered when
there was no more than 2 days of difference between the
last day of ActiGraph monitoring and the time the Active Australia Survey was completed (thus 0-, 1- or
2-day gap). ‘Sub-optimal overlap’ was considered when
there was a gap of 3 or more days between ActiGraph
and Active Australia administration.
Data reduction
The ActiGraph data were reduced with custom software
(a Microsoft Excel macro) that examined each recorded
epoch and determined the intensity of physical activity
using the number of ActiGraph counts recorded during
that epoch. Valid ActiGraph wear time was determined
as at least 600 min wear time per day (during waking
hours) on a minimum of 5 of the 7 recorded days [28,
29]. Triaxial data were collected in 1-s epochs and aggregated to 60 s. Using the Freedson et al. cut points
[42], between 1953 and 5724 counts per minute was
classified as moderate physical activity and 5725 or more
counts was classified as vigorous physical activity. As
such, the total minutes of moderate physical activity, vigorous physical activity, and total moderate and vigorous
physical activity were calculated for each day with valid
ActiGraph data. The ActiGraph data were checked for
outliers, though none where identified. More detailed information about data reduction processes can be found
in the study protocol paper [28]. Total moderate and
vigorous physical activity was dichotomised in alignment
of achieving the minimum recommendation of 150 min
of physical activity per week [43]. This dichotomized
variable was used to stratify participants into two subgroups; however, the continuous variables described
above were used for the correlational analyses.
Analyses
Descriptive statistics (χ2 and t-tests) were used to compare participants with valid ActiGraph data to those
without valid ActiGraph data for baseline demographics,
as well to compare baseline characteristics for participants who had remained in the study at the 3-month
time point and those that had dropped out. McNemar
tests were conducted to assess whether there were significant differences in the proportion of people classified
as meeting or not meeting physical activity guidelines
for the 2 separate measures (Active Australia Survey and
Actigraph accelerometer). To assess validity, Spearman
rank-order correlation coefficients were computed to assess the relationship between the Active Australia Survey
outcomes with the ActiGraph outcomes for participants
with complete data for both measures (i.e., complete
cases analysis). Spearman correlation coefficients were
chosen because self-reported physical activity data were
not normally or linearly distributed, however the
Page 4 of 11
monotonicity assumption was not violated. The use of
Spearman correlations is well accepted and common for
assessing the validity of physical activity surveys, including the Active Australia Survey, and is valuable when
comparing the results to other studies [7, 9–11, 34, 35].
All reported correlations are between corresponding
physical activity categories (e.g., ActiGraph vigorous
physical activity was correlated with Active Australia vigorous physical activity). To assess responsiveness to
change over time, a linear regression model was used to
regress the 3-month Active Australia Survey outcomes
onto the baseline Active Australia Survey outcomes for
each of the 3 variables (moderate, vigorous and moderate + vigorous physical activity); in doing so, the individual residual scores were calculated for each participant.
The same procedure was repeated for the 3-month and
baseline ActiGraph variables. Finally, Spearman
rank-order correlation coefficients were computed between the individual residual scores from the Active
Australia Survey and those from the ActiGraph for the 3
variables. This procedure allows reducing measurement
error to a larger extent when compared to directly correlating change scores [44, 45]. Due to the large sample
size it was possible to stratify the outcomes and assess
whether the correlation coefficients differed for several
outcomes (e.g. age). Fisher r to Z transformations (z)
were applied to assess whether there were significant differences in the correlations between the subgroups [46].
Statistical significance was set at an alpha level of 0.05.
Results
A total of 504 participants were randomized into the
study and 465 had valid ActiGraph data (a minimum of
600 min of wear time on 5 out of 7 days) at the baseline
of the RCT. There were no significant differences between participants with or without valid ActiGraph data
for all baseline demographics, with the exception of educational attainment: more participants with a higher
education had no valid data (χ2 = 7.22, p = 0.02). There
were no significant differences between participants who
remained into the study at 3 months and those who had
dropped out for all baseline demographics, with the exception of age: more participants with a younger age had
dropped out at 3 months (t = 3.21; p = 0.001). As Table 1
shows, nearly two thirds of participants were female
(65.1%), and about three quarters of the sample was either overweight (35.9%) or obese (39.7%). Participants of
different educational levels and ages were well represented, though a high number of participants were aged
between 50 and 64 years (39.7%). The majority of the
sample reported high physical activity awareness
(63.5%); and optimal overlap between the ActiGraph
measurement and the Active Australia Survey measurement was achieved in 55.1%. At baseline, similar
Vandelanotte et al. BMC Public Health
Page 5 of 11
(2019) 19:407
Table 1 Participant demographic characteristics and physical activity levels
N (%)
Total group
Men
Women
Male
176 (34.9)
–
–
Female
328 (65.1)
–
–
18 to 49 years of age
162 (32.1)
41 (23.3)
121 (36.9)
50 to 64 years of age
200 (39.7)
71 (40.3)
129 (39.3)
Over 65 years of age
142 (28.1)
64 (36.4)
78 (23.8)
School education
140 (27.8)
39 (22.2)
101 (30.8)
Vocational and technical education
193 (38.3)
83 (47.2)
110 (33.5)
Higher education
171 (33.9)
54 (30.7)
117 (35.7)
Healthy weight
123 (24.4)
31 (17.6)
92 (28.0)
Overweight
181 (35.9)
76 (43.2)
105 (32.0)
Obese
200 (39.7)
69 (39.2)
131 (39.9)
High physical activity awareness
320 (63.5)
102 (58.0)
218 (66.5)
Low physical activity awareness
184 (36.5)
74 (42.0)
110 (33.5)
Optimal overlap AAS & ActiGraph
256 (55.1)
90 (55.6)
166 (54.8)
Sub-optimal overlap AAS & ActiGraph
209 (44.9)
72 (44.4)
137 (45.2)
283 (56.2)
93 (52.8)
190 (57.9)
Active Australia Survey:
= < 150 min MVPA week – baseline
> 150 min MVPA week – baseline
221 (43.8)
83 (47.2)
138 (42.1)
= < 150 min MVPA week – 3 months
153 (39.0)
49 (34.0)
104 (41.9)
> 150 min MVPA week – 3 months
239 (61.0)
95 (66.0)
144 (58.1)
ActiGraph Accelerometer:
= < 150 min MVPA week – baseline
257 (55.3)
72 (44.4)
185 (61.1)
> 150 min MVPA week – baseline
208 (44.7)
90 (55.6)
118 (38.9)
= < 150 min MVPA week – 3 months
177 (47.3)
43 (31.6)
134 (56.3)
> 150 min MVPA week – 3 months
197 (52.7)
93 (68.4)
104 (43.7)
Total group (N = 504)
Men (n = 176)
Women (n = 328)
Mean ± SD
Baseline Active Australia Survey
Moderate PA (min/week)
163 ± 190
187 ± 218
150 ± 172
Vigorous PA (min/week)
46 ± 93
65 ± 122
36 ± 72
MVPA (min/week)
210 ± 243
253 ± 286
186 ± 213
Three month Active Australia Survey
Moderate PA (min/week)
238 ± 238
284 ± 281
211 ± 205
Vigorous PA (min/week)
59 ± 110
73 ± 136
51 ± 91
MVPA (min/week)
297 ± 296
358 ± 369
262 ± 238
Baseline ActiGraph
Moderate PA (min/week)
163 ± 122
190 ± 119
148 ± 122
Vigorous PA (min/week)
4.4 ± 19.6
4.6 ± 19.2
4.4 ± 19.9
MVPA (min/week)
167 ± 127
195 ± 124
153 ± 127
Three month ActiGraph
Moderate PA (min/week)
187 ± 140
226 ± 152
164 ± 127
Vigorous PA (min/week)
8.1 ± 40.2
9.8 ± 49.5
7.3 ± 33.8
MVPA (min/week)
195 ± 151
236 ± 167
171 ± 136
Note: AAS Active Australia Survey, MVPA Moderate + Vigorous Physical Activity, PA Physical Activity, SD Standard Deviation
Vandelanotte et al. BMC Public Health
(2019) 19:407
proportions of participants engaged in 150 min of moderate to vigorous physical activity according to the Active Australia Survey (43.8%) and ActiGraph (44.7%);
these proportions were not significantly different
(McNemar test = 0.00; p = 1.00). However, at 3 months
there was a larger gap between the two assessments
(61% for Active Australia Survey and 52.7% for ActiGraph), and these differences were significantly different
(McNemar test = 8.37; p = 0.004). The increase in moderate to vigorous physical activity from baseline to 3
months was 87 min per week according to the Active
Australia Survey and 28 min per week according to the
Actigraph; the difference in change over time between
the two measures was significant (t = 3.16; p = 0.002).
While nearly all correlation coefficients assessing the
validity between the Active Australia Survey and the
ActiGraph were significant, they were generally small
(see Table 2). For the total group, a correlation of ρ = 0.19
(p = 0.000; CI 95% = 0.13–0.32) was found for moderate
physical activity, ρ = 0.33 (p = 0.000; CI 95% = 0.11–0.29)
for vigorous physical activity and ρ = 0.23 (p = 0.000; CI
95% = 0.24–0.45) for moderate and vigorous physical activity combined. This general pattern, whereby the correlations for vigorous activity were higher than for the other
physical activity categories, was relatively similar when the
data were stratified according to different subgroups (see
Table 2). Few significant differences between subgroups
were observed. There were significant differences in the
correlations for vigorous physical activity between men
and women (z = 2.01; p = 0.04), as well as between participants aged from 50 to 64 years and those aged over 65
(z = 2.04; p = 0.04). There was a significant difference in
the correlation for moderate intensity physical activity between those of a healthy weight and those who were overweight (z = 2.04; p = 0.04).
Compared to the validity correlations, the correlations
expressing responsiveness to change were somewhat higher,
though still relatively small (see Table 3). For the total
group, a correlation of ρ = 0.35 (p = 0.000; CI 95% = 0.25–
0.45) was found for moderate physical activity, ρ = 0.32
(p = 0.001; CI 95% = 0.22–0.43) for vigorous physical activity and ρ = 0.19 (p = 0.000; CI 95% = 0.07–0.30) for moderate and vigorous physical activity combined. This general
pattern, whereby the correlations for vigorous activity were
lower than for the other physical activity categories, was
relatively similar when the data were stratified according to
different subgroups (see Table 3). No significant differences
between subgroups were observed.
Discussion
The aim of this study was to investigate the validity of
the Active Australia Survey stratified for different population subgroups, and to examine its responsiveness to
change over time. Overall, the results of this study
Page 6 of 11
provide little evidence for the validity of the Active
Australia Survey. The correlation coefficients in this
study are lower than 0.4, which is considered as poor by
Helmerhorst et al. (2012) [23]. Furthermore, they are,
for most variables, also lower than 0.3, which has been reported as the lower limit for demonstrating acceptable evidence of validity for self-report physical activity measures
[17]. The present results are in contrast to most other Active Australia Survey validation studies using accelerometers, as they reported correlation coefficients for total
physical activity ranging from 0.42 to 0.61 [10, 11]. Only 2
studies conducted by Timperio et al. reported correlations
below 0.3 [7, 47]. One possible explanation for the contrasting findings could be that the Active Australia Survey
was administered differently across studies (e.g., telephone
vs. paper-and-pencil administration), however previous
studies have found similar correlations irrespective of the
administration method [10, 11]. It is noteworthy to point
out that all the studies that found acceptable validity levels
had smaller samples (range: 44–76), whereas the present
study (n = 465) and those of Timperio (n = 122 and 191)
had considerably more participants [7, 47]. When comparing the validity to other physical activity questionnaires,
the outcomes of the present study are in line with those of
the systematic review of Helmerhorst et al. [23]; median
Spearman correlation coefficients for surveys assessed in
adults ranged from 0.27 to 0.30 for ‘older’ and ‘newer’
physical activity surveys respectively. Those authors concluded that it appears almost impossible to obtain a valid
estimation of a highly variable behavior such as physical
activity by self-report [23].
The present study found somewhat higher correlation
coefficients in women compared to men (only significant
for vigorous physical activity); and while two previous
studies demonstrated acceptable validity in women using
the Active Australia Survey, they did not compare these
outcomes with men [10, 34]. However, the study by
Timperio et al. found lower correlations for women
compared to men [7]. These differences may be due to
gender-based differences in the perception of intensity
or gender-based differences in recall or attention to detail [48]. The present study found the lowest correlations
between the two measures for those with the highest age
(only significant for vigorous physical activity). This is in
contrast to a study that found acceptable validity (ρ =
0.42) in participants over the age of 65 [9]. Unfortunately their study did not include younger age groups.
Cognitive degeneration has been suggested as a reason
why accurate physical activity recall may decline in old
age [23]. Alternatively, the lower correlations in older
participants may be due to changes in the perception of
physical activity intensity, whereby activities of moderate
intensity may be perceived as vigorous by some, but not
by others. No other studies have compared correlations
Vandelanotte et al. BMC Public Health
Page 7 of 11
(2019) 19:407
Table 2 Spearman Rank Correlations between baseline measures for the Active Australia Survey and the ActiGraph Accelerometer
Total group (N = 465)
Men (n = 162)
Women (n = 303)
18 to 49 years of age (n = 147)
50 to 64 years of age (n = 186)
Over 65 years of age (n = 132)
School education (n = 135)
Vocational and technical education (n = 179)
Moderate PA
Vigorous PA
MVPA
ρ (p)
ρ (p)
ρ (p)
(95% CI)
(95% CI)
(95% CI)
.199 (000)***
.331 (.000)***
.231 (.000)***
(.106–.290)
(.241–.415)
(.132–.320)
.153 (.053)
.218 (.005)**
.169 (.031)*
(−.012–.315)
(.069–.367)
(.013–.326)
.203 (.000)***
.396 (.000)***
.236 (.000)***
(.079–.313)
(.288–.504)
(.110–.348)
.228 (.006)**
.363 (.000)***
.263 (.001)**
(.054–.391)
(.202–.519)
(.074–.426)
.235 (.001)**
.412 (.000)***
.281 (.000)***
(.102–.384)
(.274–.537)
(.146–.429)
.198 (.023)*
.201 (.021)*
.214 (.014)*
(.024–.366)
(.016–.380)
(.048–.375)
.297 (.000)***
.286 (.001)**
.287 (.001)**
(.119–.460)
(.116–.444)
(.111–.450)
.203 (.006)**
.312 (.000)***
.216 (.004)**
(.050–.360)
(.176–.448)
(.056–.364)
Higher education (n = 151)
.084 (.302)
.393 (.000)***
.191 (.019)*
(−.090–.256)
(.230–.537)
(.027–.355)
Healthy weight (n = 114)
.344 (.000)***
.307 (.001)**
.361 (.000)***
(.164–.515)
(.134–.475)
(.181–.528)
.108 (.165)
.286 (.000)***
.165 (.032)*
(−.059–.274)
(.151–.429)
(−.003–.328)
.183 (.013)*
.384 (.000)***
.192 (.000)***
(.046–.312)
(.240–.523)
(.052–.324)
.205 (.000)***
.288 (.000)***
.217 (.000)***
(.092–.304)
(.170–.397)
(.103–.321)
.181 (.019)*
.377 (.000)***
.234 (.002)**
(.021–.340)
(.230–.509)
(.087–.393)
.208 (.001)**
.394 (.000)***
.256 (.000)***
(.084–.331)
(.266–.502)
(.140–.375)
.182 (.008)**
.258 (.000)***
.196 (.004)**
(.044–.316)
(.119–.393)
(.054–.332)
.144 (.021)*
.379 (.000)***
.165 (.008)**
(.017–.262)
(.258–.481)
(.042–.284)
.228 (.001)**
(.096–.355)
.283 (.000)***
(.142–.416)
.247 (.000)***
(.106–.370)
Overweight (n = 168)
Obese (n = 183)
High physical activity awareness (n = 297)
Low physical activity awareness (n = 168)
Optimal overlap AAS & ActiGraph (n = 256)
Sub-optimal overlap AAS & ActiGraph (n = 209)
= < 150 min MVPA week ActiGraph (n = 257)
> 150 min MVPA week ActiGraph (n = 208)
Note: p < 0.05 = *; p < 0.01 = **; p < 0.001 = ***
AAS Active Australia Survey, MVPA Moderate + Vigorous Physical Activity, PA Physical Activity
for those with different education levels, and the outcomes of this study suggest that having a higher education does not necessarily reflect better behavioral recall,
as correlations were often higher for those with a lower
education; moreover, the differences between all age
groups were not significant. Counterintuitive outcomes
were found for the level of physical activity awareness,
as lower physical activity awareness often resulted in
higher validity scores (though these differences were not
significant). Perhaps a lack of awareness results in lower
Vandelanotte et al. BMC Public Health
Page 8 of 11
(2019) 19:407
Table 3 Spearman Rank Correlations of residual scores expressing change between baseline and 3 months
Total group (n = 333)
Men (n = 123)
Women (n = 210)
18 to 49 years of age (n = 95)
50 to 64 years of age (n = 134)
Over 65 years of age (n = 104)
School education (n = 90)
Vocational and technical education (n = 132)
Higher education (n = 111)
Normal weight (n = 78)
Overweight (n = 128)
Obese (n = 127)
High physical activity awareness (n = 217)
Low physical activity awareness (n = 116)
= < 150 min MVPA week ActiGraph (n = 183)
> 150 min MVPA week ActiGraph (n = 150)
Moderate
Vigorous
MVPA
ρ (p)
ρ (p)
ρ (p)
(95% CI)
(95% CI)
(95% CI)
.325 (.000)***
.189 (.001)**
.356 (.000)***
(.221–.431)
(.074–.304)
(.253–.454)
.413 (.000)***
.213 (.018)*
.435 (.000)***
(.247–.558)
(.019–.396)
(.262–.573)
.251 (.000)***
.174 (.012)*
.298 (.000)***
(.119–.383)
(.037–.325)
(.160–.433)
.286 (.005)**
.195 (.058)
.366 (.000)***
(.095–.462)
(−.034–.405)
(.178–.526)
.355 (.000)***
.123 (.156)
.359 (.000)***
(.194–.506)
(−.069–.303)
(.196–.509)
.323 (.001)**
.308 (.001)**
.336 (.000)***
(.120–.495)
(.128–.474)
(.130–.514)
.284 (.007)**
.220 (.038)*
.282 (.007)**
(.085–.477)
(.003–.420)
(.077–.476)
.387 (.000)***
.230 (.008)**
.457 (.000)***
(.236–.522)
(.031–.409)
(.320–.579)
.307 (.001)**
.133 (.165)
.343 (.000)***
(.113–.484)
(−.074–.347)
(.150–.518)
.421 (.000)***
.258 (.023)*
.477 (.000)***
(.200–.599)
(.036–.470)
(.288–.658)
.217 (.014)*
.244 (.005)**
.278 (.001)**
(.036–.379)
(.056–.418)
(.098–.445)
.362 (.000)***
.070 (.435)
.346 (.000)***
(.183–.524)
(−.142–.272)
(.149–.515)
.324 (.000)***
.160 (.018)*
.361 (.000)***
(.194–.450)
(.008–.309)
(.234–.484)
.310 (.001)**
.221 (.017)*
.317 (.001)**
(.134–.474)
(.014–.400)
(.130–.489)
.279 (.000)***
.172 (.020)*
.321 (.000)***
(.128–.433)
(.007–.343)
(.173–.473)
.382 (.000)***
.237 (.004)**
.415 (.000)***
(.230–.522)
(.079–.393)
(.261–.554)
Note: p < 0.05 = *; p < 0.01 = **; p < 0.001 = ***
MVPA Moderate + Vigorous Physical Activity, PA Physical Activity
social desirability bias. With the exception of vigorous
physical activity in obese participants, the correlations
were lower for those with higher weight (the differences
were significant for moderate intensity physical activity).
The study by Timperio also examined validity levels according to weight status [7], and found a high level of variability across multiple categories that do not align with the
variables of the present study, making between-study comparisons difficult. Fjedlsoe et al. indicated that the validity
of the Active Australia Survey decreases when participants
are more active [11]. The findings of our study are in line
with those of Fjedlsoe et al., but only for vigorous physical
activity and the differences were not significant [11]. Fjeldsoe et al. indicate that a widening in measurement error
and bias may be responsible for the lower validity in highly
active participants [11]. Finally, it is not surprising to find
somewhat higher correlations when both measures cover
the same measurement period, though the differences were
small, not significant and almost negligible when compared
to the correlations of the total group.
Vandelanotte et al. BMC Public Health
Page 9 of 11
(2019) 19:407
In broad-reaching physical activity interventions, where
modest (but clinically meaningful) changes in behavior are
often observed, the responsiveness of self-report measures
to detect such changes is critical [25]. The correlations expressing responsiveness to change over time were generally low, although they were somewhat higher than the
validation correlations and, as Table 3 shows, for some
categories they were higher than 0.4, which indicates a degree of acceptability [23]. For example, correlations higher
than 0.4 were observed in men, healthy weight participants, those with vocational or technical education, and
those who engage in more than 150 min of moderate and
vigorous physical activity according to the ActiGraph for
moderate to vigorous physical activity. To our knowledge,
only two studies have attempted to examine the responsiveness to change for the Active Australia Survey. Reeves et al.
found good responsiveness to change for moderate to vigorous physical activity relative to a more detailed self-report
measure (CHAMPS) [25]. In their study (n = 381) the responsiveness index (based on Tuley’s formulae) of the Active Australia Survey was 0.50 (95%CI: 0.30–0.69) which
was considered as good responsiveness. Lee et al. used the
same methodology (i.e., responsiveness index based on
Tuley’s formulae) and found a similar responsiveness for
the Active Australia Survey (0.45; 95%CI: 0.26–0.65), although it was somewhat lower than the responsiveness for
the Actigraph in the same study (0.49; 95%CI: 0.23–0.74)
[26]. Given the scarcity of studies assessing responsiveness
to change, however, further research is required to confirm
these findings.
The large study sample, which allowed stratifying the
outcomes for specific subgroup populations, examining
responsiveness to change, and the robust study protocol
were strengths of this study. However, those who participated in this study were part of a convenience sample
recruited to participate in a randomized controlled trial.
This may have introduced bias, limiting generalisability
of the findings. It should be pointed out, however, that
the study sample was well balanced in terms of gender,
age, education and weight status. Caperchione et al. provide an in-depth description of the sample of this study
and how it compares with the general Australian population [49]. Correlation coefficients can be affected when
floor or ceiling effects are present (when more than 15%
of the sample reports the highest or lowest possible
score) [50]. As such, it is a limitation that floor effects
were observed for the vigorous physical activity variables. However, no other floor or ceiling effects were observed for any other variables. Another limitation is that
‘optimal overlap’ for the Active Australia Survey and the
ActiGraph measurement was not achieved for all participants. Other Active Australia Validation studies have
also reported to this problem [10, 34]. As discussed earlier, this only had a small influence on the observed
correlations. To make sure, however, we did run the
analyses stratified for all the specific population subgroups with only those participants demonstrating ‘good
overlap’. The differences in correlations with the currently presented outcomes were minimal, not warranting
the large drop in sample size, ensuring each cell had a
large number of participants. Finally, while the ActiGraph is acceptable and often used to assess validity of
self-report measures, it is not a gold standard and not
able to measure all types of physical activity accurately,
this may have reduced the observed correlations [24].
Furthermore, the error associated with regression equations used to derive cut-points for moderate and
vigorous-intensity physical activity is also limitation of
using accelerometers [4, 6].
Conclusions
This study provided little evidence for the validity of the
Active Australia Survey, although the responsiveness to
change was marginally better and deemed acceptable for
a number of specific subgroups. The findings are largely
in contrast to other Active Australia Survey validation
studies with smaller study samples; however they are in
line with studies with larger samples sizes, and a review
that assessed a range of different physical activity measures. Despite its practicality and low cost, findings from
studies that use the Active Australia Survey should always be interpreted with a degree of caution.
Abbreviation
BMI: Body Mass Index
Acknowledgments
The authors would like to thank the Population Research Laboratory of the
Central Queensland University for their help with collecting the data. The
work in this publication has been presented at a conference, resulting in a
published conference abstract [51].
Funding
This trial was funded by the National Health and Medical Research Council
(Project Grant number 589903). CV (ID 100427) is supported by a Future
Leader Fellowship from the National Heart Foundation of Australia. MJD (ID
100029) was supported by a Future Leader Fellowship from the National
Heart Foundation of Australia and is now supported by a Career
Development Fellowship (APP1141606) from the National Health and
Medical Research Council. ALR is supported by a post-doctoral research fellowship from the National Health and Medical Research Council of Australia
(ID 1105926). The funder did not have any role in the study other than to
provide funding.
Availability of data and materials
The data will be available upon request from the lead investigator of the
WALK 2.0 RCT (GSK) when all study outcomes have been analysed and
published.
Authors’ contributions
KM, CV, MJD, CMC and GSK, conceived the project and procured the project
funding. GSK led the coordination of the trial. KM, GSK, RRR, CV, MJD, and
CMC assisted with the protocol design. TNS managed the trial including data
collection. CV, RS, and ALR interpreted the data and drafted the manuscript.
All authors read, edited, and approved the final manuscript.
Vandelanotte et al. BMC Public Health
(2019) 19:407
Ethics approval and consent to participate
All participants provided written informed consent to participate in the
study. The study was approved by the Human Research Ethics Committees
of Western Sydney University (H8767) and CQUniversity (H11/01–005).
Consent for publication
Not applicable.
Competing interests
Author RRR is a Section Editor on the Editorial Board of BMC Public Health;
however, he had no involvement or influence with regards to the review
process for this publication. The other authors declare they have no
competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
Author details
1
Physical Activity Research Group, School of Human, Health and Social
Sciences, Central Queensland University, Rockhampton, QLD, Australia.
2
School of Medicine & Public Health; Priority Research Centre for Physical
Activity and Nutrition, Faculty of Health and Medicine, University of
Newcastle, Newcastle, NSW, Australia. 3School of Medical and Applied
Sciences, Central Queensland University, Rockhampton, QLD, Australia.
4
Department of Food, Nutrition, Dietetics and Health, Kansas State University,
Manhattan, KS, USA. 5School of Health and Exercise Sciences, University of
Technology Sydney, Sydney, NSW, Australia. 6School of Science and Health,
Western Sydney University, Sydney, NSW, Australia. 7Faculty of Physical
Education and Recreation, University of Alberta, Edmonton, Alberta, Canada.
Received: 6 September 2018 Accepted: 27 March 2019
References
1. Mathers C, Vos T, Stevenson C. The burden of disease and injury in
Australia. A/HW catalogue no. PHE 17. Canberra: Australian Institute of
Health and Welfare; 1999.
2. Haskell WL, Lee IM, Pale RR, et al. Physical activity and public health:
updated recommendation for adults from the American College of Sports
Medicine and the American Heart Association. Med Sci Sports Exerc. 2007;
39(8):1423–34.
3. Kriska AM, Casperson CJ. Introduction to a collection of physical activity
questionnaires. Med Sci Sports Exerc. 1997;29:S5–9.
4. Lagerros YT, Lagiou P. Assessment of physical activity and energy
expenditure in epidemiological research of chronic diseases. Eur J
Epidemiol. 2007;22:353–62.
5. Slootmaker SM, Schuit AJ, Chinapaw MJ, et al. Disagreement in physical
activity assessed by accelerometer and self-report in subgroups of age,
gender, education and weight status. Int J Behav Nutr Phys Act. 2009;6:17.
6. Troiano RP, McClain JJ, Brychta RJ, et al. Evolution of accelerometer
methods for physical activity research. Br J Sports Med. 2014;48:1019–23.
7. Timperio A, Salmon J, Crawford D. Validity and reliability of a physical
activity recall instrument among overweight and non-overweight men and
women. J Sci Med Sport. 2003;6(4):477–91.
8. Duncan GE, Sydeman SJ, PelTi MG, et al. Can sedentary adults accurately
recall the intensity of their physical activity? Prev Med. 2001;33:18–26.
9. Heesch KC, Hill RL, van Uffelen JGZ, et al. Are active Australia physical
activity questions valid for older adults? J Sci Med Sport. 2011;14(3):233–7.
10. Brown WJ, Burton NW, Marshall AL, et al. Reliability and validity of a
modified self-administered version of the active Australia physical activity
survey in a sample of mid-age women. Aust NZ J Public Health. 2008;32:
535–41.
11. Fjeldsoe BS, Winkler EAH, Marshall AL, et al. Active adults recall their
physical activity differently to less active adults: test–retest reliability and
validity of a physical activity survey. Health Prom J Aust. 2013;24:26–31.
12. Australian Institute of Health and Welfare. The active Australia survey: a
guide and manual for implementation, analysis and reporting. Canberra:
AIHW; 2004.
Page 10 of 11
13. Armstrong T, Bauman A, Davies J. Physical Activity Patterns of Australian
Adults. Results of the 1999 National Physical Activity Survey. Canberra:
Australian Institute of Health and Welfare; 2000.
14. Vandelanotte C, Duncan MJ, Caperchione C, et al. Physical activity trends in
Queensland (2002 to 2008): are women becoming more active then men?
Aust NZ J Public Health. 2010;34(3):248–54.
15. Guedes RC, Dias RC, Pereira LS, et al. Influence of dual task and frailty on
gait parameters of older community-dwelling individuals. Brazilian J Physical
Therapy. 2014;18(5):445–52.
16. de Menezes Caceres V, Stocks N, Adams R, et al. Physical activity moderates
the deleterious relationship between cardiovascular disease, or its risk
factors, and quality of life: findings from two population-based cohort
studies in southern Brazil and South Australia. PLoS One. 2018;13(6):
e0198769.
17. Craig CL, Marshall AL, Sjostrom M, et al. International physical activity
questionnaire: 12-country reliability and validity. Med Sci Sports Exerc. 2003;
35(8):1381–95.
18. Simpson K, Parker B, Capizzi J, et al. Validity and reliability of question 8 of
the paffenbarger physical activity questionnaire among healthy adults. J
Phys Activ Health. 2015;12:116–23.
19. Puig-Ribera A, Martin-Cantera C, Puigdomenech E, et al. Screening physical
activity in family practice: validity of the Spanish version of a brief physical
activity questionnaire. PLoS One. 2015;10:e0136870.
20. Bae WK, Cho B, Son KY. Validity and reliability of the Korean version of
neighborhood physical activity questionnaire. Kor J Fam Med. 2015;36:135–
40.
21. Hardie Murphy M, Rowe DA, Belton S, Woods CB. Validity of a two-item
physical activity questionnaire for assessing attainment of physical activity
guidelines in youth. BMC Pub Health. 2015;15:1080.
22. Ekelund U, Sepp H, Brage S, et al. Criterion-related validity of the last 7-days,
short from of the international physical activity questionnaire in Swedish
adults. Pub Health Nutr. 2006;9:258–65.
23. Helmerhorst HJF, Brage S, Warren J, et al. A systematic review of reliability
and objective criterion related validity of physical activity questionnaires. Int
J Behav Nutr Phys Activ. 2012;9:103.
24. Alley S, Jennings C, Plotnikoff RC, et al. My activity coach – using videocoaching to assist a web-based computer-tailored physical activity
intervention: a randomised controlled trial protocol. BMC Pub Health. 2014;
14:738.
25. Reeves MR, Marshall AL, Owen N, et al. Measuring physical activity change
in broad-reach intervention trials. J Phys Activ Health. 2010;7:194–202.
26. Lee WYH, Clark BK, Winkler AE, et al. Responsiveness to change of selfreport and device-based physical activity measures in the living well with
diabetes trial. J Phys Act Health. 2015;12(8):1082–7.
27. Kelly P, Fitzsimons C, Baker G. Should we reframe how we think about
physical activity and sedentary behaviour measurement? Validity and
reliability reconsidered. Int J Behav Nutr and Phys Activ. 2016;13:32.
28. Kolt GS, Rosenkranz RR, Savage TN, et al. WALK 2.0 – using web 2.0
applications to promote health-related physical activity: a randomised
controlled trial protocol. BMC Pub Health. 2013;13:436.
29. Kolt GS, Rosenkranz RR, Vandelanotte C, et al. Using web 2.0 applications to
promote health-related physical activity: findings from the WALK 2.0
randomised controlled trial. Br J Sports Med. 2017;51:1433–40.
30. Elley C, Kerse N, Arroll B, Robinson E. Effectiveness of counselling patients
on physical activity in general practice: cluster randomised controlled trial.
Brit Med J. 2003;362:793–8.
31. Vandelanotte C, Stanton R, Rebar A, Van Itallie A, et al. Physical activity
screening to recruit inactive randomised controlled trial participants: how
much is too much? Trials. 2015;16:446.
32. De Moura BP, Marins JCB, Amorim PRS. Self selected walking speed in
overweight adults: is this intensity enough to promote health benefits?
Apunts Med Esport. 2011;46(169):11–5.
33. Braham R, Rosenberg M, Begley B. Can we teach moderate intensity
activity? Adult perception of moderate intensity walking. J Sci Med Sport.
2012;15(4):322–6.
34. Gabriel PK, McClain JJ, Lee CD, et al. Evaluation of physical activity measures
used in middle-aged women. Med Sci Sports Exerc. 2009;41(7):1403–12.
35. Freene N, Waddington G, Chesworth W, et al. Validating two self-report
physical activity measures in middle-aged adults completing a group
exercise or home-based physical activity program. J Sci Med Sport. 2014;
17(6):611–6.
Vandelanotte et al. BMC Public Health
(2019) 19:407
36. Sasaki JE, John D, Freedson PS. Validation and comparison of ActiGraph
activity monitors. J Sci Med Sport. 2011;14(5):411–6.
37. Plasqui G, Bonomi A, Westerterp K. Daily physical activity assessment with
accelerometers: new insights and validation studies. Obes Rev. 2013;14(6):
451–62.
38. Kelly LA, McMillan D, Anderson A, Fippinger M, et al. Validity of actigraphs
uniaxial and triaxial accelerometers for assessment of physical activity in
adults in laboratory conditions. BMC Med Phys. 2013;13:5.
39. World Health Organization. Preventing and managing the global epidemic.
Report of a WHO Consultation. WHO technical report series 894. Geneva:
World Health Organization; 2000.
40. Schnohr C, Hojbierre L, Riegels M, et al. Does educational level influence the
effects of smoking, alcohol, physical activity and obesity on mortality? A
prospective population study. Scand J Public Health. 2004;32(4):250–6.
41. Adams S, Matthews C, Ebbeling C, et al. The effect of social desirability and
social approval on self-reports of physical activity. Am J Epidemiol. 2005;
161(4):389–98.
42. Freedson PS, Melanson E, Sirard J. Calibration of the computer science and
applications, Inc. accelerometer. Med Sci Sports Exerc. 1998;30(5):777–81.
43. Australian Government Department of Health and Ageing. Australia’s
physical activity and sedentary behaviour guidelines for adults. Canberra;
2014. Available at: http://www.health.gov.au/internet/main/publishing.nsf/
content/health-pubhlth-strateg-phys-act-guidelines#apaadult (Accessed 14
Dec 2015).
44. Gollwitzer M, Christ O, Lemmer G. Individual differences make a difference:
on the use and the psychometric properties of difference scores in social
psychology. Eur J Soc Psychol. 2014;44(7):673–82.
45. Cronbach LJ, Furby L. How we should measure ‘change’: or should we?
Psychol Bull. 1970;74(1):68–80.
46. Weaver B, Wuensch KL. SPSS and SAS programs for comparing Pearons
correlations and OLS regression coefficients. Behav Res Methods. 2013;45(3):
880–95.
47. Timperio A, Salmon J, Rosenberg M, et al. Do logbooks influence recall of
physical activity in validations studies? Med Sci Sports Exerc. 2004;36(7):
1181–6.
48. Kaushanskaya M, Marian V, Yoo J. Gender differences in adult word learning.
Acta Psychol. 2011;137(1):24–35.
49. Caperchione CM, Duncan MJ, Rosenkranz RR, et al. Recruitment, screening,
and baseline participant characteristics in the WALK 2.0 study: a randomized
controlled trial using web 2.0 applications to promote physical activity.
Contemp Clin Trials Commun. 2016;2:25–33.
50. Terwee CB, Mokkink LB, van Poppel MN, et al. Qualitative attributes and
measurement properties of physical activity questionnaires: a checklist.
Sport Med. 2010;40:525–37.
51. Vandelanotte C, Duncan MJ, Stanton R, Rosenkranz R, Rebar R, Savage T,
Mummery WK, Kolt G. Criterion validity and responsiveness to change of
the active Australia survey according to different subgroups. J Sci Med
Sport. 2017;20(Suppl1):e105.
Page 11 of 11