Monte Carlo methods were used to estimate the power of randomization tests used with single-case ... more Monte Carlo methods were used to estimate the power of randomization tests used with single-case designs involving the random assignment of treatments to phases. The design studied involved 2 treatments and 6 phases. The power was studied for 6 standardized effect sizes (0, .2, .5, .8, 1.1, and 1.4), 4 levels of autocorrelation (1st order autocorrelation coefficients of -.3, 0,
Monte Carlo methods were used to examine techniques for constructing confidence intervals around ... more Monte Carlo methods were used to examine techniques for constructing confidence intervals around multivariate effect sizes. Using interval inversion and bootstrapping methods, confidence intervals were constructed around the standard estimate of Mahalanobis distance (D2), two bias-adjusted estimates of D2, and Huberty’s I. Interval coverage and width were examined across conditions by adjusting sample size, number of variables, population effect size,
Researchers using mixed linear models often use fit criteria to select among possible covariance ... more Researchers using mixed linear models often use fit criteria to select among possible covariance structures for their data. Unfortunately, fit criteria do not always lead to the correct specification of the covariance structure, and misspecification can have negative consequences for estimation and inference. A program is presented that allows researchers to explore the sensitivity of Akaike's Information Criterion (AIC) and
This paper discusses a SAS® macro that provides three approaches to statistical inferences about ... more This paper discusses a SAS® macro that provides three approaches to statistical inferences about Mahalanobis distance. Mahalanobis distance is useful as a multivariate effect size, being an extension of the standardized mean difference (i.e., Cohen's d). This program calculates three point estimates of D2 (a sample estimate, a jackknife estimate, and an adjusted estimate advanced by Rao, 1973). Further, the
The purpose of this study was to investigate and compare the performance of a stepwise variable s... more The purpose of this study was to investigate and compare the performance of a stepwise variable selection algorithm to traditional exploratory factor analysis. The Monte Carlo study included six factors in the design; the number of common factors; the number of variables explained by the common factors; the magnitude of factor loadings; the number of variables not explained by the
Multilevel modeling has become a common analytic technique across a variety of disciplines includ... more Multilevel modeling has become a common analytic technique across a variety of disciplines including education and other social and behavioral sciences. Such models are often used when researchers examine relationships between school and/or neighborhood characteristics and some individual-level outcome (e.g., academic achievement, high school completion). Although many samples are theoretically cross-classified between multiple level-2 units, the application of cross-classified random
The quantitative methods for analyzing single-subject experimental data have expanded during the ... more The quantitative methods for analyzing single-subject experimental data have expanded during the last decade, including the use of regression models to statistically analyze the data, but still a lot of questions remain. One question is how to specify predictors in a regression model to account for the specifics of the design and estimate the effect size of interest. These quantitative effect sizes are used in retrospective analyses and allow synthesis of single-subject experimental study results which is informative for evidence-based decision making, research and theory building, and policy discussions. We discuss different design matrices that can be used for the most common single-subject experimental designs (SSEDs), namely, the multiple-baseline designs, reversal designs, and alternating treatment designs, and provide empirical illustrations. The purpose of this article is to guide single-subject experimental data analysts interested in analyzing and meta-analyzing SSED data.
While conducting intervention research, researchers and practitioners are often interested in how... more While conducting intervention research, researchers and practitioners are often interested in how the intervention functions not only at the group level, but also at the individual level. One way to examine individual treatment effects is through multiple-baseline studies analyzed with multilevel modeling. This analysis allows for the construction of confidence intervals, which are strongly recommended in the reporting guidelines of
Traditionally, average causal effects from multiple-baseline data are estimated by aggregating in... more Traditionally, average causal effects from multiple-baseline data are estimated by aggregating individual causal effect estimates obtained through within-series comparisons of treatment phase trajectories to baseline extrapolations. Concern that these estimates may be biased due to event effects, such as history and maturation, motivates our proposal of a between-series estimator that contrasts participants in the treatment to those in the baseline phase. Accuracy of the new method was assessed and compared in a series of simulation studies where participants were randomly assigned to intervention start points. The within-series estimator was found to have greater power to detect treatment effects but also to be biased due to event effects, leading to faulty causal inferences. The between-series estimator remained unbiased and controlled the Type I error rate independent of event effects. Because the between-series estimator is unbiased under different assumptions, the 2 estimates complement each other, and the difference between them can be used to detect inaccuracies in the modeling assumptions. The power to detect inaccuracies associated with event effects was found to depend on the size and type of event effect. We empirically illustrate the methods using a real data set and then discuss implications for researchers planning multiple-baseline studies.
The purpose of this study is to illustrate the multilevel meta-analysis of results from single-su... more The purpose of this study is to illustrate the multilevel meta-analysis of results from single-subject experimental designs of different types, including AB phase designs, multiple-baseline designs, ABAB reversal designs, and alternating treatment designs. Current methodological work on the meta-analysis of single-subject experimental designs often focuses on combining simple AB phase designs or multiple-baseline designs. We discuss the estimation of the average intervention effect estimate across different types of single-subject experimental designs using several multilevel meta-analytic models. We illustrate the different models using a reanalysis of a meta-analysis of single-subject experimental designs (Heyvaert, Saenen, Maes, & Onghena, in press). The intervention effect estimates using univariate 3-level models differ from those obtained using a multivariate 3-level model that takes the dependence between effect sizes into account. Because different results are obtained and the multivariate model has multiple advantages, including more information and smaller standard errors, we recommend researchers to use the multivariate multilevel model to meta-analyze studies that utilize different single-subject designs. (PsycINFO Database Record (c) 2015 APA, all rights reserved).
Journal of Occupational and Environmental Hygiene, 2015
Acclimation in a hot environment is one potent means to decrease the heat strain of work in a hot... more Acclimation in a hot environment is one potent means to decrease the heat strain of work in a hot environment. However, with diminished heat exposure, positive adaptations of acclimation may be lost. This rate of loss is equivocal and, once established, could be used to prescribe the time for re-acclimation. The purpose of this study was to determine the rate of loss of heat acclimation over a period of 6 weeks and determine the time needed for re-acclimation after 2 weeks and 4 weeks of de-acclimation in ten healthy participants. All participants first underwent an initial acclimation period (a 3-day plateau in Tre was used to signify acclimation). Based on the mean time to acclimate in Phase 1 (mean time to acclimate = 6.1±1.4 days), the loss of acclimation was mapped and participants were randomly assigned to one of two groups: one that underwent one 2-hr heat exposure at 1, 3, and 5 weeks post-acclimation, and one that underwent one 2-hr heat exposure session at 2,4, and 6 weeks. Complete loss of acclimation occurred in 6 weeks and, as expected, work HR and Tre increased with increasing time away from the heat (p<0.05). Based on the time for total loss of acclimation from Phase 1, participants in Phase 2 (n = 8) first underwent acclimation. Then, after either a 2-week or 4-week absence from the heat, participants returned to the laboratory for re-acclimation. While not statistically significant yet practically significant (p = 0.18; one-tailed confidence interval), average days for re-acclimation in the 2-week group tended to be fewer than in the 4-week group (days for re-acclimation = 3.8 ± 1.2 and 5.3 ± 1.9, respectively). Based on these general trends, for occupational settings, a re-acclimation period of 4 days is recommended after 2 weeks absence from the heat, 5 days for 4 weeks absence from the heat, and complete acclimation (6 days) after 6 weeks absence or more from the heat.
... LINDA M. RAFFAELE MENDEZ, HOWARD M. KNOFF, AND JOHN M. FERRON ... by gender andrace, (b) the ... more ... LINDA M. RAFFAELE MENDEZ, HOWARD M. KNOFF, AND JOHN M. FERRON ... by gender andrace, (b) the duplicated count of OSS by gender and race, and (c) the total number of each type of incident (eg, disruptive behavior) that resulted in OSS by gender and race. ...
Computer simulation methods were used to examine the sensitivity of model fit criteria to misspec... more Computer simulation methods were used to examine the sensitivity of model fit criteria to misspecification of the first-level error structure in two-level models of change, and then to examine the impact of misspecification on estimates of the variance parameters, estimates of the fixed effects, and tests of the fixed effects. Fit criteria frequently failed to identify the correct model when series lengths were short. Misspecification led to substantially biased estimates of variance parameters. The estimates of the fixed effects, however, remained unbiased for most conditions, and the tests of fixed effects were robust to misspecification for most conditions. The problems in the fixed effects occurred when nonlinear growth trajectories were coupled with data that were unequally spaced by different amounts for different individuals.
With the growing popularity of meta-analytic techniques to analyze and synthesize results across ... more With the growing popularity of meta-analytic techniques to analyze and synthesize results across sets of empirical studies, have come concerns about the sensitivity of traditional tests in meta-analysis to violations of assumptions. This is particularly distressing because the tenability of such assumptions in primary studies is often impossible to evaluate unless sufficient details are reported. Robust estimates of effect size, such as Cliff's δ , may yield superior inferences about the population effect size. The purpose of this study was to compare standardized mean differences (Cohen's d and Hedges' g) and δ in terms of the accuracy and precision of interval estimates of population mean effect size in meta-analysis. Factors investigated in the Monte Carlo study included characteristics of both the populations from which samples were drawn (distribution shape, variance heterogeneity, and population effect size) and the corpus of studies in each meta-analysis (sampl...
Monte Carlo methods were used to estimate the power of randomization tests used with single-case ... more Monte Carlo methods were used to estimate the power of randomization tests used with single-case designs involving the random assignment of treatments to phases. The design studied involved 2 treatments and 6 phases. The power was studied for 6 standardized effect sizes (0, .2, .5, .8, 1.1, and 1.4), 4 levels of autocorrelation (1st order autocorrelation coefficients of -.3, 0,
Monte Carlo methods were used to examine techniques for constructing confidence intervals around ... more Monte Carlo methods were used to examine techniques for constructing confidence intervals around multivariate effect sizes. Using interval inversion and bootstrapping methods, confidence intervals were constructed around the standard estimate of Mahalanobis distance (D2), two bias-adjusted estimates of D2, and Huberty’s I. Interval coverage and width were examined across conditions by adjusting sample size, number of variables, population effect size,
Researchers using mixed linear models often use fit criteria to select among possible covariance ... more Researchers using mixed linear models often use fit criteria to select among possible covariance structures for their data. Unfortunately, fit criteria do not always lead to the correct specification of the covariance structure, and misspecification can have negative consequences for estimation and inference. A program is presented that allows researchers to explore the sensitivity of Akaike's Information Criterion (AIC) and
This paper discusses a SAS® macro that provides three approaches to statistical inferences about ... more This paper discusses a SAS® macro that provides three approaches to statistical inferences about Mahalanobis distance. Mahalanobis distance is useful as a multivariate effect size, being an extension of the standardized mean difference (i.e., Cohen's d). This program calculates three point estimates of D2 (a sample estimate, a jackknife estimate, and an adjusted estimate advanced by Rao, 1973). Further, the
The purpose of this study was to investigate and compare the performance of a stepwise variable s... more The purpose of this study was to investigate and compare the performance of a stepwise variable selection algorithm to traditional exploratory factor analysis. The Monte Carlo study included six factors in the design; the number of common factors; the number of variables explained by the common factors; the magnitude of factor loadings; the number of variables not explained by the
Multilevel modeling has become a common analytic technique across a variety of disciplines includ... more Multilevel modeling has become a common analytic technique across a variety of disciplines including education and other social and behavioral sciences. Such models are often used when researchers examine relationships between school and/or neighborhood characteristics and some individual-level outcome (e.g., academic achievement, high school completion). Although many samples are theoretically cross-classified between multiple level-2 units, the application of cross-classified random
The quantitative methods for analyzing single-subject experimental data have expanded during the ... more The quantitative methods for analyzing single-subject experimental data have expanded during the last decade, including the use of regression models to statistically analyze the data, but still a lot of questions remain. One question is how to specify predictors in a regression model to account for the specifics of the design and estimate the effect size of interest. These quantitative effect sizes are used in retrospective analyses and allow synthesis of single-subject experimental study results which is informative for evidence-based decision making, research and theory building, and policy discussions. We discuss different design matrices that can be used for the most common single-subject experimental designs (SSEDs), namely, the multiple-baseline designs, reversal designs, and alternating treatment designs, and provide empirical illustrations. The purpose of this article is to guide single-subject experimental data analysts interested in analyzing and meta-analyzing SSED data.
While conducting intervention research, researchers and practitioners are often interested in how... more While conducting intervention research, researchers and practitioners are often interested in how the intervention functions not only at the group level, but also at the individual level. One way to examine individual treatment effects is through multiple-baseline studies analyzed with multilevel modeling. This analysis allows for the construction of confidence intervals, which are strongly recommended in the reporting guidelines of
Traditionally, average causal effects from multiple-baseline data are estimated by aggregating in... more Traditionally, average causal effects from multiple-baseline data are estimated by aggregating individual causal effect estimates obtained through within-series comparisons of treatment phase trajectories to baseline extrapolations. Concern that these estimates may be biased due to event effects, such as history and maturation, motivates our proposal of a between-series estimator that contrasts participants in the treatment to those in the baseline phase. Accuracy of the new method was assessed and compared in a series of simulation studies where participants were randomly assigned to intervention start points. The within-series estimator was found to have greater power to detect treatment effects but also to be biased due to event effects, leading to faulty causal inferences. The between-series estimator remained unbiased and controlled the Type I error rate independent of event effects. Because the between-series estimator is unbiased under different assumptions, the 2 estimates complement each other, and the difference between them can be used to detect inaccuracies in the modeling assumptions. The power to detect inaccuracies associated with event effects was found to depend on the size and type of event effect. We empirically illustrate the methods using a real data set and then discuss implications for researchers planning multiple-baseline studies.
The purpose of this study is to illustrate the multilevel meta-analysis of results from single-su... more The purpose of this study is to illustrate the multilevel meta-analysis of results from single-subject experimental designs of different types, including AB phase designs, multiple-baseline designs, ABAB reversal designs, and alternating treatment designs. Current methodological work on the meta-analysis of single-subject experimental designs often focuses on combining simple AB phase designs or multiple-baseline designs. We discuss the estimation of the average intervention effect estimate across different types of single-subject experimental designs using several multilevel meta-analytic models. We illustrate the different models using a reanalysis of a meta-analysis of single-subject experimental designs (Heyvaert, Saenen, Maes, & Onghena, in press). The intervention effect estimates using univariate 3-level models differ from those obtained using a multivariate 3-level model that takes the dependence between effect sizes into account. Because different results are obtained and the multivariate model has multiple advantages, including more information and smaller standard errors, we recommend researchers to use the multivariate multilevel model to meta-analyze studies that utilize different single-subject designs. (PsycINFO Database Record (c) 2015 APA, all rights reserved).
Journal of Occupational and Environmental Hygiene, 2015
Acclimation in a hot environment is one potent means to decrease the heat strain of work in a hot... more Acclimation in a hot environment is one potent means to decrease the heat strain of work in a hot environment. However, with diminished heat exposure, positive adaptations of acclimation may be lost. This rate of loss is equivocal and, once established, could be used to prescribe the time for re-acclimation. The purpose of this study was to determine the rate of loss of heat acclimation over a period of 6 weeks and determine the time needed for re-acclimation after 2 weeks and 4 weeks of de-acclimation in ten healthy participants. All participants first underwent an initial acclimation period (a 3-day plateau in Tre was used to signify acclimation). Based on the mean time to acclimate in Phase 1 (mean time to acclimate = 6.1±1.4 days), the loss of acclimation was mapped and participants were randomly assigned to one of two groups: one that underwent one 2-hr heat exposure at 1, 3, and 5 weeks post-acclimation, and one that underwent one 2-hr heat exposure session at 2,4, and 6 weeks. Complete loss of acclimation occurred in 6 weeks and, as expected, work HR and Tre increased with increasing time away from the heat (p<0.05). Based on the time for total loss of acclimation from Phase 1, participants in Phase 2 (n = 8) first underwent acclimation. Then, after either a 2-week or 4-week absence from the heat, participants returned to the laboratory for re-acclimation. While not statistically significant yet practically significant (p = 0.18; one-tailed confidence interval), average days for re-acclimation in the 2-week group tended to be fewer than in the 4-week group (days for re-acclimation = 3.8 ± 1.2 and 5.3 ± 1.9, respectively). Based on these general trends, for occupational settings, a re-acclimation period of 4 days is recommended after 2 weeks absence from the heat, 5 days for 4 weeks absence from the heat, and complete acclimation (6 days) after 6 weeks absence or more from the heat.
... LINDA M. RAFFAELE MENDEZ, HOWARD M. KNOFF, AND JOHN M. FERRON ... by gender andrace, (b) the ... more ... LINDA M. RAFFAELE MENDEZ, HOWARD M. KNOFF, AND JOHN M. FERRON ... by gender andrace, (b) the duplicated count of OSS by gender and race, and (c) the total number of each type of incident (eg, disruptive behavior) that resulted in OSS by gender and race. ...
Computer simulation methods were used to examine the sensitivity of model fit criteria to misspec... more Computer simulation methods were used to examine the sensitivity of model fit criteria to misspecification of the first-level error structure in two-level models of change, and then to examine the impact of misspecification on estimates of the variance parameters, estimates of the fixed effects, and tests of the fixed effects. Fit criteria frequently failed to identify the correct model when series lengths were short. Misspecification led to substantially biased estimates of variance parameters. The estimates of the fixed effects, however, remained unbiased for most conditions, and the tests of fixed effects were robust to misspecification for most conditions. The problems in the fixed effects occurred when nonlinear growth trajectories were coupled with data that were unequally spaced by different amounts for different individuals.
With the growing popularity of meta-analytic techniques to analyze and synthesize results across ... more With the growing popularity of meta-analytic techniques to analyze and synthesize results across sets of empirical studies, have come concerns about the sensitivity of traditional tests in meta-analysis to violations of assumptions. This is particularly distressing because the tenability of such assumptions in primary studies is often impossible to evaluate unless sufficient details are reported. Robust estimates of effect size, such as Cliff's δ , may yield superior inferences about the population effect size. The purpose of this study was to compare standardized mean differences (Cohen's d and Hedges' g) and δ in terms of the accuracy and precision of interval estimates of population mean effect size in meta-analysis. Factors investigated in the Monte Carlo study included characteristics of both the populations from which samples were drawn (distribution shape, variance heterogeneity, and population effect size) and the corpus of studies in each meta-analysis (sampl...
Uploads
Papers by John Ferron