Power of Test

In an accountability era that seems to be defined primarily by test scores, raising student achievement, and school rankings that lead to either rewards or sanctions, school leaders often feel satisfaction, pride, exhilaration or debilitating anxiety, shame, blame and guilt. Novice and veteran school leaders alike are not exempt from being publicly identified as leaders in 'poor performing' or 'high performing' schools by the media. As parents and the community increasingly reify the power of test data, school leaders become the unexpected victims or heroes of an accountability system within which emotional involvement is often one of the most difficult aspects to deal with. Emotions as a social construction will be examined in this paper, in which sociological, political and institutional factors mitigate or militate educational leaders' feelings within a context of accountability.

The gold standard for an empirical science is the replicability of its research results. But the estimated average replicability rate of key-effects that top-tier psychology journals report falls between 36 and 39% (objective vs. subjective rate; Open Science Collaboration, 2015). So the standard mode of applying null-hypothesis significance testing (NHST) fails to adequately separate stable from random effects. Therefore, NHST does not fully convince as a statistical inference strategy. We argue that the replicability crisis is "home-made" because more sophisticated strategies can deliver results the successful replication of which is sufficiently probable. Thus, we can overcome the replicability crisis by integrating empirical results into genuine research programs. Instead of continuing to narrowly evaluate only the stability of data against random fluctuations (discovery context), such programs evaluate rival hypotheses against stable data (justification context).

Following Cohen (1965), many researchers treat setting the β error-rate to 0.20 as a reasonable convention, and similarly set the α error-rate to 0.05. The 4:1 ratio this yields is a strong bias favoring statistically significant empirical results ("discoveries") over replicable ones. We briefly evaluate (but reject) how Cohen's justifies this ratio. Practical reasons seem to add nothing in defense of an asymmetrical ratio. On epistemological grounds, we advocate 1:1 as the right ratio.

Before replication becomes mainstream, the potential for generating theoretical knowledge better be clear. Replicating statistically significant nonrandom data shows that an original study made a discovery; replicating a specified theoretical effect shows that an original study corroborated a theory. Yet only in the latter case is replication a necessary, sound, and worthwhile strategy.

Normality tests are used in the statistical analysis to determine whether a normal distribution is acceptable as a model for the data analysed. A wide range of available tests employs different properties of normal distribution to compare empirical and theoretical distributions. In the present paper, we perform the Monte Carlo simulation to analyse test power. We compare commonly known and applied tests (standard and robust versions of the Jarque-Bera test, Lilliefors test, chi-square goodness-of-fit t est, S hapiro-Francia test, Cramer-von Mises goodness-of-fit test, Shapiro-Wilk test, D'Agostino test, and Anderson-Darling test) to the test based on robust L-moments. In the text, in Jarque-Bera type test the moment characteristics of skewness and kurtosis are replaced with their robust versions-L-skewness and L-kurtosis. The distributions with heavy tails (lognormal, Weibull, loglogistic and Student) are used to draw random samples to show the performance of tests when applied on data with outliers. Small sample properties (from 10 observations) are analysed up to large samples of 200 observations. Our results concerning the properties of the classical tests are in line with the conclusion of other recent articles. We concentrate on properties of the test based on L-moments. This normality test is comparable to well-performing and reliable tests; however, it is outperformed by the most powerful Shapiro-Wilks and Shapiro-Francia tests. It works well for Student (symmetric) distribution, comparably with the most frequently used Jarque-Berra tests. As expected, the test is robust to the presence of outliers in comparison with sensitive tests based on product moments or correlations. The test turns out to be very universally reliable.

Typical complex sample designs lead to informative samples, which means that the distribution of a variable in the sample is different from its distribution in the population. To determine and measure the impact of informativeness, we compare design-based and model-based variances of estimated parameters (as well as the estimated parameters themselves) in a logistic model under the assumption that the

STEVEN COOK1, SEAN HOLLY2 and PAUL TURNER3 1Coventry Business School, University of Coventry, Priory Street, Coventry, CV1 5FB, UK, E-mail: steven.cook@coventry.ac.uk; 2Department of Applied Economics, University of Cambridge, UK; 3Department of Economics, University of Sheffield, UK ... Abstract. The power of the recently proposed Escribano–Pfann (1998) model of asymmetry is examined via Monte Carlo simulation. As found previously for an alternative asymmetric model, the power of this model is seen to be low. ... Since the start of the ...

This paper uses Monte Carlo methods to investigate the effects of asymmetricadjustment on estimates of the parameters of the equilibrium relationshipbetween a set of variables. We demonstrate that simple least squares estimatesand the implicit estimates from a symmetric error correction model both leadto biases in the constant term. This bias increases with the size of theasymmetry and shows no tendency to decline with the sample size. We also showthat if the biased estimates of the equilibrium relationship are then used todevide the ...

Classical resource- and the less studied ratio-dependent models of predator–prey relationships provide divergent predictions as to the sustained ecological effects of bottom-up forcing. While resource-dependent models, which consider only instantaneous prey density in modelling predator responses, predict community responses that are dependent on the number of trophic levels in a system, ratio-dependent models, which consider the number of prey per consumer, predict proportional increase in each level irrespective of chain length. The two models are only subtly different for systems with two or three trophic levels but in the case of four trophic levels, predict opposite effects of enrichment on primary producers. Despite the poor discriminatory power of tests of the models in systems with two or three trophic levels, field tests in estuarine and marine systems with four trophic levels have been notably absent. Sampling of phytoplankton, macroinvertebrates, invertebrate-feeding fishes, piscivorous fishes in Kooloonbung Creek, Hastings River estuary, eastern Australia, subject to over 20 years of sewage discharge, revealed increased abundances in all four trophic levels at the disturbed location relative to control sites. Increased abundance of phytoplankton at the disturbed site was counter to the predictions of resource-dependent models, which posit a reduction in the first trophic level in response to enrichment. By contrast, the increase in abundance of this first trophic level and the proportionality of increases in abundances of each of the four trophic groups to nitrogen loading provided strong support for ratio dependency. This first evidence of ratio dependence in an estuarine system with four trophic levels not only demonstrates the applicability of ecological theory which seeks to simplify the complexity of systems, but has implications for management. Although large nutrient inputs frequently induce mortality of invertebrates and fish, we have shown that smaller inputs may in fact enhance biomass of all trophic levels.

We examine the long-run relationship between stock prices and goods prices to gauge whether stock market investment can hedge against inflation. Data from 16 OECD countries over the period 1970–2006 are used. We account for different inflation regimes with the use of ...

We examine the power of the Granger-Lee [Granger, CWJ, Lee, TH, 1989. Investigation of production, sales and inventory relationships using multicointegration and non-symmetric error correction models. Journal of Applied Econometrics 4, S145–S159.] test for asymmetric adjustment in an error correction model. Monte Carlo simulations indicate that such tests typically have low power in rejecting the null of symmetric adjustment. The power of the test increases with the sample size and with the ratio of the variance of the independent ...

The main purpose of this work is the derivation of relations for calculation of „power function for tests of null hypotheses on mutual relation of linear regression functions". The normality of error disturbances of processed data is the condition for power function ...

The nonparametric approach to consumer-demand analysis-based on revealed-preference axioms-is reviewed. Particular attention is paid to questions of size and power of tests for consistency of data with the existence of a stable, well-behaved utility function that could have generated the data. An application to Australian meat demand is used to show how these notions can be quantified and how prior