The document discusses key concepts in statistical hypothesis testing including:
1) Type I and Type II errors - rejecting a true null hypothesis or failing to reject a false null hypothesis.
2) Error rates - the probability of making Type I or Type II errors, denoted by alpha and beta. Lowering one generally increases the other.
3) The null hypothesis is what is being tested and is presumed true until evidence suggests otherwise. A statistically significant result means the null hypothesis can be rejected.
The document discusses key concepts in statistical hypothesis testing including:
1) Type I and Type II errors - rejecting a true null hypothesis or failing to reject a false null hypothesis.
2) Error rates - the probability of making Type I or Type II errors, denoted by alpha and beta. Lowering one generally increases the other.
3) The null hypothesis is what is being tested and is presumed true until evidence suggests otherwise. A statistically significant result means the null hypothesis can be rejected.
The document discusses key concepts in statistical hypothesis testing including:
1) Type I and Type II errors - rejecting a true null hypothesis or failing to reject a false null hypothesis.
2) Error rates - the probability of making Type I or Type II errors, denoted by alpha and beta. Lowering one generally increases the other.
3) The null hypothesis is what is being tested and is presumed true until evidence suggests otherwise. A statistically significant result means the null hypothesis can be rejected.
The document discusses key concepts in statistical hypothesis testing including:
1) Type I and Type II errors - rejecting a true null hypothesis or failing to reject a false null hypothesis.
2) Error rates - the probability of making Type I or Type II errors, denoted by alpha and beta. Lowering one generally increases the other.
3) The null hypothesis is what is being tested and is presumed true until evidence suggests otherwise. A statistically significant result means the null hypothesis can be rejected.
1. Introduction: In statistical hypothesis testing, a type I error is the rejection of a true null hypothesis (also known as a "false positive" finding or conclusion), while a type II error is the non-rejection of a false null hypothesis (also known as a "false negative" finding or conclusion). Much of statistical theory revolves around the minimization of one or both of these errors, though the complete elimination of either is a statistical impossibility for non- deterministic algorithms. By selecting a low threshold (cut-off) value and modifying the alpha (p) level, the quality of the hypothesis test can be increased. The knowledge of Type I errors and Type II errors is widely used in medical science, biometrics and computer science. Intuitively, type I errors can be thought of as errors of commission, and type II errors as errors of omission. For example, in the context of binary classification, when trying to decide whether an input image X is an image of a dog: an error of commission (type I) is classifying X as a dog when it isn't, whereas an error of omission (type II) is classifying X as not a dog when it is.
2. Types of Errors: In statistical test theory, the notion of a statistical error is an integral part of hypothesis testing. The test goes about choosing about two competing propositions called null hypothesis, denoted by H0 and alternative hypothesis, denoted by H1 . This is conceptually similar to the judgement in a court trial. The null hypothesis corresponds to the position of defendant: just as he is presumed to be innocent until proven guilty, so is the null hypothesis presumed to be true until the data provide convincing evidence against it. The alternative hypothesis corresponds to the position against the defendant. If the result of the test corresponds with reality, then a correct decision has been made. However, if the result of the test does not correspond with reality, then an error has occurred. There are two situations in which the decision is wrong. The null hypothesis may be true, whereas we reject H0. On the other hand, the alternative hypothesis H1 may be true, whereas we do not reject H0. Two types of error are distinguished: Type I error and type II error.
(A) Type I Error
The first kind of error is the rejection of a true null hypothesis as the result of a test procedure. This kind of error is called a type I error and is sometimes called an error of the first kind. In terms of the courtroom example, a type I error corresponds to convicting an innocent defendant.
The second kind of error is the failure to reject a false null hypothesis as the result of a test procedure. This sort of error is called a type II error and is also referred to as an error of the second kind.
3. Error Rate: A perfect test would have zero false positives and zero false negatives. However, statistics is a game of probability, and it cannot be known for certain whether statistical conclusions are correct. Whenever there is uncertainty, there is the possibility of making an error. Considering this nature of statistics science, all statistical hypothesis tests have a probability of making type I and type II errors. The type I error rate or significance level is the probability of rejecting the null hypothesis given that it is true. It is denoted by the Greek letter α (alpha) and is also called the alpha level. Usually, the significance level is set to 0.05 (5%), implying that it is acceptable to have a 5% probability of incorrectly rejecting the true null hypothesis. The rate of the type II error is denoted by the Greek letter β (beta) and related to the power of a test, which equals 1-β. These two types of error rates are traded off against each other: for any given sample set, the effort to reduce one type of error generally results in increasing the other type of error.
Fig. 01: The results obtained from negative sample (left curve) overlap with the results obtained from positive samples (right curve). By moving the result cutoff value (vertical bar), the rate of false positives (FP) can be decreased, at the cost of raising the number of false negatives (FN), or vice-versa.
The same idea can be expressed in terms of the rate of correct results and therefore used to minimize error rates and improve the quality of hypothesis test. To reduce the probability of committing a Type I error, making the alpha (p) value more stringent is quite simple and efficient. To decrease the probability of committing a Type II error, which is closely associated with analyses' power, either increasing the test's sample size or relaxing the alpha level could increase the analyses' power. A test statistic is robust if the Type I error rate is controlled. Varying different threshold (cut-off) value could also be used to make the test either more specific or more sensitive, which in turn elevates the test quality. For example, imagine a medical test, in which experimenter might measure the concentration of a certain protein in the blood sample. Experimenter could adjust the threshold (black vertical line in the figure) and people would be diagnosed as having diseases if any number is detected above this certain threshold. According to the image, changing the threshold would result in changes in false positives and false negatives, corresponding to movement on the curve.
5. Null Hypothesis: It is standard practice for statisticians to conduct tests in order to determine whether or not a "speculative hypothesis" concerning the observed phenomena of the world (or its inhabitants) can be supported. The results of such testing determine whether a particular set of results agrees reasonably (or does not agree) with the speculated hypothesis. On the basis that it is always assumed, by statistical convention, that the speculated hypothesis is wrong, and the so-called "null hypothesis" that the observed phenomena simply occur by chance (and that, as a consequence, the speculated agent has no effect) - the test will determine whether this hypothesis is right or wrong. This is why the hypothesis under test is often called the null hypothesis (most likely, coined by Fisher (1935), because it is this hypothesis that is to be either nullified or not nullified by the test. When the null hypothesis is nullified, it is possible to conclude that data support the "alternative hypothesis" (which is the original speculated one). The consistent application by statisticians of Neyman and Pearson's convention of representing "the hypothesis to be tested" (or "the hypothesis to be nullified") with the expression H0 has led to circumstances where many understand the term "the null hypothesis" as meaning "the nil hypothesis" - a statement that the results in question have arisen through chance. This is not necessarily the case - the key restriction, as per Fisher (1966), is that "the null hypothesis must be exact, that is free from vagueness and ambiguity, because it must supply the basis of the 'problem of distribution,' of which the test of significance is the
solution." As a consequence of this, in experimental science the null hypothesis is
generally a statement that a particular treatment has no effect; in observational science, it is that there is no difference between the value of a particular measured variable, and that of an experimental prediction.
6. Statistical Level of Significance:
If the probability of obtaining a result as extreme as the one obtained, supposing that the null hypothesis were true, is lower than a pre-specified cut-off probability (for example, 5%), then the result is said to be statistically significant and the null hypothesis is rejected. British statistician Sir Ronald Aylmer Fisher (1890–1962) stressed that the "null hypothesis" is never proved or established, but is possibly disproved, in the course of experimentation. Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis.
7. References:
"Type I Error and Type II Error". explorable.com. Retrieved 14 December 2019.
Chow, Y. W.; Pietranico, R.; Mukerji, A. (27 October 1975). "Studies of oxygen binding energy to hemoglobin molecule". Biochemical and Biophysical Research Communications. 66 (4): 1424–1431. A modern introduction to probability and statistics : understanding why and how. Dekking, Michel, 1946-. London: Springer. 2005. A modern introduction to probability and statistics : understanding why and how. Dekking, Michel, 1946-. London: Springer. 2005. Sheskin, David (2004). Handbook of Parametric and Nonparametric Statistical Procedures. CRC Press. p. 54. Smith, R. J.; Bryant, R. G. (27 October 1975). "Metal substitutions incarbonic anhydrase: a halide ion probe study". Biochemical and Biophysical Research Communications. 66 (4): 1281–1286. Lindenmayer, David. (2005). Practical conservation biology. Burgman, Mark A. Collingwood, Vic.: CSIRO Pub. Chow, Y. W.; Pietranico, R.; Mukerji, A. (27 October 1975). "Studies of oxygen binding energy to hemoglobin molecule". Biochemical and Biophysical Research Communications. 66 (4): 1424–1431. Smith, R. J.; Bryant, R. G. (27 October 1975). "Metal substitutions incarbonic anhydrase: a halide ion probe study". Biochemical and Biophysical Research Communications. 66 (4): 1281–1286. Smith, R. J.; Bryant, R. G. (27 October 1975). "Metal substitutions incarbonic anhydrase: a halide ion probe study". Biochemical and Biophysical Research Communications. 66 (4): 1281–1286.
Moroi, K.; Sato, T. (15 August 1975). "Comparison between procaine and isocarboxazid metabolism in vitro by a liver microsomal amidase-esterase". Biochemical Pharmacology. 24(16): 1517–1521. NEYMAN, J.; PEARSON, E. S. (1928). "On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference Part I". Biometrika. 20A (1–2): 175–240. C.I.K.F. (July 1951). "Probability Theory for Statistical Methods. By F. N. David. [Pp. ix + 230. Cambridge University Press. 1949. Price 155.]". Journal of the Staple Inn Actuarial Society. 10 (3): 243–244. Neyman, J.; Pearson, E. S. (30 October 1933). "The testing of statistical hypotheses in relation to probabilities a priori". Mathematical Proceedings of the Cambridge Philosophical Society. 29 (4): 492–510. Fisher, R.A. (1966). The design of experiments. 8th edition. Hafner: Edinburgh.
