We demonstrate the application of a novel four-step elicitation technique for generating expert p... more We demonstrate the application of a novel four-step elicitation technique for generating expert predictions of the concentration of pharmaceutical and over-the-counter drugs expected to be present in recycled water used for supplementing drinking water over the next twenty years (indirect potable reuse or IPR). This work is part of a larger project that is assessing the risks of IPR projects using relative risk assessments and developing safe concentration levels for drugs and other non-regulated chemicals. The project is also predicting contaminants that may be of future concern based on pharmaceutical production trends and use, their fate during water reclamation treatment processes, and testing expert judgment of future exposures against these predictions. The method is a structured elicitation of expert judgment and offers an alternative to other methods for examining risks and providing transparent risk assessment where data may be lacking. In such instances as environmental he...
... 3 PAV KALINOWSKI Statistical Cognition Laboratory, School of Psychological Science, La Trobe ... more ... 3 PAV KALINOWSKI Statistical Cognition Laboratory, School of Psychological Science, La Trobe University p.kalinowski@latrobe.edu.au ... American Psychologist, 24(2), 8391.PAV KALINOWSKI La Trobe University Victoria, 3086 Australia
Since the mid-1980s, confidence intervals (CIs) have been standard in medical journals. We sought... more Since the mid-1980s, confidence intervals (CIs) have been standard in medical journals. We sought lessons for psychology from medicine's experience with statistical reform by investigating two attempts by Kenneth Rothman to change statistical practices. We examined 594 American Journal of Public Health (AJPH) articles published between 1982 and 2000 and 110 Epidemiology articles published in 1990 and 2000. Rothman's editorial instruction to report CIs and not p values was largely effective: In AJPH, sole reliance on p values dropped from 63% to 5%, and CI reporting rose from 10% to 54%; Epidemiology showed even stronger compliance. However, compliance was superficial: Very few authors referred to CIs when discussing results. The results of our survey support what other research has indicated: Editorial policy alone is not a sufficient mechanism for statistical reform. Achieving substantial, desirable change will require further guidance regarding use and interpretation of CIs and appropriate effect size measures. Necessary steps will include studying researchers' understanding of CIs, improving education, and developing empirically justified recommendations for improved statistical practice.
Abstract Null-hypothesis significance tests (NHSTs) have received much criticism, especially duri... more Abstract Null-hypothesis significance tests (NHSTs) have received much criticism, especially during the last two decades. Yet many behavioral and social scientists are unaware that NHSTs have drawn increasing criticism, so this essay summarizes key criticisms. The essay also recommends alternative ways of assessing research findings. Although these recommendations are not complex, they do involve ways of thinking that many behavioral and social scientists find novel. Instead of making NHSTs, researchers should adapt their ...
ABSTRACT 1. In field surveys, ecological researchers and practitioners routinely make quantitativ... more ABSTRACT 1. In field surveys, ecological researchers and practitioners routinely make quantitative judgements that are known to vary in quality. Feedback about judgement accuracy is crucial for improving estimation performance, yet is not usually afforded to fieldworkers. One reason it is rare lies in the difficulty of obtaining ‘true values’ (e.g. percentage cover) to learn from. Often, the only information we can access is other people’s estimates of the same thing. Group average estimates tend to be remarkably accurate. By extension, receiving feedback about group averages may improve the estimation performance of individuals, dispensing with the need for ‘true values’ to learn from. 2. In Experiment 1, we tested whether feedback using group averages might improve estimates of species abundance as much as feedback using true values. However, not all feedback approaches are effective. In Experiment 2, we compared two feedback formats for presenting information about group estimates of percentage cover. In both experiments, we used a novel 4-point interval estimation approach to quantify uncertainty that is known to reduce overconfidence, but is yet to be applied in ecology. 3. Results from Experiment 1 show that feedback about group averages improved performance (calibration and accuracy) almost as much as feedback about the truth, despite the fact that group averages were generally not close to true values. In contrast, group averages in Experiment 2 were remarkably close to true values, but the only participants who improved their estimates were those who evaluated their own performance during the feedback session, using active feedback with a calibration component. 4. Feedback reminds surveyors not to give over-precise estimates, and to appropriately reflect uncertainty. It improves calibration and accuracy of abundance estimates, and could reasonably improve estimates of other quantities. Drawing on the wisdom of crowds, group averages could be used as a proxy for true values in feedback procedures. However, the format for delivering feedback matters. Actively engaging participants by having them evaluate their own estimation performance appears critical to improving their subsequent judgements, compared with passive feedback. We advocate the introduction of feedback into the training of ecologists. Keywords: calibration feedback, cover estimation, expert judgement, interval, overconfidence, species abundance, uncertainty, Wisdom of Crowds
Laterality: Asymmetries of Body, Brain and Cognition, 2013
How the brain is lateralised for emotion processing remains a key question in contemporary neurop... more How the brain is lateralised for emotion processing remains a key question in contemporary neuropsychological research. The right hemisphere hypothesis asserts that the right hemisphere dominates emotion processing, whereas the valence hypothesis holds that positive emotion is processed in the left hemisphere and negative emotion is controlled by the right hemisphere. A meta-analysis was conducted to assess unilateral brain-damaged individuals' performance on tasks of facial emotion perception according to valence. A systematic search of the literature identified seven articles that met the conservative selection criteria and could be included in a meta-analysis. A total of 12 meta-analyses of facial expression perception were constructed assessing identification and labelling tasks according to valence and the side of brain damage. The results demonstrated that both left and right hemisphere damage leads to impairments in emotion perception (identification and labelling) irrespective of valence. Importantly, right hemisphere damage prompted more pronounced emotion perception impairment than left hemisphere damage, across valence, suggesting right hemisphere dominance for emotion perception. Furthermore, right hemisphere damage was associated with a larger tendency for impaired perception of negative than positive emotion across identification and labelling tasks. Overall the findings support Adolphs, Jansari, and Tranel (2001) model whereby the right hemisphere preferentially processes negative facial expressions and both hemispheres process positive facial expressions.
Error bars commonly appear in figures in publications, but experimental biologists are often unsu... more Error bars commonly appear in figures in publications, but experimental biologists are often unsure how they should be used and interpreted. In this article we illustrate some basic features of error bars and explain how they can help communicate data and assist correct interpretation. Error bars may show confidence intervals, standard errors, standard deviations, or other quantities. Different types of error bars give quite different information, and so figure legends must make clear what error bars represent. We suggest eight simple rules to assist with effective use and interpretation of error bars.
Journal of Consulting and Clinical Psychology, 2005
Philip Kendall&am... more Philip Kendall's (1997) editorial encouraged authors in the Journal of Consulting and Clinical Psychology (JCCP) to report effect sizes and clinical significance. The present authors assessed the influence of that editorial--and other American Psychological Association initiatives to improve statistical practices--by examining 239 JCCP articles published from 1993 to 2001. For analysis of variance, reporting of means and standardized effect sizes increased over that period, but the rate of effect size reporting for other types of analyses surveyed remained low. Confidence interval reporting increased little, reaching 17% in 2001. By 2001, the percentage of articles considering clinical (not only statistical) significance was 40%, compared with 36% in 1996. In a follow-up survey of JCCP authors (N=62), many expressed positive attitudes toward statistical reform. Substantially improving statistical practices may require stricter editorial policies and further guidance for authors on reporting and interpreting measures.
We demonstrate the application of a novel four-step elicitation technique for generating expert p... more We demonstrate the application of a novel four-step elicitation technique for generating expert predictions of the concentration of pharmaceutical and over-the-counter drugs expected to be present in recycled water used for supplementing drinking water over the next twenty years (indirect potable reuse or IPR). This work is part of a larger project that is assessing the risks of IPR projects using relative risk assessments and developing safe concentration levels for drugs and other non-regulated chemicals. The project is also predicting contaminants that may be of future concern based on pharmaceutical production trends and use, their fate during water reclamation treatment processes, and testing expert judgment of future exposures against these predictions. The method is a structured elicitation of expert judgment and offers an alternative to other methods for examining risks and providing transparent risk assessment where data may be lacking. In such instances as environmental he...
... 3 PAV KALINOWSKI Statistical Cognition Laboratory, School of Psychological Science, La Trobe ... more ... 3 PAV KALINOWSKI Statistical Cognition Laboratory, School of Psychological Science, La Trobe University p.kalinowski@latrobe.edu.au ... American Psychologist, 24(2), 8391.PAV KALINOWSKI La Trobe University Victoria, 3086 Australia
Since the mid-1980s, confidence intervals (CIs) have been standard in medical journals. We sought... more Since the mid-1980s, confidence intervals (CIs) have been standard in medical journals. We sought lessons for psychology from medicine's experience with statistical reform by investigating two attempts by Kenneth Rothman to change statistical practices. We examined 594 American Journal of Public Health (AJPH) articles published between 1982 and 2000 and 110 Epidemiology articles published in 1990 and 2000. Rothman's editorial instruction to report CIs and not p values was largely effective: In AJPH, sole reliance on p values dropped from 63% to 5%, and CI reporting rose from 10% to 54%; Epidemiology showed even stronger compliance. However, compliance was superficial: Very few authors referred to CIs when discussing results. The results of our survey support what other research has indicated: Editorial policy alone is not a sufficient mechanism for statistical reform. Achieving substantial, desirable change will require further guidance regarding use and interpretation of CIs and appropriate effect size measures. Necessary steps will include studying researchers' understanding of CIs, improving education, and developing empirically justified recommendations for improved statistical practice.
Abstract Null-hypothesis significance tests (NHSTs) have received much criticism, especially duri... more Abstract Null-hypothesis significance tests (NHSTs) have received much criticism, especially during the last two decades. Yet many behavioral and social scientists are unaware that NHSTs have drawn increasing criticism, so this essay summarizes key criticisms. The essay also recommends alternative ways of assessing research findings. Although these recommendations are not complex, they do involve ways of thinking that many behavioral and social scientists find novel. Instead of making NHSTs, researchers should adapt their ...
ABSTRACT 1. In field surveys, ecological researchers and practitioners routinely make quantitativ... more ABSTRACT 1. In field surveys, ecological researchers and practitioners routinely make quantitative judgements that are known to vary in quality. Feedback about judgement accuracy is crucial for improving estimation performance, yet is not usually afforded to fieldworkers. One reason it is rare lies in the difficulty of obtaining ‘true values’ (e.g. percentage cover) to learn from. Often, the only information we can access is other people’s estimates of the same thing. Group average estimates tend to be remarkably accurate. By extension, receiving feedback about group averages may improve the estimation performance of individuals, dispensing with the need for ‘true values’ to learn from. 2. In Experiment 1, we tested whether feedback using group averages might improve estimates of species abundance as much as feedback using true values. However, not all feedback approaches are effective. In Experiment 2, we compared two feedback formats for presenting information about group estimates of percentage cover. In both experiments, we used a novel 4-point interval estimation approach to quantify uncertainty that is known to reduce overconfidence, but is yet to be applied in ecology. 3. Results from Experiment 1 show that feedback about group averages improved performance (calibration and accuracy) almost as much as feedback about the truth, despite the fact that group averages were generally not close to true values. In contrast, group averages in Experiment 2 were remarkably close to true values, but the only participants who improved their estimates were those who evaluated their own performance during the feedback session, using active feedback with a calibration component. 4. Feedback reminds surveyors not to give over-precise estimates, and to appropriately reflect uncertainty. It improves calibration and accuracy of abundance estimates, and could reasonably improve estimates of other quantities. Drawing on the wisdom of crowds, group averages could be used as a proxy for true values in feedback procedures. However, the format for delivering feedback matters. Actively engaging participants by having them evaluate their own estimation performance appears critical to improving their subsequent judgements, compared with passive feedback. We advocate the introduction of feedback into the training of ecologists. Keywords: calibration feedback, cover estimation, expert judgement, interval, overconfidence, species abundance, uncertainty, Wisdom of Crowds
Laterality: Asymmetries of Body, Brain and Cognition, 2013
How the brain is lateralised for emotion processing remains a key question in contemporary neurop... more How the brain is lateralised for emotion processing remains a key question in contemporary neuropsychological research. The right hemisphere hypothesis asserts that the right hemisphere dominates emotion processing, whereas the valence hypothesis holds that positive emotion is processed in the left hemisphere and negative emotion is controlled by the right hemisphere. A meta-analysis was conducted to assess unilateral brain-damaged individuals' performance on tasks of facial emotion perception according to valence. A systematic search of the literature identified seven articles that met the conservative selection criteria and could be included in a meta-analysis. A total of 12 meta-analyses of facial expression perception were constructed assessing identification and labelling tasks according to valence and the side of brain damage. The results demonstrated that both left and right hemisphere damage leads to impairments in emotion perception (identification and labelling) irrespective of valence. Importantly, right hemisphere damage prompted more pronounced emotion perception impairment than left hemisphere damage, across valence, suggesting right hemisphere dominance for emotion perception. Furthermore, right hemisphere damage was associated with a larger tendency for impaired perception of negative than positive emotion across identification and labelling tasks. Overall the findings support Adolphs, Jansari, and Tranel (2001) model whereby the right hemisphere preferentially processes negative facial expressions and both hemispheres process positive facial expressions.
Error bars commonly appear in figures in publications, but experimental biologists are often unsu... more Error bars commonly appear in figures in publications, but experimental biologists are often unsure how they should be used and interpreted. In this article we illustrate some basic features of error bars and explain how they can help communicate data and assist correct interpretation. Error bars may show confidence intervals, standard errors, standard deviations, or other quantities. Different types of error bars give quite different information, and so figure legends must make clear what error bars represent. We suggest eight simple rules to assist with effective use and interpretation of error bars.
Journal of Consulting and Clinical Psychology, 2005
Philip Kendall&am... more Philip Kendall's (1997) editorial encouraged authors in the Journal of Consulting and Clinical Psychology (JCCP) to report effect sizes and clinical significance. The present authors assessed the influence of that editorial--and other American Psychological Association initiatives to improve statistical practices--by examining 239 JCCP articles published from 1993 to 2001. For analysis of variance, reporting of means and standardized effect sizes increased over that period, but the rate of effect size reporting for other types of analyses surveyed remained low. Confidence interval reporting increased little, reaching 17% in 2001. By 2001, the percentage of articles considering clinical (not only statistical) significance was 40%, compared with 36% in 1996. In a follow-up survey of JCCP authors (N=62), many expressed positive attitudes toward statistical reform. Substantially improving statistical practices may require stricter editorial policies and further guidance for authors on reporting and interpreting measures.
Uploads
Papers by Fiona Fidler