The goal of this article is to discuss psychometric issues that are important in neuropsychologic... more The goal of this article is to discuss psychometric issues that are important in neuropsychological assessment and to present some recent advances in empirically derived interpretation methods. The article is divided into four main sections. The first two sections provide an overview of some of the basic psychometric concepts that make up the foundation of assessment. The first section presents an overview of how the adequacy of the normative sample and the shape of the score distribution can impact the interpretation of test performance. The second section presents a review of the role of measurement error and considers how the level of scores, their rank in the distribution, and the presence of ceiling or floor effects can impact interpretation. The third section explores the issue of normal variability and the prevalence of "abnormal" test scores in healthy people. The final section provides an overview of various methods for interpreting change in test performance over time. The authors present this psychometric information in a manner that should be easily understood by most clinicians, with examples that employ commonly used neuropsychological tests, such as the Wechsler Adult Intelligence Scale–Third Edition (WAIS-III; Wechsler, 1997a), Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; Wechsler, 2008), Wechsler Memory Scale–Third Edition (WMS-III; Wechsler, 1997b), Wechsler Memory Scale–Fourth Edition (WMS-IV; Wechsler, 2009), Children’s Memory Scale (CMS; Cohen, 1997), California Verbal Learning Test–Second Edition (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), and the Neuropsychological Assessment Battery (NAB; Stern & White, 2003).
Journal of Clinical and Experimental Neuropsychology, Jun 1, 1994
A computer-administered memory test was given to normal subjects instructed to feign brain damage... more A computer-administered memory test was given to normal subjects instructed to feign brain damage, normal controls, and traumatic brain-injured (TBI) patients with complaints of memory dysfunction. The test, a revised version of an instrument developed by Hiscock and Hiscock (1989) employed forced, two-choice recognition of previously presented five-digit numbers. Two levels of item difficulty and three retention intervals were used. Control and TBI subjects performed at near ceiling level on easy items and items with short retention intervals. Feigning subjects performed worse than TBI and control subjects at all levels of item difficulty and all retention intervals. Subjects groups were maximally distinguishable from one another by performance on difficult items and items with the longest retention interval. All TBI patients performed at or above chance level. Only 15% of the feigning subjects performed below chance level on any section of the test. Jack-knifed discriminant function analysis correctly classified 83% of all subjects into their respective groups. Although evaluation of patient performance relative to chance probability is useful for indicating the presence of extremely exaggerated memory deficits, criteria derived from sample distributions of group scores were superior for evaluation of motivation in less obvious cases. The Victoria Revision may be useful for detecting true memory deficit as well as dissimulation.
HIV-1 infected persons who are pursuing disability benefits are increasingly seeking neuropsychol... more HIV-1 infected persons who are pursuing disability benefits are increasingly seeking neuropsychological assessment for purposes of corroborating functional impairment. Thus, research on the utility of measures of symptom validity among these patients is needed. Recently, Mittenberg, Azrin, Millsaps, and Heilbronner (1993) proposed a malingering index score for the WechslerMemoryScale-Revised that is derived by subtracting the Attention/Concentration Index (ACI) score from the General Memory Index Score (GMI). This study is a cross-validation of the specificity of the GMI-ACI Malingering Index in a sample of 55 non-compensation-seeking HIV-positive (HIV+) patients. An overall false-positive rate of 7% was observed for the GMI-ACI Malingering Index. However, further analyses showed that GMI-ACI Malingering Index scores were correlated with GMI scores such that false-positive errors were substantially higher (18%) among patients who obtained above-average GMI scores. These findings suggest a cautious approach to application of the GMI-ACI Malingering Index, particularly among patients who obtain above-average GMI scores.
Clinical norms for the 64-item Wisconsin Card Sorting Test (WCST-64) are presented. The norms wer... more Clinical norms for the 64-item Wisconsin Card Sorting Test (WCST-64) are presented. The norms were derived from 303 persons presenting for emergency services who met criteria for uncomplicated mild head injury. Most data were obtained within 2 days of injury, and the remaining data were obtained within 10 days of injury. The norms may be used to help determine whether or not a person's scores on the WCST-64 are typical of those seen shortly after sustaining an uncomplicated mild head injury.
ABSTRACT The purpose of this chapter is to illustrate how an understanding of the psychometric pr... more ABSTRACT The purpose of this chapter is to illustrate how an understanding of the psychometric properties of tests, normative samples, and test scores are an essential foundation for meaningful and accurate clinical interpretations and reduces the likelihood of misinterpreting test results. Our goal is to present this information in an easy-to-understand format that facilitates clinicians’ knowledge of basic psychometrics in the context of test score interpretation. Clinical examples using commonly used tests will be provided throughout to illustrate the relevance and utility of these concepts in clinical practice. With regard to sample distributions, we will review concepts relating to non-normality and the influence of score distribution characteristics on derived scores. Floor and ceiling effects, equivalence of normative data sets, and truncated distributions will be discussed with regard to test items and test norms. When comparing scores between tests, we will review the role of test measurement error. We will also discuss normal variability and briefly comment on the prevalence of low test scores in healthy people, and how to use this information for supplementing clinical judgment. Finally, we will provide an overview of various methods for interpreting change in test performance over time.
The goals of the study were to determine how neuropsychological functioning is related to depress... more The goals of the study were to determine how neuropsychological functioning is related to depressive status in persons with head injury, and to quantify this relationship from a clinically relevant standpoint. Participants were 175 adults involved in litigation, referred for evaluation of suspected head injury. Depression status was measured using the Depression Content (Dep) scale of the MMPI-2. Depression status was related to measures of visual attention and psychomotor skills, but not to other neuropsychological domains such as verbal ability, visual-spatial reasoning, or encoding/organization. However, differences between low Dep and high Dep groups were minimal from a clinical standpoint. Depression appeared to contribute to an increased risk of impaired neuropsychological performance across domains, but only in persons not severely compromised by neuropsychological deficits. Overall, the results indicated a small effect of depression on neuropsychological functioning that is likely only detectable in persons whose neuropsychological compromise is relatively minimal.
Introduction Previous studies (IAT, PET, fMRI) demonstrate that early brain injury can lead to at... more Introduction Previous studies (IAT, PET, fMRI) demonstrate that early brain injury can lead to atypical functional organization of language [l-3]. In normal controls and recovering aphasics, language studies often reveal left, right or bilateral language dominance, usually involving homologous areas in both hemispheres [4]. Less frequently, one hemisphere is dominant for speech production, while the other is dominant for verbal comprehension [l]. We present a patient with unusual language organization that does not fit these patterns. Non-homologous bifrontal regions were activated on a verb generation task. Furthermore, verb generation and verb reading tasks revealed an interhemispheric diaschisis.
The purpose of this study was to evaluate the concurrent validity and clinical utility of a short... more The purpose of this study was to evaluate the concurrent validity and clinical utility of a short form of the Visual Form Discrimination Test in persons with closed head injuries (n = 62). Given the homogeneity of the items, and patients' apparent consistency in responding, we hypothesized that scores from an eight-item short form would be highly similar to the scores from the full version of the test (i.e. 16 items). The mean difference between the short form and full version was less than one point, and the correlation between the two forms was 0.86. Applying a clinical decision rule resulted in a 'normal/impaired' overall correct classification rate of 98.4%. The short form shows considerable promise as a brief test of visual discrimination in persons with closed head injuries.
Victoria Symptom Validity Test (VSVT) scores from six nonlitigants with neurological illness acco... more Victoria Symptom Validity Test (VSVT) scores from six nonlitigants with neurological illness accompanied by dense anterograde amnesia or severe memory impairment are presented. All of these patients obtained perfect or near perfect scores on the VSVT. These data add to the literature suggesting that the VSVT is insensitive to genuine neurologically-based memory impairment and provide an additional floor-level clinical benchmark against which to compare the performance of litigants.
Over the past 10 years, widespread and concerted research efforts have led to increasingly sophis... more Over the past 10 years, widespread and concerted research efforts have led to increasingly sophisticated and efficient methods and instruments for detecting exaggeration or fabrication of cognitive dysfunction. Despite these psychometric advances, the process of diagnosing malingering remains difficult and largely idiosyncratic. This article presents a proposed set of diagnostic criteria that define psychometric, behavioral, and collateral data indicative of possible, probable, and definite malingering of cognitive dysfunction, for use in clinical practice and for defining populations for clinical research. Relevant literature is reviewed, and limitations and benefits of the proposed criteria are discussed.
This study examined the effect of vagus nerve stimulation on quality of life in children with epi... more This study examined the effect of vagus nerve stimulation on quality of life in children with epilepsy using a validated quality-of-life scale and an empirical technique that accounts for measurement error in assessing individual change (the reliable change index). Participants were 34 children with severe intractable epilepsy who underwent vagus nerve stimulation and 19 children with intractable epilepsy who received medical management. Parent-completed epilepsy-specific and global ratings at baseline and after 1 year indicated that most children had no changes in quality of life following vagus nerve stimulation (52%-77%), similar to the comparison group. There was a trend for decreases to be less common in the vagus nerve stimulation group (14% vs 37%, P < .07), but there was no relation between improved quality of life and seizure control. The results raise questions about the mechanisms that underlie changes in quality of life after vagus nerve stimulation in this group of children.
Journal of Clinical and Experimental Neuropsychology, Oct 1, 2000
The present study used well-defined traumatic brain injury (TBI) and mixed neurological (other th... more The present study used well-defined traumatic brain injury (TBI) and mixed neurological (other than TBI) and psychiatric samples to examine the specificity and sensitivity to Malingered Neurocognitive Dysfunction (MND) of four individual California Verbal Learning Test (CVLT) variables and eight composite CVLT malingering indicators. Participants were 275 traumatic brain injury and 352 general clinical patients seen for neuropsychological evaluation. The TBI patients were assigned to one of five groups using the Slick, Sherman, and Iverson (1999) criteria: no incentive, incentive only, suspect, and malingering (both Probable MNDand Definite MND). Within TBI, persons with the strongest evidence for malingering (Probable and Definite) had the most extreme scores. Good sensitivity (approximately 50%) in the context of excellent specificity (> 95%) was found in the TBI samples. Issues related to the appropriate clinical application of these data are discussed.
The goal of this article is to discuss psychometric issues that are important in neuropsychologic... more The goal of this article is to discuss psychometric issues that are important in neuropsychological assessment and to present some recent advances in empirically derived interpretation methods. The article is divided into four main sections. The first two sections provide an overview of some of the basic psychometric concepts that make up the foundation of assessment. The first section presents an overview of how the adequacy of the normative sample and the shape of the score distribution can impact the interpretation of test performance. The second section presents a review of the role of measurement error and considers how the level of scores, their rank in the distribution, and the presence of ceiling or floor effects can impact interpretation. The third section explores the issue of normal variability and the prevalence of "abnormal" test scores in healthy people. The final section provides an overview of various methods for interpreting change in test performance over time. The authors present this psychometric information in a manner that should be easily understood by most clinicians, with examples that employ commonly used neuropsychological tests, such as the Wechsler Adult Intelligence Scale–Third Edition (WAIS-III; Wechsler, 1997a), Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; Wechsler, 2008), Wechsler Memory Scale–Third Edition (WMS-III; Wechsler, 1997b), Wechsler Memory Scale–Fourth Edition (WMS-IV; Wechsler, 2009), Children’s Memory Scale (CMS; Cohen, 1997), California Verbal Learning Test–Second Edition (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), and the Neuropsychological Assessment Battery (NAB; Stern & White, 2003).
Journal of Clinical and Experimental Neuropsychology, Jun 1, 1994
A computer-administered memory test was given to normal subjects instructed to feign brain damage... more A computer-administered memory test was given to normal subjects instructed to feign brain damage, normal controls, and traumatic brain-injured (TBI) patients with complaints of memory dysfunction. The test, a revised version of an instrument developed by Hiscock and Hiscock (1989) employed forced, two-choice recognition of previously presented five-digit numbers. Two levels of item difficulty and three retention intervals were used. Control and TBI subjects performed at near ceiling level on easy items and items with short retention intervals. Feigning subjects performed worse than TBI and control subjects at all levels of item difficulty and all retention intervals. Subjects groups were maximally distinguishable from one another by performance on difficult items and items with the longest retention interval. All TBI patients performed at or above chance level. Only 15% of the feigning subjects performed below chance level on any section of the test. Jack-knifed discriminant function analysis correctly classified 83% of all subjects into their respective groups. Although evaluation of patient performance relative to chance probability is useful for indicating the presence of extremely exaggerated memory deficits, criteria derived from sample distributions of group scores were superior for evaluation of motivation in less obvious cases. The Victoria Revision may be useful for detecting true memory deficit as well as dissimulation.
HIV-1 infected persons who are pursuing disability benefits are increasingly seeking neuropsychol... more HIV-1 infected persons who are pursuing disability benefits are increasingly seeking neuropsychological assessment for purposes of corroborating functional impairment. Thus, research on the utility of measures of symptom validity among these patients is needed. Recently, Mittenberg, Azrin, Millsaps, and Heilbronner (1993) proposed a malingering index score for the WechslerMemoryScale-Revised that is derived by subtracting the Attention/Concentration Index (ACI) score from the General Memory Index Score (GMI). This study is a cross-validation of the specificity of the GMI-ACI Malingering Index in a sample of 55 non-compensation-seeking HIV-positive (HIV+) patients. An overall false-positive rate of 7% was observed for the GMI-ACI Malingering Index. However, further analyses showed that GMI-ACI Malingering Index scores were correlated with GMI scores such that false-positive errors were substantially higher (18%) among patients who obtained above-average GMI scores. These findings suggest a cautious approach to application of the GMI-ACI Malingering Index, particularly among patients who obtain above-average GMI scores.
Clinical norms for the 64-item Wisconsin Card Sorting Test (WCST-64) are presented. The norms wer... more Clinical norms for the 64-item Wisconsin Card Sorting Test (WCST-64) are presented. The norms were derived from 303 persons presenting for emergency services who met criteria for uncomplicated mild head injury. Most data were obtained within 2 days of injury, and the remaining data were obtained within 10 days of injury. The norms may be used to help determine whether or not a person's scores on the WCST-64 are typical of those seen shortly after sustaining an uncomplicated mild head injury.
ABSTRACT The purpose of this chapter is to illustrate how an understanding of the psychometric pr... more ABSTRACT The purpose of this chapter is to illustrate how an understanding of the psychometric properties of tests, normative samples, and test scores are an essential foundation for meaningful and accurate clinical interpretations and reduces the likelihood of misinterpreting test results. Our goal is to present this information in an easy-to-understand format that facilitates clinicians’ knowledge of basic psychometrics in the context of test score interpretation. Clinical examples using commonly used tests will be provided throughout to illustrate the relevance and utility of these concepts in clinical practice. With regard to sample distributions, we will review concepts relating to non-normality and the influence of score distribution characteristics on derived scores. Floor and ceiling effects, equivalence of normative data sets, and truncated distributions will be discussed with regard to test items and test norms. When comparing scores between tests, we will review the role of test measurement error. We will also discuss normal variability and briefly comment on the prevalence of low test scores in healthy people, and how to use this information for supplementing clinical judgment. Finally, we will provide an overview of various methods for interpreting change in test performance over time.
The goals of the study were to determine how neuropsychological functioning is related to depress... more The goals of the study were to determine how neuropsychological functioning is related to depressive status in persons with head injury, and to quantify this relationship from a clinically relevant standpoint. Participants were 175 adults involved in litigation, referred for evaluation of suspected head injury. Depression status was measured using the Depression Content (Dep) scale of the MMPI-2. Depression status was related to measures of visual attention and psychomotor skills, but not to other neuropsychological domains such as verbal ability, visual-spatial reasoning, or encoding/organization. However, differences between low Dep and high Dep groups were minimal from a clinical standpoint. Depression appeared to contribute to an increased risk of impaired neuropsychological performance across domains, but only in persons not severely compromised by neuropsychological deficits. Overall, the results indicated a small effect of depression on neuropsychological functioning that is likely only detectable in persons whose neuropsychological compromise is relatively minimal.
Introduction Previous studies (IAT, PET, fMRI) demonstrate that early brain injury can lead to at... more Introduction Previous studies (IAT, PET, fMRI) demonstrate that early brain injury can lead to atypical functional organization of language [l-3]. In normal controls and recovering aphasics, language studies often reveal left, right or bilateral language dominance, usually involving homologous areas in both hemispheres [4]. Less frequently, one hemisphere is dominant for speech production, while the other is dominant for verbal comprehension [l]. We present a patient with unusual language organization that does not fit these patterns. Non-homologous bifrontal regions were activated on a verb generation task. Furthermore, verb generation and verb reading tasks revealed an interhemispheric diaschisis.
The purpose of this study was to evaluate the concurrent validity and clinical utility of a short... more The purpose of this study was to evaluate the concurrent validity and clinical utility of a short form of the Visual Form Discrimination Test in persons with closed head injuries (n = 62). Given the homogeneity of the items, and patients' apparent consistency in responding, we hypothesized that scores from an eight-item short form would be highly similar to the scores from the full version of the test (i.e. 16 items). The mean difference between the short form and full version was less than one point, and the correlation between the two forms was 0.86. Applying a clinical decision rule resulted in a 'normal/impaired' overall correct classification rate of 98.4%. The short form shows considerable promise as a brief test of visual discrimination in persons with closed head injuries.
Victoria Symptom Validity Test (VSVT) scores from six nonlitigants with neurological illness acco... more Victoria Symptom Validity Test (VSVT) scores from six nonlitigants with neurological illness accompanied by dense anterograde amnesia or severe memory impairment are presented. All of these patients obtained perfect or near perfect scores on the VSVT. These data add to the literature suggesting that the VSVT is insensitive to genuine neurologically-based memory impairment and provide an additional floor-level clinical benchmark against which to compare the performance of litigants.
Over the past 10 years, widespread and concerted research efforts have led to increasingly sophis... more Over the past 10 years, widespread and concerted research efforts have led to increasingly sophisticated and efficient methods and instruments for detecting exaggeration or fabrication of cognitive dysfunction. Despite these psychometric advances, the process of diagnosing malingering remains difficult and largely idiosyncratic. This article presents a proposed set of diagnostic criteria that define psychometric, behavioral, and collateral data indicative of possible, probable, and definite malingering of cognitive dysfunction, for use in clinical practice and for defining populations for clinical research. Relevant literature is reviewed, and limitations and benefits of the proposed criteria are discussed.
This study examined the effect of vagus nerve stimulation on quality of life in children with epi... more This study examined the effect of vagus nerve stimulation on quality of life in children with epilepsy using a validated quality-of-life scale and an empirical technique that accounts for measurement error in assessing individual change (the reliable change index). Participants were 34 children with severe intractable epilepsy who underwent vagus nerve stimulation and 19 children with intractable epilepsy who received medical management. Parent-completed epilepsy-specific and global ratings at baseline and after 1 year indicated that most children had no changes in quality of life following vagus nerve stimulation (52%-77%), similar to the comparison group. There was a trend for decreases to be less common in the vagus nerve stimulation group (14% vs 37%, P < .07), but there was no relation between improved quality of life and seizure control. The results raise questions about the mechanisms that underlie changes in quality of life after vagus nerve stimulation in this group of children.
Journal of Clinical and Experimental Neuropsychology, Oct 1, 2000
The present study used well-defined traumatic brain injury (TBI) and mixed neurological (other th... more The present study used well-defined traumatic brain injury (TBI) and mixed neurological (other than TBI) and psychiatric samples to examine the specificity and sensitivity to Malingered Neurocognitive Dysfunction (MND) of four individual California Verbal Learning Test (CVLT) variables and eight composite CVLT malingering indicators. Participants were 275 traumatic brain injury and 352 general clinical patients seen for neuropsychological evaluation. The TBI patients were assigned to one of five groups using the Slick, Sherman, and Iverson (1999) criteria: no incentive, incentive only, suspect, and malingering (both Probable MNDand Definite MND). Within TBI, persons with the strongest evidence for malingering (Probable and Definite) had the most extreme scores. Good sensitivity (approximately 50%) in the context of excellent specificity (> 95%) was found in the TBI samples. Issues related to the appropriate clinical application of these data are discussed.
Uploads