July, 2005 In Press, Neuropsychologia L-Dopa Impairs Learning, But Spares Generalization, in Parkinson’s Disease Daphna Shohamy1, Catherine E. Myers2, Kindiya D. Geghman3, Jacob Sage4, and Mark A. Gluck3 1 Department of Psychology, Stanford University 2 Department of Psychology, Rutgers University 3 Center for Molecular and Behavioral Neuroscience, Rutgers University 4 Department of Neurology, UMDNJ/Robert Wood Johnson Medical School, New Brunswick, New Jersey Corresponding Author: Daphna Shohamy Department of Psychology Stanford University Jordan Hall, Bldg. 420 Stanford, CA USA Tel: (650) 724-9515 Fax: (650) 725-5699 Word Count: 5617 Dopamine and Learning in Parkinson’s Disease ABSTRACT In this study we examined the effect of dopaminergic modulation on learning and memory. Parkinson’s patients were tested on vs. off dopaminergic medication, using a two-phase learning and transfer task. We found that dopaminergic medication was associated with impaired learning of an incrementally acquired concurrent discrimination task, while patients withdrawn from dopaminergic medication performed as well as controls. In addition, we found a dissociation of the effect of dopamine within a single two-phase task: patients tested on medication were not impaired at the ability to generalize based on learned information. The deficit among medicated patients appeared to be related specifically to the concurrent, incremental, feedback-based nature of the task: such a deficit was not found in a version of the task in which demands for concurrent error-processing learning were reduced. Taken together with a growing body of evidence emphasizing a role for midbrain dopamine in errorcorrecting, feedback-based learning processes, the present results suggest a framework for understanding previously conflicting results regarding the effect of medication on learning and memory in Parkinson’s disease. Keywords: Dopamine, Learning, Memory, Basal Ganglia, Cognition 2 Dopamine and Learning in Parkinson’s Disease INTRODUCTION Converging evidence suggests that the midbrain dopamine system plays an important role in learning and memory. Electrophysiological studies have shown that midbrain dopamine neurons may contribute to reward-related or noveltyrelated learning (Schultz et al., 1997; Schultz, 2000, 2002; Horvitz, 2000). Functional imaging studies in humans have also indicated a role for midbrain dopamine regions in several aspects of incremental learning, such as in the processing of reward, of expectancy of reward, and of error-correcting feedback (Poldrack et al., 2001; Delgado et al., 2000,2004; Knutson et al., 2001, Aron et al, 2004). Neuropsychological studies of patients with dopamine dysfunction have also shown that midbrain dopamine may play an important role in particular types of learning and memory. In Parkinson’s disease, there is a profound loss of dopamine-containing neurons in the substantia nigra compacta (SNc), leading to dopamine depletion in the striatum. Studies have shown that the loss of dopamine that occurs in Parkinson’s disease leads to a variety of learning and memory deficits, particularly on tasks that involve incremental, feedback-based learning of cue-outcome associations (Gotham et al., 1988; Canavan et al., 1989; Vriezen and Moskovitch, 1990; Knowlton et al., 1996; Swainson et al., 2000; Cools et al, 2001a, 2001b; Myers et al., 2003; Shohamy et al., 2004a, 2004b). By contrast, Parkinson’s patients are generally not impaired on performance of tasks which involve declarative, non-feedback based learning, or tasks that require 3 Dopamine and Learning in Parkinson’s Disease flexible use of knowledge (Knowlton et al., 1996; Myers et al., 2003, Shohamy et al., 2004a) - functions which are thought to rely on the medial temporal lobe (Squire and Zola, 1996; Eichenbaum, 2002; Gluck and Myers, 1993; Gabrieli, 1998; Robbins, 1996). Taken together, these findings imply that modulation of dopamine levels in Parkinson’s disease should have selective effects on learning and memory function depending on the specific task demands. Studies examining the effect of dopaminergic medication on cognitive function in Parkinson’s disease are generally consistent with this idea. Parkinson’s disease is most commonly treated with L-dopa, a dopamine precursor synthesized into dopamine in the brain leading to increased dopamine levels. Studies which specifically examined the effect of L-dopa treatment on cognition suggest that the effect of L-dopa depends on the specific task demands – with Ldopa sometimes remediating, sometimes having no effect, and sometimes impairing cognition (Gotham et al., 1988; Swainson et al., 2000; Cools et al., 2001a, Fournet et al., 2000; Mattay et al., 2002; Frank et al., 2004). However, most of these prior studies focused on ‘frontal’-like executive function tasks (such as working memory, planning, and set-shifting) and did not directly examine learning and memory per se. For example, Parkinson’s patients are impaired on the Tower of London task and associated spatial working memory tests, and Ldopa ameliorates this deficit (Lange et al., 1992; Owen et al., 1992; Owen et al., 1993). Overall, there is considerable evidence suggesting that L-dopa often improves cognitive performance on tasks that depend on ‘frontal’ executive or working memory processes, especially in mild to moderate Parkinson’s patients. 4 Dopamine and Learning in Parkinson’s Disease By contrast, less is known of the impact of L-dopa on learning and memory, and most studies reporting learning and memory impairments in Parkinson’s disease have tested only medicated patients (e.g., Canavan et al., 1989; Knowlton et al., 1996; Myers et al., 2003, Shohamy et al., 2004a, 2004b). Recent studies have begun to examine the effect of L-dopa on learning and memory. These have shown that L-dopa sometimes improves and sometimes worsens performance, depending on the specific task demands (Frank et al., 2004; Swainson et al., 2000; Cools et al., 2001; Czernecki et al., 2001). For example, Cools et al. (2001) demonstrated that L-dopa impaired performance on a probabilistic reversal task, but facilitated task-switching performance in the same patients. Frank et al. (2004) examined the effect of L-dopa on a reinforcement based learning task, and found that L-dopa impaired learning that was based on negative outcomes, but facilitated learning that was based on positive outcomes. These findings emphasize the fact that the effects of L-dopa can differ even within a single task, depending on highly specific modifications to the task demands. Understanding the circumstances under which L-dopa facilitates or impairs learning and memory is important not only from a clinical perspective, but could also potentially provide important insights into the neural mechanisms underlying the role of dopamine in learning and memory. In particular, electrophysiological studies demonstrate that midbrain dopamine neurons respond to behaviorally important stimuli in a temporally-specific, stimulus-specific manner: the signal occurs only in response to certain stimuli, and it is rapid and brief (Schultz et al., 1997; Schultz, 2002; Horvitz, 2000). These studies suggest that phasic dopamine 5 Dopamine and Learning in Parkinson’s Disease signals (as opposed to tonic, ongoing dopamine release) may be critical for learning that involves incremental acquisition of stimulus-outcome associations via error-correcting feedback. L-dopa, however, is thought to cause global increases in tonic dopamine levels in target areas, such as the neostriatum, consistent with recent pharmacological studies in rodents suggesting that L-dopa acts via nondopaminergic neurons (Miller & Abercrombie, 1999; Yamato et al., 2001; Tanaka et al., 1999). If midbrain dopamine signals are indeed critical for providing stimulus-specific, feedback-based information, enhanced levels of dopamine in the striatum coming from the ‘wrong’ neurons at the ‘wrong’ time may disrupt or mask critical stimulus-specific and temporally-specific signals essential for feedback-based error-correction learning. The purpose of the present study was to examine the effect of L-dopa on learning and memory in patients with mild to moderate Parkinson’s disease, using an incremental learning task. In this task, subjects are presented with a series of pairs of objects, and are required to learn to respond to the rewarded object in each pair. This task is similar to other incremental learning tasks previously shown to be impaired in Parkinson’s patients (e.g. Canavan et al., 1989; Myers et al., 2003). In addition, we sought to assess whether the effects of medication are specific to incremental learning. To that end, following acquisition, subjects were tested on a transfer/generalization phase, in which they were required to use what they have learned in the first phase to predict rewarded objects among a new set of stimuli. This kind of transfer has been shown to rely 6 Dopamine and Learning in Parkinson’s Disease on the medial temporal lobe (Myers et al., 2003; Eichenbaum et al., 1989; Preston et al., 2004), and is expected to be intact in patients with Parkinson’s disease. In addition, given that transfer is not based on trial-by-trial feedback, rather presumably on representational changes that occur over time, performance on the transfer phase would not be expected to be affected by Ldopa. Finally, we sought to assess which specific aspects of incremental learning might be most critical in contributing to learning deficits in Parkinson’s disease. Drawing on electrophysiological, modeling and neuroimaging evidence for the role of midbrain dopamine regions in error-correcting feedback-based learning, we hypothesized that L-dopa would impair learning processes that rely on such error-correcting feedback, but might spare learning that does not involve such processes. To that end, in Experiment 2 we manipulated the degree to which learning involved error-processing and compared learning under concurrent learning conditions, with learning of the same task in a shaping (reduced error) condition. We predicted that while the concurrent incremental learning task might be impaired with L-dopa, the reduced-error shaping version would be spared. Overall, we expected this study to shed light on the effect of L-dopa on incremental learning, on the degree to which this effect is specific to incremental feedback-based learning, as opposed to transfer, and the degree to which it is affected by error-correction processes. 7 Dopamine and Learning in Parkinson’s Disease Experiment 1 METHODS Participants Participants included 24 individuals with a diagnosis of idiopathic Parkinson’s disease, randomly assigned to be tested ‘on’ medication (n=12; 7 men, 5 women), or ‘off’ medication (n=12; 8 men; 4 women). Patients for this study were recruited from the Parkinson’s disease clinic at ColumbiaPresbyterian Medical Center (New York) and from the motor disorders clinic, Robert Wood Johnson University Hospital (New Jersey), having met diagnostic criteria for Parkinson’s disease as assessed by a neurologist and having given informed consent to participate. Parkinson’s patients were in the mild to moderate stages of the disease, with scores on the Hoehn-Yahr scale of motor function (Hoehn and Yahr, 1967) that ranged from 1 to 3 (in the ‘on’ state). All Parkinson’s patients were nondemented. Parkinson’s patients were also screened for clinical depression, as indicated by scores below 15 on the Beck Depression Inventory (Beck, Steer and Brown, 1996). All patients included in the study were treated with L-dopa, were stable on their medication doses for at least 3 months, and were responding well to the medication. Four subjects were also receiving treatment with dopamine agonists (two each in the off/on medication subgroups, either pramipexole or ropinirole). None of the patients were being treated with anti-cholinergic medication, nor with anti-depressants). Patients in the ‘on’ medication group 8 Dopamine and Learning in Parkinson’s Disease were tested within 2 hours since their last dose of medication. Patients tested ‘off’ medication had refrained from taking medication for a minimum of 16 hours. An equivalent number of age-matched healthy controls (n=12; 5 male; 7 female) were recruited and were screened for the presence of any neurological disorder or history of psychiatric illness including depression. Patient and control information is presented in Table 1. Controls did not differ significantly from the Parkinson’s ‘on’ or ‘off’ groups on age, education, or mini-mental state exam (MMSE; Folstein, Folstein and McHugh, 1975) [ANOVA, group ('on','off','control') by age, education, or MMSE, p>0.5]. All studies conformed to research guidelines established by Rutgers University and the Federal Government. Disease Duration PD on PD off Control Hoehn and Yahr (at test) 2.3 UPDRS (on) 6.1 Hoehn and Yahr (‘on’) 2.3 Age Education MMSE BDI 26.4 UPDR S (at test) 26.4* 64.5 16.0 29.6 6.8 (1.2) (0.1) (0.1) (4.4) (4.4) (1.5) (1.0) (0.2) (1.0) 6.7 2.2 2. 6 27.5 46.1* 62.1 16.8 29.4 6.7 (1.2) (0.2) (0.2) (3.8) (4.2) (2.3) (0.5) (0.4) (1.2) - - - - - 65.0 15.7 28.9 - (1.9) (0.9) (0.4) Table 1. Demographic and disease information for patients (PD) and controls (MMSE = Mini Mental State Exam; UPDRS = Unified Parkinson’s Disease Rating Scale; BDI = Beck Depression Inventory; Duration, age, and education in years. SE in parentheses). *Significantly different at p=0.05 level. 9 Dopamine and Learning in Parkinson’s Disease Behavioral Task General Description The task consisted of two phases. In Phase 1 (acquisition) subjects learned a concurrent discrimination. Subjects were presented with a series of pairs of objects, and on each trial were required to predict which of two objects was associated with reward. Each pair of objects differed in either color or shape, but not both, so that there was one relevant and one irrelevant dimension to the discrimination. In phase 2 (transfer), the pairs of objects continued to differ along the previously relevant dimension, but the irrelevant dimension changed. Sample trial events are shown in Figure 1. Apparatus Behavioral experiments were automated on an iBook computer programmed in the SuperCard language (Allegiant Technologies, San Diego, CA). Testing took place in a quiet room, with the participant seated in front of the computer at a comfortable viewing distance. The keyboard was masked except for two keys, labeled “LEFT” and “RIGHT” which the participant could press to record a response. 10 Dopamine and Learning in Parkinson’s Disease Stimuli The stimuli and procedures of Experiment 1 replicated those used in an earlier study (Myers et al., 2002). Phase 1 of the experiment was a concurrent discrimination. Stimuli consisted of sixteen colored shapes, organized into eight discrimination pairs. Four of the pairs differed in color (relevant feature) but not in shape (irrelevant feature); four pairs differed in shape (relevant feature) but not color (irrelevant feature). Within each discrimination pair, one stimulus was designated as rewarded. Assignments of particular color, shape and reward to discrimination pairs were made according to a pseudorandom procedure, but were held constant across the experiment. The full stimulus set is shown in Figure 2. Phase 2 of the experiment was a transfer test. Stimuli consisted of sixteen colored shapes which were partial recombinations of the shape and color features in phase 1: Each of the eight discrimination pairs was organized around the same relevant features as in phase 1; only the irrelevant features were novel. The features that were rewarded in phase 1 were also rewarded in phase 2. Thus, a set of response rules that emphasized the relevant features in phase 1 would perfectly predict the rewarded stimuli in phase 2. Alternatively, a set of response rules that emphasized the entire stimulus (including relevant and irrelevant features) in phase 1 would not transfer well to the new feature combinations in phase 2. Insert Figure 1 about here. 11 Dopamine and Learning in Parkinson’s Disease Procedure At the start of the experiment, the following instructions appeared on the screen: “Welcome to the experiment. You will see pairs of objects. Each time, there is a smiley face hidden under one of the two objects. It looks like this: . Find as many as you can.” The experimenter read these instructions aloud and then clicked the computer mouse button to begin phase one of the experiment. On each trial of phase 1, participants saw one of the eight discrimination pairs. Trials were organized into blocks, each containing 16 trials: one presentation of each discrimination pair in each possible left-right ordering. Trials in a block occurred in a pseudorandom but fixed order. Figure 1A shows screen events in a typical trial. Below the stimuli, a prompt appeared: “Which object is the smiley face under? Use the “LEFT” or “RIGHT” key to choose.” Participants then responded by pressing one of the two labeled keys. If it was the rewarded stimulus, a smiley face icon was revealed underneath and displayed for one second. The object then returned to its original position, obscuring the smiley face icon below. The objects were then removed and a new trial initiated. There was no limit on response time. Phase 1 continued until the participant completed 16 consecutive trials correctly, or for a maximum of 96 trials (6 blocks). Insert Figure 2 about here. As soon as phase 1 terminated, phase 2 began without any warning that task demands had shifted. The screen events were identical to phase 1 (Figure 12 Dopamine and Learning in Parkinson’s Disease 1B) except that the discrimination pairs were altered as described above. Again, trials were organized into blocks of 16 trials, one with each discrimination pair in each possible left-right ordering, in a pseudorandom but fixed order. Phase 2 continued until the participant completed 16 consecutive trials correctly, or to a maximum of 48 trials (3 blocks). The entire procedure, including phase 1 and phase 2, took approximately 15-20 minutes to complete. Data Collection On each trial, the computer recorded the discrimination pair, its left-right ordering, the desired response, and the participant’s response. For both phases, the total errors in each phase was recorded. RESULTS Acquisition (Phase 1) All healthy control participants, and all but one participant in the 'off' medication group, reached performance criterion of phase 1 within the 96 trial maximum. By contrast, seven participants in the ‘on’ medication group failed to 13 Dopamine and Learning in Parkinson’s Disease reach the performance criterion in phase 1. Overall, this was a significant difference [Chi-square, χ2 (2)= 13.8, p<0.001)]. Figure 3A shows the mean total errors for each group in phase 1. An analysis of variance (ANOVA) with phase 1 errors as the dependent variable and group (Parkinson’s ‘on’ med, Parkinson’s ‘off’ med, controls) as the independent variable revealed a significant difference in phase 1 performance [F(2,33)=9.3, p<0.001]. Post-hoc Tukey pairwise comparisons revealed that this effect was due to significantly more errors in the Parkinson’s ‘on’ medication group compared with either the Parkinson’s ‘off’ group (p<0.01) or the control group (p<0.001), while the Parkinson’s ‘off’ group did not differ significantly from the control group (p=0.7). There was no effect of gender, age, or motor score on performance (all p>0.05). Although too small a number of subjects in the PD ‘on’ group reached criterion on phase 1 (n=5) to allow separate statistical analyses of phase 1 performance in this subgroup, a comparison of the mean number of errors among this group suggested that, similar to those subjects that failed to reach criterion performance, they made more errors during acquisition than either the control or the PD ‘off’ group (mean number of errors 24.1, SE = 4.9). This subgroup of non-learners also did not differ substantially on any demographic or medication measures (no differences in age, education, MMSE, stage of disease, or years since onset; the distribution of gender and of subjects treated with agonists was the same in both subgroups). Insert Figure 3 about here. 14 Dopamine and Learning in Parkinson’s Disease Transfer (Phase 2) Following prior studies (Myers et al., 2002), phase 2 data from those subjects who failed to reach criterion performance in phase 1 were excluded from phase 2 analysis. This exclusion was necessary since any analysis of transfer phase performance is illogical for a participant who failed to learn the associations in phase 1. Indeed, those subjects who failed phase 1 also failed phase 2 (mean number of errors among non-learners was 37.0, SE=6.04). Among the remaining subjects (5 subjects tested ‘on’, 11 subjects tested ‘off’’, 12 control subjects), all subjects reached criterion performance in phase 2. Figure 3B shows that the mean phase 2 errors was similar among all groups; an analysis of variance (ANOVA) with phase 2 errors as the dependent variable found no significant effect of group (F (2,25)=0.56, p=0.6), no effect of subject gender, age, or motor score (all p>0.05). Experiment 1: Discussion Experiment 1 found that Parkinson’s patients tested ‘on’ L-dopa medication were significantly impaired on an incremental learning task. This impairment was not found in a group of matched patients who were tested while withdrawn from dopaminergic medication for approximately 16 hours; these ‘off’ medication patients learned the task as well as healthy controls. 15 Dopamine and Learning in Parkinson’s Disease This effect does not appear to be due to any general effects of L-dopa or L-dopa withdrawal on motor or cognitive functioning. Withdrawing patients from their medication in this manner does result in a temporary worsening of motor symptoms (as evidenced by the difference in motor scores, shown in Table 1). However, the effects of L-dopa do not appear to be due to general cognitive changes, since L-dopa has been previously shown to either enhance or impair cognitive function, depending on the task demands (e.g. Cools et al., 2001; Frank et al., 2004). Preliminary evidence suggests that the L-dopa related deficit was selective to the incremental acquisition phase of the task, and was not found for the transfer phase, where subjects were required to generalize what they had learned to a set of new stimuli. This result replicates previous findings on a similar task (Myers et al., 2003), which reported that Parkinson’s patients (medicated) were slow to learn, but those that did learn were able to transfer as well as control subjects (while individuals with hippocampal atrophy showed the opposite pattern). The present study extends these findings and suggests that the patients’ deficit on acquisition is associated specifically with the effects of medication, while medication does not impair transfer. However, examining transfer performance is dependent on the fact that subjects were able to reach criterion learning in phase 1. Because many medicated patients in the present study failed to reach criterion performance in phase 1, any conclusions regarding performance on the transfer phase are limited, given that it is not clear to what 16 Dopamine and Learning in Parkinson’s Disease degree the intact transfer performance might be biased by the fact that only phase 1 learners were included in the analysis. Experiment 2 aimed to address this issue, as well to gain a better understanding of the specific cognitive processes affected by L-dopa in Experiment 1. In particular, we sought to evaluate the extent to which the effect of L-dopa on learning is modulated by error-correcting feedback. To that end, we reduced the error load by developing a version of the task where subjects are shown the correct outcome to each pair, and then each discrimination pair is first trained to criterion prior to the introduction of the next pair (Experiment 2, Shaping condition), and we compared performance on this reduced-error version to a concurrent discrimination version (as in Experiment 1; Experiment 2, Concurrent condition). In addition, with the aim of gaining better insight into performance on the generalization/transfer phase, we reduced the memory load of the task in both conditions, to allow more subjects to reach criterion performance on phase 1. Experiment 2 As suggested by recent elecrophysiological, neuroimaging and neuropsychological studies, one possible interpretation of the L-dopa related impairment found in Experiment 1 is that L-dopa selectively impairs errorcorrecting, feedback-based learning processes. Prior neuroimaging studies have shown that while incremental trial-and-error learning depends on midbrain dopaminergic regions, learning the same information without error-correcting 17 Dopamine and Learning in Parkinson’s Disease feedback (i.e. by simply observing stimuli and outcomes) relies on distinct brain regions, particularly the medial temporal lobes (Poldrack et al., 2001; Aron et al., 2004). Consistent with these findings, we have shown recently that training Parkinson’s patients on an ‘observational’ version of a probabilistic learning task remediates learning impairments, while having no impact on performance among control participants (Shohamy et al., 2004a). Therefore, we hypothesized that modifying the present task by reducing demands for error-correcting feedback might alleviate the L-dopa related deficit. To evaluate the extent to which the effect of L-dopa on learning is modulated by the role of error-correcting feedback, we revised the concurrent discrimination task of Experiment 1 as follows. On the first trial with a new discrimination pair, the subject was shown the correct answer. Additionally, initial training was done by shaping; instead of interleaving all the discrimination pairs, subjects were trained on one pair to criterion (several consecutive correct responses), then a new pair was added and training continued until the subject reached criterion on both, and so on until all the pairs were learned. These changes were intended to reduce the need for trial-and-error learning and also to reduce the chances that subjects would "guess" incorrectly on their first trial with a new stimulus. 18 Dopamine and Learning in Parkinson’s Disease METHODS Participants Participants included 24 individuals with a diagnosis of idiopathic Parkinson’s disease tested ‘on’ medication and an equivalent number of agematched healthy controls, randomly assigned to participate in the Concurrent condition or the Shaping condition (n=12 for each group, each condition). Patient recruitment and screening procedures were identical to those described in Experiment 1, and subjects were taken from the same patient pool as Experiment 1. Because the intention of Experiment 2 was to explore the basis of the impairment found in Experiment 1 among Parkinson’s patients tested on medication, all patients in Experiment 2 were tested on medication. As in Experiment 1, all patients were being treated with L-dopa; a small number of patients were additionally treated with dopaminergic agonists (n=3 in the Concurrent condition; n=2 in the Shaping condition; either pramipexole or ropinirole). Patient and control information is presented in Table 2. There were no significant differences in age or education between the groups or the conditions [ANOVA with age or education as dependent variables and condition (Concurrent, Shaping) and group (Patients, Controls) as independent variables, all p>0.5]. 19 Dopamine and Learning in Parkinson’s Disease PD Concurrent PD Shaping Controls Concurrent Controls Shaping Parkinson’s Disease Duration 6.3 Hoehn andYahr UPDRS Age Educatio n MMSE BDI 2.0 25.5 65.0 16.3 28.8 7.2 (1.1) (0.2) (3.9) (3.0) (0.9) (0.4) (1.5) 5.9 2.1 24.2 63.4 16.3 28.5 7.1 (0.3) (0.1) (2.4) (1.7) (0.2) (0.1) (3.6) - - - 61.0 15.6 29.7 - (3.0) (0.5) (0.1) 64.5 16.4 28.9 (3.3) (0.7) (0.4) - - - - Table 2. Demographic and disease information for patients and controls (MMSE = Mini Mental State Exam; UPDRS = Unified Parkinson’s Disease Rating Score; BDI = Beck Depression Inventory; Duration, age, and education in years. SE in parentheses). Behavioral Task Apparatus & Procedure Concurrent condition In this condition, subjects were required to learn a concurrent discrimination task identical to that described in Experiment 1, except that this version required subjects to learn a reduced number of stimulus-outcome associations - 6 object pairs in the present experiment, compared to 8 in 20 Dopamine and Learning in Parkinson’s Disease Experiment 1. All other procedures were identical to those described in Experiment 1. Shaping condition In this condition, subjects were required to learn the same 6 stimulusoutcome associations as in the concurrent condition, but here the associations were learned using a shaping paradigm: For each stimulus, subjects first observed a single trial where they saw a pair of objects, and saw the correct answer revealed by the computer (without making a response; “observational” trial). Subsequently, the subject was presented with “standard” response-based trials, for that particular pair (for each trial, the subject responded “left” or “right” based on what they thought the correct object was, followed by responsecontingent feedback). After reaching a criterion of 4 correct consecutive responses (or a maximum of 12 trials) for each object pair, subjects were presented with a new pair introduced by a single observational trial, subsequently followed by response-feedback trials. For each sub-phase of this task, subjects were tested on the new pair, as well as on all previously learned pairs, gradually building up towards the full set of 6 pairs. Thus, importantly, the last phase of acquisition on the Shaping task was identical to all phases of the Concurrent task. All other procedures were identical across conditions. 21 Dopamine and Learning in Parkinson’s Disease RESULTS Acquisition (Phase 1) In the Concurrent condition, one control and six of the Parkinson’s patients failed to reach the performance criterion in phase 1. This was a near-significant difference [Chi-square, χ2 (1)= 3.23, p<0.07)]. By contrast, in the Shaping condition, all subjects in both groups reached criterion performance in phase 1. Figure 4A shows the mean errors for Parkinson’s patients and controls on acquisition of the Concurrent condition, compared with the Shaping condition. Consistent with our prediction, Parkinson’s patients were impaired at learning the Concurrent condition, but were not impaired at learning the Shaping condition. An ANOVA on number of errors (dependent variable) by group and condition (independent variables) revealed a significant main effect of condition [F(1,44)=25.52, p<0.001], a main effect of group [F(1,44)=6.26, p<0.05] and a significant group X condition interaction [F(1,44)=13.12, p<0.001]. Post hoc Tukey analyses confirmed that this was due to a significant difference between Parkinson’s and controls on the Concurrent condition (p<0.001), but not on the Shaping condition (p=0.8). Post hoc analyses of performance across conditions showed that Parkinson’s patients were significantly worse on the Concurrent condition compared to the Shaping condition (p < 0.001), whereas there was no difference between the conditions for control subjects (p = 0.7). Insert Figure 4 about here. 22 Dopamine and Learning in Parkinson’s Disease Transfer (Phase 2) Phase 2 data from those subjects who failed to reach criterion performance in phase 1 were excluded from phase 2 analysis. Among the remaining subjects (6 Parkinson’s patients and 11 controls on the Concurrent condition, 12 Parkinson’s patients and 12 controls on the Shaping condition), all subjects reached criterion performance in phase 2 for both conditions. Figure 4B shows that the mean phase 2 errors was similar among patients and controls and across conditions. An ANOVA on number of errors (dependent variable) by group and condition (independent variables) revealed no significant main effects or interactions [Main effect of group, F(1,37)=2.0, p=0.2; main effect of condition, F(1,37)=0.7, p=0.4; group by condition interaction, F(1,37)=0.04, p=0.8]. Experiment 2: Discussion In Experiment 2, we sought to evaluate the extent to which L-dopa related impairments on a concurrent learning task are modulated by error-correction processes. The findings from this experiment replicate those from Experiment 1, showing that medicated Parkinson’s patients are impaired at a concurrently trained incremental learning task even when the task involves reduced memory load. More importantly, we found that Parkinson’s patients were not impaired on 23 Dopamine and Learning in Parkinson’s Disease this task when they were trained on a ‘shaping’ version designed to involve reduced error-correcting processes. It is important to note that in both versions subjects ultimately learn to make the correct responses to an identical number of concurrent discriminations, with the final phases of the shaping version identical to the concurrent version. Thus, the critical difference between the conditions lies in the learning process, and the degree to which this process relies on trial-by-trial error processing. It seems unlikely that the differences between Parkinson’s patients’ performance on the two tasks are due to reduced loads in a general learning mechanism, given that performance among controls did not differ significantly between the two versions (and given that the errors among controls do not indicate a ceiling effect). In contrast to the L-dopa related learning impairment on acquisition, we found that both medicated and non-medicated patients performed normally on the transfer phase of the task, when they were required to generalize what they had learned to a novel context. These findings are consistent with recent electrophysiological and neuroimaging studies implicating midbrain dopamine in error-correcting feedback processes (e.g. Schultz, 2000; Aron et al., 2004). [add more discussion and refs to imaging papers]. These results are also consistent with recent studies with Parkinson’s patients demonstrating that while Parkinson’s patients are impaired on a feedback-based incremental learning task, they are not impaired on a nonfeedback ‘observational’ version of the same task (Shohamy et al., 2004a). It is 24 Dopamine and Learning in Parkinson’s Disease worth noting that in the prior study, the observational version eliminated stimulusdependent responding, in addition to feedback; in the present study, by contrast, subjects were still required to produce stimulus-related responses to learn the correct outcome. GENERAL DISCUSSION The present study found that Parkinson’s patients tested in a dopamine replete state, shortly after receiving dopaminergic medication, were impaired on an incremental learning task, while patients tested off medication, in a dopamine deplete state, performed as well as controls on the same task. These detrimental effects of dopaminergic medication were not found when subjects were required to learn same task in a ‘shaping’ version which involved decreased errorprocessing demands (while control subjects performed similarly under both conditions). Furthermore, the effects of medication were specific to learning: when the same subjects were challenged to generalize what they had learned to a novel context, Parkinson’s patients performed as well as controls, regardless of whether they were tested ‘on’ or ‘off’ medication. These findings suggest that Ldopa is associated with learning impairments which are selective to concurrent, feedback-based learning of incrementally acquired associations. The results of the current study provide behavioral evidence from humans that dopaminergic systems are critically involved in incremental learning. The 25 Dopamine and Learning in Parkinson’s Disease present findings converge with recent electrophysiological, computational and neuroimaging evidence for the role of midbrain dopamine systems in errorcorrecting, feedback-based learning processes (Schultz et al., 2002; Poldrack et al, 2001; Aron et al., 2004; Knutson et al., 2001). Specifically, recent imaging studies have suggested that the dorsal striatum, in particular (which is particularly affected by early stages of Parkinson’s disease) is important in reward and feedback-based learning (Haruno et al., 2004; Delgado et al., 2005; King-Casas et al., 2005). As such, our findings propose a framework for understanding previously conflicting results regarding the effect of L-dopa on learning and memory function in Parkinson’s disease. Specifically, based on the wealth of recent evidence indicating that the midbrain dopamine system plays an important role in stimulus-specific, feedbackbased learning, we have hypothesized that global increases in dopamine following L-dopa treatment may obstruct the learning-related temporally-specific, stimulus-specific dopamine signal in mild to moderately affected Parkinson’s patients, by providing the ‘wrong’ signal at the ‘wrong’ time. The results of the present study are consistent with this hypothesis, demonstrating an L-dopa related impairment on a concurrent, incremental learning task, which is alleviated when the error-processing, feedback-based demands of the learning task are reduced (despite the fact that both tasks involve learning identical sets of associations, and despite the fact that control subjects perform similarly on both tasks). Further, the detrimental effects of L-dopa are selective to the learning 26 Dopamine and Learning in Parkinson’s Disease phase of the task, and do not appear when the same patients are required to transfer what they have learned to a novel context. This pattern of impaired learning and spared transfer is exactly opposite to the pattern of impairments observed in individuals with damage to the hippocampal system on this task (Myers et al., 2002) and other tasks (e.g. Myers et al., 2003; Schacter, 1985). Thus, these findings fit in with recent evidence suggesting that cortico-striatal and hippocampal brain systems play distinct roles in learning and memory, with the cortico-striatal system contributing to incremental, stimulus-response learning, and the hippocampal system contributing to the formation of flexible, episodic, stimulus-stimulus representations (e.g. Gabrieli, 1998, Squire and Zola, 1996; Poldrack et al., 2001, Myers et al., 2003, Shohamy et al., 2004a). The present results further emphasize the role of dopamine in modulating incremental learning, and suggest that the incremental learning deficits found in Parkinson’s patients in prior studies (e.g. Knowlton et al., 1996; Myers, 2003; Shohamy et al., 2004a, 2004b) may be due, at least in part, to disruption of dopaminergic transmission with L-dopa, rather than the disruption of striatal function caused by the disease itself. The finding that L-dopa differentially impacts cognitive function depending on task demands is consistent with recent findings. Cools et al. have proposed that L-dopa mediated dopamine “overdose” may account for the differential effects of L-dopa on different attentional and executive function tasks, with Ldopa alleviating deficits in dopamine-depleted circuits, but causing impairments 27 Dopamine and Learning in Parkinson’s Disease in non-depleted circuits (Cools et al., 2001a). In support of this hypothesis, they found that L-dopa impaired probabilistic reversal learning but enhanced taskswitching performance. It is interesting to note that in the Cools et al. study, the two tasks differ not only in the neural circuitry they are presumed to rely on, but also in the kinds of learning processes they involve. In particular, while the probabilistic reversal (impaired with L-dopa) involves feedback-based learning that relies on temporally specific, stimulus-specific information, the task-switching ability (remediated with L-dopa) does not. It is worth emphasizing that these two hypotheses regarding the impact of L-dopa on cognitive function are not mutually exclusive and could both be factors in understanding how and where L-dopa improves or impairs cognitive function in Parkinson’s disease. In fact, given that L-dopa is provided systemically, one would expect effects on cognition to be mediated at the synaptic level within the midbrain, as well as more globally in widespread neural circuitry linking the striatum with frontal cortex. Examining the effects of L-dopa at both these levels will be critical for fully understanding why L-dopa sometimes impairs and sometimes facilitates performance. Several studies have reported positive effects of L-dopa on cognitive performance on varying tasks (e.g. Cools et al., 2001; Frank et al., 2004; Shohamy et al., 2005). We have previously reported positive effects of Ldopa on a sequence learning task, where subjects were required to learn to predict chains of events leading to reward (Shohamy et al., 2005). The dissociation of the effect of L-dopa on concurrent learning vs. sequence learning indicates that performance of these two tasks relies on dissociable cognitive and 28 Dopamine and Learning in Parkinson’s Disease neural processes. The present hypothesis taken together with the Cools et al. (2001) hypotheses regarding effects of L-dopa on frontal function suggest that one reason for this dissociation may be that the sequence learning task relies more heavily on frontal-based working memory and attention processes, as compared to the present task. Future studies will examine potential interactions between these two proposed consequences of L-dopa medication, as well as how such interactions may explain differences in the effect of L-dopa on different learning tasks. The present results are also generally consistent with a more recent report demonstrating differential effects of dopaminergic medication on positive vs. negative reinforcement-based learning (Frank et al., 2004; Frank et al., in press). This study used a different paradigm in which a series of probabilistic competitions were held between two alternative stimuli, one of which was always the winner, allowing separate analyses of positive vs. negative outcome based learning. This study found that Parkinson’s patients tested off medication were particularly impaired at learning from positive outcomes, compared to negative outcomes, while dopaminergic medication reversed this effect: patients tested on medication were particularly impaired at learning based on negative outcomes compared to positive outcomes. These findings are conceptually similar to those in the present study, demonstrating that Parkinson’s disease and dopaminergic medication interfere with patients’ ability to process feedback. Future studies are necessary to address the degree to which differential positive vs. negative based 29 Dopamine and Learning in Parkinson’s Disease learning may contribute to the effects of medication on the kind of incremental learning paradigms described here. Conclusions The present results suggest that dopaminergic treatment in Parkinson’s disease is associated with impairments to learning and memory function. The detrimental effect of dopaminergic medication can be understood in the context of the role of midbrain dopamine systems in reward-related, error-correcting incremental learning processes. As such, the present findings suggest a means by which to understand the varied pattern of facilitated vs. impaired learning processes in Parkinson’s patients following L-dopa treatment. In addition, the present results shed light on the differential contributions of different brain systems to learning and memory function, with a dopaminergic modulated cortico-striatal system contributing to incremental, error-correcting feedbackbased learning, and a medial temporal lobe system supporting formation of episodic, flexible, stimulus-stimulus representations. 30 Dopamine and Learning in Parkinson’s Disease Acknowledgments For his guidance and assistance with patient recruitment, the authors wish to thank Dr. Lucien Cote of the Columbia Presbyterian Medical Center. 31 Dopamine and Learning in Parkinson’s Disease REFERENCES Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA. Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 2004; 92: 1144-52. Beck AT, Steer, R.A., and Brown, G.K. Manual for the Beck Depression Inventory. San Antonio: Psychological Corp, 1996. Canavan AG, Passingham RE, Marsden CD, Quinn N, Wyke M, Polkey CE. The performance on learning tasks of patients in the early stages of Parkinson's disease. Neuropsychologia 1989; 27: 141-56. Cools R, Barker RA, Sahakian BJ, Robbins TW. Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex 2001; 11: 1136-43. Cools R, Barker RA, Sahakian BJ, Robbins TW. Mechanisms of cognitive set flexibility in Parkinson's disease. Brain 2001; 124: 2503-12. Delgado MR, Miller MM, Inati S, Phelps EA. An fMRI study of reward-related probability learning. Neuroimage 2005; 24:862-73. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 2000; 84: 3072-7. Delgado MR, Stenger VA, Fiez JA. Motivation-dependent responses in the human caudate nucleus. Cereb Cortex 2004; 14: 1022-30. 32 Dopamine and Learning in Parkinson’s Disease Eichenbaum H, Mathews P, Cohen NJ. Further studies of hippocampal representation during odor discrimination learning. Behav Neurosci 1989; 103: 1207-16. Eichenbaum H. The Cognitive Neuroscience of Memory. New York: Oxford University Press, 2002. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189-98. Fournet N, Moreaud O, Roulin JL, Naegele B, Pellat J. Working memory functioning in medicated Parkinson's disease patients and the effect of withdrawal of dopaminergic medication. Neuropsychology 2000; 14: 247-53. Frank MJ, Seeberger LC, O'Reilly R C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004; 306: 1940-3. Frank MJ. Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci In Press. Gabrieli JD. Cognitive neuroscience of human memory. Annu Rev Psychol 1998; 49: 87-115. Gluck MA, Myers CE. Hippocampal mediation of stimulus representation: a computational theory. Hippocampus 1993; 3: 491-516. Gotham AM, Brown RG, Marsden CD. 'Frontal' cognitive function in patients with Parkinson's disease 'on' and 'off' levodopa. Brain 1988; 111 ( Pt 2): 299-321. 33 Dopamine and Learning in Parkinson’s Disease Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, Kawato M. A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 2004; 24:1660-5. Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology 1967; 17: 427-42. Horvitz JC. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 2000; 96: 651-6. King-Casas B, Tomlin D, Anen C, Camerer CF, Quartz SR, Montague PR. Getting to know you: reputation and trust in a two-person economic exchange. Science 2005; 308:78-83. Knowlton BJ, Mangels JA, Squire LR. A neostriatal habit learning system in humans. Science 1996; 273: 1399-402. Knutson B, Fong GW, Adams CM, Varner JL, Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 2001; 12: 3683-7. Lange KW, Robbins TW, Marsden CD, James M, Owen AM, Paul GM. L-dopa withdrawal in Parkinson's disease selectively impairs cognitive performance in tests sensitive to frontal lobe dysfunction. Psychopharmacology (Berl) 1992; 107: 394-404. Mattay VS, Tessitore A, Callicott JH, Bertolino A, Goldberg TE, Chase TN, et al. Dopaminergic modulation of cortical function in patients with Parkinson's disease. Ann Neurol 2002; 51: 156-64. 34 Dopamine and Learning in Parkinson’s Disease Miller DW, Abercrombie ED. Role of high-affinity dopamine uptake and impulse activity in the appearance of extracellular dopamine in striatum after administration of exogenous L-DOPA: studies in intact and 6-hydroxydopaminetreated rats. J Neurochem 1999; 72: 1516-22. Myers CE, Kluger A, Golomb J, Ferris S, de Leon MJ, Schnirman G, et al. Hippocampal atrophy disrupts transfer generalization in nondemented elderly. J Geriatr Psychiatry Neurol 2002; 15: 82-90. Myers CE, Shohamy D, Gluck MA, Grossman S, Kluger A, Ferris S, et al. Dissociating hippocampal versus basal ganglia contributions to learning and transfer. J Cogn Neurosci 2003; 15: 185-93. Owen AM, James M, Leigh PN, Summers BA, Marsden CD, Quinn NP, et al. Fronto-striatal cognitive deficits at different stages of Parkinson's disease. Brain 1992; 115 ( Pt 6): 1727-51. Owen AM, Roberts AC, Hodges JR, Summers BA, Polkey CE, Robbins TW. Contrasting mechanisms of impaired attentional set-shifting in patients with frontal lobe damage or Parkinson's disease. Brain 1993; 116 ( Pt 5): 1159-75. Poldrack RA, Clark J, Pare-Blagoev EJ, Shohamy D, Creso Moyano J, Myers C, et al. Interactive memory systems in the human brain. Nature 2001; 414: 546-50. Preston AR, Shrager Y, Dudukovic NM, Gabrieli JD. Hippocampal contribution to the novel use of relational information in declarative memory. Hippocampus 2004; 14: 148-52. Robbins TW. Refining the taxonomy of memory. Science 1996; 273: 1353-4. 35 Dopamine and Learning in Parkinson’s Disease Schacter DL. Priming of old and new knowledge in amnesic patients and normal subjects. Ann N Y Acad Sci 1985; 444: 41-53. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science 1997; 275: 1593-9. Schultz W, Dickinson A. Neuronal coding of prediction errors. Annu Rev Neurosci 2000; 23: 473-500. Schultz W. Getting formal with dopamine and reward. Neuron 2002; 36: 241-63. Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA, Poldrack RA. Corticostriatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 2004; 127: 851-9. Shohamy D, Myers CE, Onlaor S, Gluck MA. Role of the basal ganglia in category learning: how do patients with Parkinson's disease learn? Behav Neurosci 2004; 118: 676-86. Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA. The role of dopamine in cognitive sequence learning: evidence from Parkinson's disease. Behav Brain Res 2005; 156: 191-9. Squire LR, Zola SM. Structure and function of declarative and nondeclarative memory systems. Proc Natl Acad Sci U S A 1996; 93: 13515-22. Swainson R, Rogers RD, Sahakian BJ, Summers BA, Polkey CE, Robbins TW. Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication. Neuropsychologia 2000; 38: 596-612. 36 Dopamine and Learning in Parkinson’s Disease Tanaka H, Kannari K, Maeda T, Tomiyama M, Suda T, Matsunaga M. Role of serotonergic neurons in L-DOPA-derived extracellular dopamine in the striatum of 6-OHDA-lesioned rats. Neuroreport 1999; 10: 631-4. Vriezen ER, Moscovitch M. Memory for temporal order and conditional associative-learning in patients with Parkinson's disease. Neuropsychologia 1990; 28: 1283-93. Yamato H, Kannari K, Shen H, Suda T, Matsunaga M. Fluoxetine reduces LDOPA-derived extracellular DA in the 6-OHDA-lesioned rat striatum. Neuroreport 2001; 12: 1123-6. 37 Dopamine and Learning in Parkinson’s Disease Figure Legends Figure 1. Stimulus set used for concurrent discrimination and transfer. Each pair of objects differed either by color or by shape. For transfer, the relevant dimension stayed the same, while the irrelevant dimension was changed. Figure 2. (A) Screen events on a sample trial of phase 1. Top: On each trial, the discrimination pair is presented in either left-right order and a prompt appears. Bottom: If the participant responds correctly (in this case the mushroom-like shape), the chosen object is raised to reveal a smiley face icon underneath. (B) Screen events on a sample trial of phase 2: events are similar to phase 1, but the objects are changed so that the relevant dimension (here the shape) is the same, whereas the irrelevant dimension (here the color approximated by grayscale) is novel. Figure 3. Performance on the incremental learning task described in Experiment 1; (A) Mean total errors (+/- SEM) on acquisition of the concurrent discriminations (phase 1); patients tested on L-dopa were impaired, but those tested off L-dopa were not (B) All groups performed equally well and made few errors on the transfer phase (phase 2). 38 Dopamine and Learning in Parkinson’s Disease Figure 4. Total errors (+/- SEM) on Concurrent versus Shaping conditions of the incremental learning task (A) Parkinson’s patients tested on medication were impaired on the Concurrent condition, but performed as well as controls on the Shaping version (B) Both groups performed equally well on the transfer phase of the task, for both the Concurrent and the Shaping conditions. 39 Dopamine and Learning in Parkinson’s Disease A. B. Phase 2 Phase 1 Figure 1 40 Dopamine and Learning in Parkinson’s Disease Discrimination pairs Probe Train vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. vs. Phase 3 retraining vs. vs. vs. Figure 2 vs. 41 Dopamine and Learning in Parkinson’s Disease B. Phase 1 40 40 35 35 30 30 25 25 Total # Errors Total # Errors A. 20 15 20 15 10 10 5 5 0 0 CON PD ‘on’ PD ‘off’ Phase 2 CON PD ‘on’ PD ‘off’ Figure 3 42 Dopamine and Learning in Parkinson’s Disease Phase 1 Phase 2 A. B. Controls Controls 40 PD 35 35 30 30 25 25 Total # Errors Total # Errors 40 20 15 20 15 10 10 5 5 0 0 Concurrent PD Shaping Concurrent Figure 4 43 Shaping