Journal of Management http://jom.sagepub.com/ If You Pay for Skills, Will They Learn? Skill Change and Maintenance Under a Skill-Based Pay System Erich C. Dierdorff and Eric A. Surface Journal of Management 2008 34: 721 originally published online 5 February 2008 DOI: 10.1177/0149206307312507 The online version of this article can be found at: http://jom.sagepub.com/content/34/4/721 Published by: http://www.sagepublications.com On behalf of: Southern Management Association Additional services and information for Journal of Management can be found at: Email Alerts: http://jom.sagepub.com/cgi/alerts Subscriptions: http://jom.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://jom.sagepub.com/content/34/4/721.refs.html Downloaded from jom.sagepub.com by guest on July 26, 2010 If You Pay for Skills, Will They Learn? Skill Change and Maintenance Under a Skill-Based Pay System† Erich C. Dierdorff* Department of Management, Kellstadt Graduate School of Business, DePaul University, 1 East Jackson Blvd., Suite 7000, Chicago, IL 60604 Eric A. Surface SWA Consulting Inc., 3901 Barrett Drive, Suite 200, Raleigh, NC 27609 Although the use of skill-based pay has increased in popularity, empirical investigations of the effectiveness of this compensation strategy have been scarce. The fundamental premise of skillbased pay is that contingent monetary reward will promote individual learning. The authors empirically examine this essential principle with data spanning 5 years, using latent growth analysis. Results demonstrate that skill-based pay is related to individual skill change and maintenance. Whether or not individuals earn skill-based pay on their initial attempt is associated with subsequent rates of learning. In addition, the frequency with which skill-based pay is received and the total amount earned are both associated with skill development and maintenance. Keywords: skill-based pay; person-based pay; skill change; latent growth analysis Since codification in the Equal Pay Act of 1964, the notion of skills as compensable factors has been widely recognized. More recently, many organizations have begun to shift from exclusive use of job-based compensation systems to incorporating person-contingent †We sincerely thank Jack Donnelly for his support of this article. The views, opinions, and findings contained in this article are solely those of the authors and should not be construed as an official U.S. Army Special Operations Command, Department of Army, or Department of Defense position, policy, or decision. *Corresponding author: Tel.: 312-362-8495; fax: 312-362-6973 E-mail address: edierdor@depaul.edu Journal of Management, Vol. 34 No. 4, August 2008 721-743 DOI: 10.1177/0149206307312507 © 2008 Southern Management Association. All rights reserved. 721 Downloaded from jom.sagepub.com by guest on July 26, 2010 722 Journal of Management / August 2008 approaches (Mahoney, 1989). The motivation for these modifications is to enable organizations to better deal with changes in the contemporary world of work that now require greater emphasis on strategic human resource management (Heneman, Ledford, & Gresham, 2000). Of these person-based pay systems, one of the most developed is skill-based pay (SBP), which can facilitate implementation of business strategy and help build high-performance organizations (Lawler, 2000; Lawler & Jenkins, 1992). Put simply, SBP approaches are base wage systems that structure pay levels and differentials relative to the skills employees possess or demonstrate (Jenkins, Ledford, Gupta, & Doty, 1992). Thus, pay in these systems is contingent on what an individual is capable of performing and not simply the specific work role that is occupied (Hills, 1989). Within compensation practice, some estimates indicate SBP use in almost two thirds of Fortune 1000 companies (Lawler, Mohrman, & Ledford, 1998), whereas other, more conservative estimates suggest use in 20% to 30% of companies (e.g., Gross & Griffith, 2003). Much of the popularity of SBP can be attributed to the benefits frequently touted within practitioner-oriented journals. For example, these sources have purported that by using SBP organizations will reap such desirable outcomes as a flexible workforce, lowered labor costs, and increased quality and productivity (Bunning, 1989; Gupta, Jenkins, & Curington, 1986; Johnson & Ray, 1993; Lawler & Ledford, 1987; Shareef, 1994). Employees also seem to hold generally favorable attitudes regarding the use of SBP (Heneman & Ledford, 1998). However, with very few exceptions (e.g., Lee, Law, & Bobko, 1999; Shaw, Gupta, Mitra, & Ledford, 2005), the majority of evidence regarding the impact of SBP comes from exploratory surveys (e.g., Jenkins, et al., 1992) and anecdotal cases (e.g., Ledford & Bergel, 1991), both of which are limited in the presentation of empirical data (Murray & Gerhart, 2000). One issue that to date has yet to receive any empirical treatment is whether or not SBP is actually associated with individual skill change. In fact, individual-level outcomes of SBP have been largely absent from previous research, as studies have favored examinations of outcomes at the broader organizational level. This paucity is surprising considering that the quintessence of any SBP system is the motivation of skill change, specifically through the use of contingent wage increases (Gerhart & Milkovich, 1992; Murray & Gerhart, 1998). Such purpose is exemplified by the often-used slogan of SBP, “learn to earn.” Furthermore, the fundamental premise of SBP is that developing the system’s targeted skills will subsequently transfer to on-the-job improvements (Lawler, 2000). However, the extent to which SBP is associated with skill development remains an unanswered and uninvestigated question. Murray and Gerhart (2000) echo the untested nature of this SBP supposition by stating “the assumption seems to have been—if you pay for it, they will learn” (p. 272). The present study seeks to empirically address the issue of whether or not SBP is related to individual skill change. Over a 5-year period, we analyze changes in skill proficiency occurring both within individuals and between individuals. Incorporating aspects of skill change that occur within and between persons is essential to SBP research, as rates of skill development, skill maintenance, and the effects of SBP can be expected to substantially vary across individuals (Ackerman, 1987; Murray & Gerhart, 2000). Because so little empirical attention has been given to SBP, this research not only represents a significant contribution to the literature but also provides data vital to evidence-based compensation practice (Rynes & Gerhart, 2000). Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 723 Skill Proficiency Under SBP Systems Pay is a significant component of workplace motivation and performance (Lawler, 1971). In SBP systems, the use of pay is explicitly intended to motivate work behavior toward the development of select skills deemed important by the organization. The responsibility for learning these skills falls largely on employees themselves, with organizations primarily providing support for skill development and maintenance (e.g., formal training) and contingent reward for skill demonstration. Research by Murray and Gerhart (1998) examined the outcomes of SBP and may offer some indirect evidence of skill change facilitation, assuming that plant-level productivity gains and quality improvements were made possible through newly developed skill proficiency. However, such an assumption does not allow one to draw conclusions as to whether SBP is associated with individual skill change, only that the introduction of SBP is in some way related to overall outcomes. Thus, important questions remain, such as does earning SBP relate to subsequent skill change? Does the size of an SBP increase affect skill changes? And what are the effects of early success at earning SBP? The premise on which SBP systems are predicated is that contingent monetary reward will reinforce learning. This premise is firmly rooted in behavioral management theory (Stajkovic & Luthans, 2003), in which learning and behavior are viewed as being significantly influenced by contingent consequences. Indeed, using contingent rewards to motivate learning and other desired workplace behaviors has been long recognized as an effective human resource and leadership practice (e.g., Farr, 1976; Judge & Piccolo, 2004; Luthans & Kreitner, 1975, 1985; Podsakoff, Bommer, Podsakoff, & MacKenzie, 2006; Podsakoff, Todor, & Skov, 1982). Contingently applied monetary rewards are also one of the most widely recognized reinforcers of work-related behavior (Rynes & Gerhart, 2000; Stajkovic & Luthans, 2001), with metaanalytic evidence of effects on subsequent performance (Stajkovic & Luthans, 1997, 2003). In short, the use of contingent monetary rewards to motivate employee behavior is both theoretically grounded and a highly reliable management practice (Pfeffer, 1995). As a form of contingent reward, there are a variety of reasons to believe that implementing SBP would indeed fulfill the system’s central purpose of motivating changes in skill proficiency. For example, identifying and incorporating particular skills and proficiency levels within an SBP system, and communicating these details to employees, provides valuable information regarding the desired “direction” of work behavior (Martocchio, 2006). Direction is a key element of work motivation, serving to focus an individual’s attention and effort (Kanfer, 1990; Locke & Latham, 1990). In the compensation literature, such directional information is often referred to as “line of sight” or “line of influence” (Heneman et al., 2000; Lawler, 2000) and encompasses the connection between behavior and reward. Thus, to the extent that SBP systems define desired skills, levels of proficiency, and the rewards associated with skill change, they should serve as motivating forces that promote individual skill development. The training literature also purports the importance of environmental characteristics on learning, transfer, and maintenance (Goldstein, 1986). Here again, SBP systems not only provide information delineating the skills and proficiency levels that are valued by the organization but also reward continued learning, transfer, and maintenance of these valued skills. Individuals will often display rather different patterns of skill change and maintenance over time, even for those persons with highly similar rates of change in the short term (Ford & Downloaded from jom.sagepub.com by guest on July 26, 2010 724 Journal of Management / August 2008 Kraiger, 1995). In training parlance, such longitudinal skill changes are typically termed “learning curves” (Goldstein & Ford, 2002; Wexley & Latham, 1991) or “maintenance curves” (Baldwin & Ford, 1988). Relevant to compensation, these learning or maintenance curves are affected by organizational reward systems. For example, the lack of rewards within a particular work environment is likely to result in lower motivation for maintaining or developing one’s skills (Baldwin & Ford, 1988). Because SBP systems reinforce particular levels of demonstrated skill proficiency with monetary rewards, the motivational impact of this pay strategy should be evident in differential learning or maintenance curves across individuals. For example, behavioral management theory suggests that pay increases from SBP heighten the probability that employees earning such increases will subsequently engage in behavior related to further skill development or maintenance (i.e., SBP serving as reinforcement). Moreover, earning or failing to earn in SBP provides direct feedback regarding an individual’s performance capability. Empirical evidence shows that such performance feedback shapes skill acquisition and is more salient during early attempts at behavior (Mitchell, Hopper, Daniels, George-Falvy, & James, 1994). Thus, an individual’s initial attempt at demonstrating the requisite skill proficiency for SBP will provide early feedback regarding performance capability. The implication for SBP and longitudinal skill change (learning or maintenance curves) is that individuals who experience early success in terms of demonstrating skill proficiency at the level requisite for SBP are more likely to continue skill development than those who do not demonstrate such proficiency. In this sense, early earning of SBP should lead to ensuing motivation to change or maintain one’s skills. Hypothesis 1: Initial earning of SBP is positively associated with rates of skill change. Another implication for SBP and employees’ skill changes is that those individuals who repeatedly demonstrate requisite skill proficiencies that lead to increased pay should be further motivated to continue their skill development. That is, to the degree that an SBP system allows for and rewards skill change, individuals who continually qualify and receive SBP should display higher rates of skill growth and maintenance than individuals who improve skills but do not qualify for SBP increases. Empirical support for such effects can be seen in the training literature, in which extrinsic reinforcement consequences (e.g., pay) and feedback have been shown to affect the transfer and maintenance of learning within the work environment (Rouiller & Goldstein, 1993; Tracey, Tannenbaum, & Kavanagh, 1995). Thus, the more often individuals earn SBP, which reinforces and provides positive feedback regarding skill change, the more likely these individuals are to continue their skill development or maintenance. Hypothesis 2: The frequency of SBP is positively associated with rates of skill change. In addition to providing monetary rewards contingent on skill demonstration, another fundamental design feature of SBP systems is that pay increases are differentially sized depending on the type or proficiency level of the focal skill. Thus, as employees acquire a greater number of skills or become more proficient in a particular skill, they are rewarded with larger amounts of SBP (Bunning, 1992; Lawler & Ledford, 1987). The underlying rationale for these Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 725 pay differentials is to motivate further pursuit of skill development and to reward higher levels of skill proficiency. This design feature suggests that not only will the frequency with which an individual receives SBP be important to skill change and maintenance, but the amount of SBP that an individual receives will also be influential. Because greater skill proficiency is associated with higher levels of pay under SBP systems, motivation to further one’s skill development is likely to ensue. In this regard, employees who receive larger amounts of contingent reward under SBP systems are more likely to display greater rates of skill growth. Hypothesis 3: The amount of SBP is positively associated with rates of skill change. Method Description of the Organization and SBP System Participants in the current study were members of the U.S. Army Special Operations Forces (SOF), including soldiers in Special Forces, Civil Affairs, and Psychological Operations units. Army SOF is a large military organization whose personnel routinely conduct a variety of specialized training and operational missions around the globe as part of regionally focused units. A critical component to the successful achievement of the organization’s strategic goals is the foreign language capability of its personnel (U.S. Department of Defense, 2006). Foreign language acquisition is an essential element of training for Army SOF personnel, and most units are required to have multiple language capabilities associated with their regional focus (e.g., South America). Recognizing the importance of foreign language skills for effective performance in multiple career fields, the Department of Defense utilizes a SBP system specifically intended to motivate the development and maintenance of linguistic skills and to align human resource management with organizational strategy.1 The SBP plan consists of three pay blocks that represent three levels of language skill, with each level indicating greater proficiency in a particular foreign language. Individuals are permitted to earn SBP for skill proficiency in more than one foreign language. The pay policy in place at the time of this research provided SBP as monthly increases to base pay from $100 to $150 to $200 depending on the demonstrated skill proficiency. To monitor foreign language skills and to administer the SBP system, the Department of Defense conducts formal proficiency testing. Individuals receiving military-sponsored language training are required to have their skills assessed and certified initially following training and recertified on an annual basis. Individuals who learn a foreign language on their own may request to be tested and qualify for SBP as well. Of importance, this certifying assessment utilizes an objective and established measure of the focal skills, which is an essential feature of effective SBP systems (Lawler, 2000). Although active-duty Army SOF personnel receive some initial formal language training, the degree to which individuals pursue additional skill change varies because it is typically at their discretion; that is, some individuals may chose to increase skills in a given language or across multiple languages, whereas others may not. Both traditional instructional resources, such as scheduled classes or tutors, and nontraditional instructional resources, such as language learning software or language labs, are available to personnel at their units. For example, the Downloaded from jom.sagepub.com by guest on July 26, 2010 726 Journal of Management / August 2008 Department of Army has made language learning software available to all Army personnel through the Army Knowledge Online portal Web site. Sample and Measures Archival data describing individuals’ language skills and SBP records across their first five annual skill assessments were examined in the current study. Participants were tested approximately each year using the same version of the proficiency test for their assigned language. The archival data set spanned 7 years (1998 to 2005) because SOF personnel are typically deployed for unspecified periods, often precluding precise testing dates across individuals. Thus, because of operational assignments, there was some variability in the exact dates of the “annual” skill assessments. This variability was very small, as all individuals were assessed within 10- to 14-month periods (i.e., annually, plus or minus 2 months). The archival data set provided a full sample of 2,592 individuals tested in 1998. Individuals for whom five skill assessments were unavailable were excluded from the sample (n = 487, 19%). The final sample consisted of 2,105 individuals. The excluded sample was not significantly different (p > .05) from the final sample with respect to skill proficiency or other study variables. All sample participants received some formal initial language training and were Army SOF personnel. In addition, every participant was eligible for SBP under the system and occupied a work role in which foreign language requirements were relevant to performance. Important to note is that participants are classified as “language-enabled personnel” and not linguists (e.g., translators, interpreters, etc.). In other words, foreign language skills enhance the performance of their work roles, as opposed to linguists, for whom such skills compose their primary responsibilities. The majority of participants were male (greater than 95%). In terms of personnel classifications, 78% were noncommissioned officers, 11% were warrant officers, and 11% were officers. As mentioned earlier, the compensation policy allows individuals to earn SBP in multiple languages. However, such individuals represented a very small percentage of the sample, with those testing in two SBP-eligible languages composing less than 4.0% of the sample and those testing in more than two SBP-eligible languages composing less than 0.5%. Skill proficiency. Foreign language proficiency was based on scores on the Defense Language Proficiency Test (DLPT).2 The DLPT is an instrument designed by the Defense Language Institute to measure listening and reading skill proficiency and is divided into two assessment components (listening and reading tests). Higher scores on each DLPT component indicate greater skill proficiency. The organization bases SBP wages on both reading and listening proficiency scores. Thus, a composite DLPT proficiency score was computed for each individual as the average of DLPT reading and listening scores and was used to model skill change across the study’s time period. Composite scores in our sample ranged from 10 to 60. Skill blocks. Specific DLPT scores are organized into seven broader ratings of language proficiency: 0, 0+, 1, 1+, 2, 2+, and 3 (for more details on these ratings, see Silva & White, 1993). Of these seven proficiency ratings, the organization designates 2, 2+, and 3 as cutoffs Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 727 for three SBP pay blocks. Specific DLPT scores that range from 40 to 45 translate to the broader proficiency rating of 2, scores ranging from 46 to 49 translate to a rating of 2+, and scores greater than 50 translate to a rating of 3. To receive SBP, individuals are required to qualify for the minimum skill block cutoff indicated by a rating of 2/2 for listening and reading, respectively (i.e., DLPT scores ranging from 40 to 45). The next skill block cutoff is indicated by a rating of 2+/2+. The highest skill block is recognized with a proficiency rating of 3/3. These skill blocks we used to operationalize the three variables described next. It is very important to note that skill change can occur regardless of earning or not earning SBP. For example, a person could significantly increase skill proficiency from DLPT scores of 40 to 45 (i.e., 2/2 proficiency rating) but not qualify for the next higher SBP skill block (i.e., 2+/2+ proficiency rating). Initial application of SBP. This variable was operationalized as a dichotomous score representing whether or not individuals earned SBP on their first attempt that was based on their initial skill assessment. This variable was coded as 0 (no SBP earned; i.e., proficiency rating < 2/2) and 1 (SBP earned; i.e., proficiency rating 2/2, 2+/2+, or 3/3). Frequency of SBP. The frequency with which individuals received SBP across the allotted period was operationalized as the number of times an individual qualified for SBP by demonstrating at least the minimum requisite skill proficiency (i.e., 2/2 proficiency rating). Only the 2nd, 3rd, and 4th years were used to operationalize this measure for two reasons. First, our primary interest resided in testing whether or not earning was associated with subsequent skill change. Thus, using the skill assessment from the 5th year would necessitate an additional 6th assessment. Second, we chose not to incorporate the 1st year into this measure because earning SBP during one’s initial attempt was the specific focus of Hypothesis 1. No differentiation was made with regard to the level of the skill block attained, but simply whether or not SBP was received during the 2nd through 4th years. It is important to reiterate that skill change can occur independently of earning SBP and that such change is rewarded with SBP only when proficiency levels exceed one of the skill block cutoffs (e.g., 2+/2+). Individuals who qualify for SBP receive the monetary reward for 1 year and must be recertified to again qualify. Amount of SBP. The amount of SBP that an individual received across the allotted period was operationalized as the sum of skill block levels across the 2nd, 3rd, and 4th yearly time points (skill assessments). To reiterate, there are three compensable skill blocks for a given language (i.e., 2/2, 2+/2+, and 3/3). Thus, values for this measure could range from a low of 1, when a person qualifies only once for SBP and at Skill Block 1, to a high of 9, when a person qualifies for SBP at Skill Block 3 during Years 2, 3, and 4. For the same reasons as the frequency of SBP measure, only the 2nd, 3rd, and 4th years were used to operationalize the amount of wage increase because of SBP. Again, skill change can occur regardless of the amount of SBP earned, but such skill change is rewarded with larger amounts of SBP when greater proficiency levels are attained (e.g., an increase from a rating of 2/2 to a rating of 2+/2+). Downloaded from jom.sagepub.com by guest on July 26, 2010 728 Journal of Management / August 2008 Control Variables Language difficulty. We controlled for the level of difficulty inherent to learning various languages using a four-category government classification system. These categories reflect the increasing difficulty of a native English speaker to learn the focal language (Silva & White, 1993; Surface, Dierdorff, & Donnelly, 2004). For example, Spanish is classified as a Category I language, German as a Category II, Russian as a Category III, and Arabic as a Category IV. Approximately 49% of participants sought skill development in Category I languages, 11% in Category II languages, 23% in Category III languages, and 17% in Category IV languages. This system is frequently used for military and educational guidelines and policy. For example, the American Council on Education uses language difficulty in part to create recommendations for awarding college credit for some language proficiency (Surface & Dierdorff, 2003). Cognitive ability. General cognitive ability plays a substantial role in both successful learning (Ree, Carretta, & Teachout, 1995; Ree & Earles, 1991) and job performance (Hunter & Schmidt, 1998; Ree, Earles, & Teachout, 1994). Thus, it was important to control for individuals’ levels of cognitive ability when examining skill change. General cognitive ability was assessed using the Armed Forces Qualification Test, which provides a composite score consisting of two verbal and two math tests from the ASVAB (Ree & Carretta, 1994). This test is commonly used as a measure of general cognitive ability (Carretta & Doub, 1998). Base salary. Base salaries varied across participants, but, as described above, the amounts of SBP awarded for each skill block remained constant regardless of salary. Base salary differentials could affect the perceived value of an SBP increase. We controlled for base salary using pay grade data. The U.S. Army uses a standardized pay grade system that defines ranges of base salary for noncommissioned officers, warrant officers, and officers. Integrating these pay grades across personnel types created 17 levels of base salary across the participants. Level of education. Individuals with more exposure to formal education might be more comfortable in learning settings, such as the language training opportunities made available by the organization. We controlled for participants’ levels of education, which were assessed with seven response categories. These categories and associated sample percentages were as follows: no high school diploma (< 1%), high school diploma (56%), some college (19%), bachelor’s degree (17%), master’s degree (6%), doctorate (< 1%), and not available (1%). Occupation specialty. This variable was meant to control for any broad differences between participants’ job responsibilities or contexts that could potentially influence skill change. Examples of the occupation specialties within the sample include psychological operations specialists, civil affairs specialists, Special Forces engineering sergeants, Special Forces communications sergeants, and Special Forces medical sergeants. A total of 10 delineated categories composed the sample, with an additional category of “not elsewhere classified” for individuals with nonspecified military specialties (less than 9% of the sample). Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 729 Analytic Strategy Because earning SBP was directly predicated on skill proficiency, we needed to first establish temporal precedence for the potential association of SBP-related variables with subsequent skill change. To accomplish this, we conducted repeated multiple regression analysis. Eight separate regression models were conducted corresponding to each yearly time frame (e.g., Year 1 to Year 2, Year 2 to Year 3, etc.), with four models examining the effects of simply earning SBP or not and the other four models examining the effects of the amount of SBP earned. In each of these models, the dependent variable was the subsequent year’s skill proficiency (e.g., in the Year 1 to Year 2 models, the dependent variable was Year 2 skill proficiencies). Independent variables in these models were the five study control variables, an additional control variable of individuals’ prior-year skill proficiency (e.g., in the Year 1 to Year 2 model, individuals’ skill proficiency at Year 1 was entered as a control), and the SBP-related variable. Controlling for individuals’ prior-year skill proficiency allowed for a more comprehensive examination of the impact of earning SBP, and in what amount, on subsequent skill change by holding constant the skill level at which individuals entered each yearly time frame. To test our three hypotheses regarding the association of SBP with changes in skill proficiency, we used latent growth modeling (LGM). This structural equation modeling technique provides a unified and flexible framework to analyze aspects of change over time (Chan, 2002). LGM overcomes many of the problems that characterize traditional approaches to analyzing longitudinal data, such as repeated measures ANOVA, time series analysis, and repeated measures regression (for comparisons, see Chan, 1998; Duncan, Duncan, Strycker, Li, & Alpert, 1999). Particular to our study, LGM allows us to overcome limitations associated with the repeated multiple regressions used to establish temporal precedence of SBP on subsequent skill proficiency. Such limitations primarily stem from incorrect specifications of measurement error in these regression models and violations of the independence assumption of error terms across repeated measurements that are typical of multiwave designs (Chan, 2002). A specific strength of LGM analysis is that it simultaneously examines both between individual and within individual changes in skill development across time (Hofmann, Jacobs, & Barrata, 1993; Hofmann, Jacobs, & Gerras, 1992; Willett & Sayer, 1994). In other words, LGM can identify and describe the nature of intraindividual change over time and examine interindividual differences in these change patterns. As previously mentioned, rates of skill change can be expected to substantially vary across individuals (Ackerman, 1987), as will the motivational impact of SBP on skill change (Murray & Gerhart, 2000). These characteristics make LGM an appropriate technique by permitting a more comprehensive examination of the potential influence of SBP on individuals’ changes in skill proficiency. Our analyses followed common LGM conventions (e.g., Muthen, 1991) and proceeded in two broad stages. The purpose of the first stage was to accurately and parsimoniously model growth (i.e., longitudinal changes in skill proficiency). Determining the most appropriate model to describe growth is a crucial and initial step in LGM analysis. Two trajectories of change were tested: linear growth and quadratic growth. A linear trajectory is characterized by steady change (gain or loss) over time, whereas a quadratic trajectory represents the amount of curvature (acceleration or deceleration) in trajectory beyond linear change. Also Downloaded from jom.sagepub.com by guest on July 26, 2010 730 Journal of Management / August 2008 Figure 1 General Latent Growth Modeling Model of Skill-Based Pay Predicting Skill Change Skill-based Pay Intercept Slope linear 0 1 1 LP1 0 1 1 1 1 LP2 Slope quad 2 3 LP3 4 1 4 9 LP4 16 LP5 Note: LP1 to LP5 = language proficiency Time 1 to Time 5. Uniqueness and disturbance terms are omitted for clarity. modeled in this stage were two types of error structure: homoscedastic (error variances equal across measurements) and heteroscedastic (error variances freely estimated). A benefit of correctly modeling error structure is that more accurate significance tests are enabled (Willett & Sayer, 1994). Of importance, results from this stage allowed for proper specification of longitudinal skill changes across participants. Using the appropriate change model specified in the first stage, the second stage of analysis tested three conditional LGM models corresponding to each hypothesis. This was accomplished by introducing SBP-related variables (e.g., frequency of SBP) to account for variance in skill change trajectories. Figure 1 depicts an example of a tested conditional LGM model. The purpose of this stage of analysis was to examine whether or not SBP Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 731 accounted for individual differences in intraindividual skill change, in other words, testing the extent to which each SBP variable accounted for differences in rates of skill growth between individuals. Results Table 1 presents means, standard deviations, and correlations for study variables. Language difficulty was significantly and inversely related to language skill proficiency at all five time points. Levels of base salary and education were significantly and positively related to the five skill proficiency assessments. Although significant, correlations between cognitive ability and skill proficiency were relatively low (r = .06), perhaps because of range restriction as cognitive ability is one aspect on which Army SOF are selected. The repeated measures of language skill proficiency were also significantly related. These interrelationships decreased as the time between specific assessments increased (e.g., Time 1 vs. Time 2 vs. Time 5). In addition, the SBP-related variables were significantly related to skill proficiency. Eight regression models were used to determine temporal precedence for SBP-related variables on subsequent skill change. The dependent variables in the models were subsequent year skill proficiency levels. The first four multiple regression models were conducted to determine the effect of earning or not earning SBP. Here, results showed that earning or not earning SBP explained significant variance in ensuing skill proficiency, even after entering the five control variables and prior-year skill level. R2 values in these models ranged from .85 to .92, with standardized betas for the SBP predictor ranging from .03 to .11 (p ≤ .05). The next four regression models examined the effect of the amount of SBP earned. Results indicated that the amount of SBP earned also explained significant variance in each subsequent year’s skill proficiency (R2 values ranged from .85 to .92), with standardized betas for the SBP predictor ranging from .05 to .13 (p ≤ .05). Across all eight regressions, prior-year skill levels displayed the largest effects on subsequent-year skill proficiency, with standardized betas ranging from .73 to .88 (p ≤ .01). Taken collectively, these regression results clearly indicate temporal precedence in that SBP was predictive of subsequent-year skill across all yearly time frames. With evidence of temporal precedence provided by the repeated regressions, we moved to LGM. Covariance data and the CALIS procedure of SAS Version 9 were used to conduct all LGM analyses. Consistent with recommendations from structural equation modeling literature (e.g., MacCallum & Austin, 2000; Marsh, Balla, & McDonald, 1988), several fit indices were used for judging the adequacy of model fit. Four specific fit indices were consulted: normed fit index (NFI; Bentler & Bonett, 1980), comparative fit index (CFI; Bentler, 1990), goodness-of-fit index (GFI; Jöreskog & Sörbom, 1996), and root mean square error of approximation (RMSEA; Browne & Cudeck, 1993). Values for NFI and CFI above .95 are considered indicators of reasonable fit (Hu & Bentler, 1998, 1999). Browne and Cudeck (1993) suggest that RMSEA values less than .05 represent good fit and values less than .08 represent reasonable fit. Table 2 presents results from the first stage of LGM analysis in which various change trajectories and error variance structures were tested to determine the most accurate and Downloaded from jom.sagepub.com by guest on July 26, 2010 732 Table 1 Means, Standard Deviations, and Correlations Among Study Variables Downloaded from jom.sagepub.com by guest on July 26, 2010 Variable 1. Skill proficiency T1 2. Skill proficiency T2 3. Skill proficiency T3 4. Skill proficiency T4 5. Skill proficiency T5 6. Language difficulty 7. Cognitive ability 8. Level of base salary 9. Level of education 10. SBP frequency 11. SBP amount 12. SBP initial application M SD 1 36.92 37.04 38.95 39.07 39.15 2.08 70.14 7.41 2.59 1.34 2.81 0.43 10.99 11.77 12.23 12.39 12.68 1.18 19.29 3.39 1.04 1.42 3.40 0.49 — .92 .91 .90 .90 –.44 .06 .13 .24 .85 .86 .84 2 — .95 .94 .93 –.49 .06 .15 .26 .88 .88 .80 3 4 5 6 7 8 9 10 11 — .96 .95 –.51 .06 .15 .24 .88 .87 .80 — .96 –.51 .06 .15 .25 .88 .86 .78 — –.52 .06 .15 .25 .87 .85 .78 — .17 –.07 –.05 –.37 –.36 –.31 — .03 .17 .04 –.01 .04 — .43 .22 .22 .21 — .14 .13 .12 — .92 .87 — .83 Note: SBP = skill-based pay. N = 2,105. All correlations above .05 are significant at p ≤ .01. 12 — Dierdorff, Surface / Skill-Based Pay and Skill Change 733 Table 2 Fit Statistics Latent Growth Modeling Models Describing Changes in Skill Proficiency Model I II III IV Change Linear Error Structure Fixed Free Linear + quadratic Fixed Free χ2 349.63** 131.15** 129.47** 25.62** df Δχ2(within) Δχ2(between) 14 10 10 6 — 218.48** — 103.85** — — 220.16** 95.53** RMSEA .11 (.10–.12) .08 (.06–.09) .08 (.06–.09) .04 (.03–.06) GFI CFI NFI .94 .97 .98 .99 .99 .99 .99 .99 .99 .99 .99 .99 Note: N = 2,105. Δχ2 (within) = nested comparisons within type of growth trajectory (e.g., Models I and II); Δχ2 (between) = nested comparison across type of growth trajectories (e.g., Models I and III); RMSEA = root mean square error of approximation; GFI = goodness of fit index; CFI = comparative fit index; NFI = normed fit index. **p < .01. parsimonious growth model. To determine an improvement in fit of one model compared to another, χ2 difference tests were used (Bentler & Bonett, 1980; Medsker, Williams, & Holahan, 1994). Significant differences in χ2 values across nested models indicate an overall improvement in model fit. In this study, significant improvements demarcate a more appropriate modeling of longitudinal skill changes. Four unconditional growth models were tested and are labeled as Models I to IV in Table 2 to facilitate interpretation. Taken collectively, the best fitting model of skill change over five annual assessments was Model IV. This model posited linear and quadratic change with freely estimated error variances and displayed adequate fit statistics across all four indices (RMSEA = .04, GFI = .99, CFI = .99, NFI = .99). Model IV also resulted in significant reductions in χ2 compared to the other three unconditional models. In general, LGM provides five estimates that encompass individual differences in individuallevel change: (a) mean intercept is the sample estimate of the average skill proficiency score across individuals at the initial assessment, (b) mean slope is the average slope (skill change) across individuals, (c) variance of the intercept represents the extent of variability across individuals in their skill proficiency at the initial assessment, (d) variance of the slope represents the variability in change trajectories (rates of skill change) across individuals, and (e) covariance of growth parameters shows the relationship between baseline skill and rates of skill change. The variances of the intercept and slope factors are particularly useful because they indicate the degree to which individuals differ in both their initial skill proficiencies and their rates of change. If significant variance exists in these parameters, one may introduce variables to attempt to account for these individual differences in longitudinal change. The best fitting unconditional model (Model IV) depicted skill change across the five assessments as both linear and quadratic in nature. All parameter estimates in this unconditional model were significantly different from zero (p ≤ .05). The mean slope for linear change was positive, indicating an average pattern of increasing skill proficiency across the overall sample. The mean slope for quadratic change in this unconditional model was negative, indicating that in addition to the linear growth in skill proficiency there was a general downward “bend” (deceleration) in skill proficiency across the five annual skill assessments. Downloaded from jom.sagepub.com by guest on July 26, 2010 734 Journal of Management / August 2008 Table 3 Means and Variances for Growth Parameters of Conditional Models Parameter Alpha vector Mean intercept Mean linear Mean quadratic Psi matrix Variance intercept Variance linear Variance quadratic Covariance intercept–linear Covariance intercept–quadratic Covariance linear–quadratic Model I Model II Model III 16.04** 0.05 — 31.18** –0.29* 0.08 29.84** –0.27* 0.08 33.22** 0.96** — 0.04 — — 19.05** 2.57** 0.15** −1.16* –0.19 –0.52** 17.34** 2.72** 0.14** –0.17 0.10 –0.53** Note: N = 2,105. Conditional Model I corresponds to Hypothesis 1, Conditional Model II corresponds to Hypothesis 2, and Conditional Model III corresponds to Hypothesis 3. *p < .05. **p < .01. Significant variance was found for both the intercept and slope (linear and quadratic) factors, signifying the existence of individual differences in skill change. Results showed that individuals significantly differed in their initial skill proficiency (σ2 = 111.05, p ≤ .01), their linear rates of change (σ2 = 3.66, p ≤ .01), and their quadratic rates of change (σ2 = 0.17, p ≤ .01). Given the evidence of significant variance in skill change provided by the unconditional model, we proceeded to the second stage of LGM analysis, which introduced SBP-related variables to account for this between-individual variability in skill proficiency change while controlling for language difficulty, base salary, cognitive ability, education level, and occupation specialty. Tables 3 and 4 display results pertaining to the three conditional models conducted to test Hypotheses 1 to 3 (labeled as Conditional Models I to III, respectively). Table 3 shows the means and variances for the latent growth parameters in each conditional model. These results mirror the results from the unconditional model in both direction and statistical significance. The alpha vector represents the average true population means for each growth parameter. The psi matrix values are variances and covariances of the growth parameters and, because they are significant, convey meaningful individual differences. Table 4 shows parameter estimates (beta weights) for SBP variables used in each conditional model and provides direct results for hypothesis testing. Hypothesis 1 predicted that initial earning of SBP would be positively associated with subsequent rates of skill change. Conditional Model I tested this hypothesis and provided adequate fit (RMSEA = .02, GFI = .99, CFI = .99, NFI = .99). Because the SBP variable of interest in this model was based on an individual’s language skill proficiency at his or her initial assessment (Time Point 1), only language skill proficiencies at Time Points 2 through 5 were used. Longitudinal change in skill proficiency was modeled only as linear. In comparison to the conditional LGM model depicted as Figure 1 earlier, the current model excludes language proficiency at Time 1 and the quadratic slope factor.3 After controlling for language difficulty, base salary, cognitive ability, education level, and occupation specialty, Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 735 Table 4 Parameters Estimates for Predictors in Conditional Models Model I II III Variable Intercept Base salary Cognitive ability Education Occupation specialty Language difficulty SBP initial application Base salary Cognitive ability Education Occupation specialty Language difficulty SBP frequency Base salary Cognitive ability Education Occupation specialty Language difficulty SBP amount 0.03 0.04 0.83 0.07 –2.97 16.65 –0.05 0.03 0.41 0.01 –1.58 6.06 –0.04 0.05 0.49 0.10 –1.55 2.59 (0.01) (0.07)** (0.06)** (0.01) (–0.18)** (0.62)** (–0.01) (0.05)** (0.03)** (0.01) (–0.10)** (0.29)** (–0.01) (0.09)** (0.03)** (0.02)* (–0.09)** (0.28)** Linear –0.01 0.01 0.01 0.01 –0.21 0.15 0.06 0.01 0.02 0.01 –0.56 0.51 0.06 0.01 0.06 0.03 –0.60 0.17 (–0.06) (0.32)** (0.02) (0.05) (–0.51)** (0.21)* (0.24)* (0.13) (0.03) (0.03) (–0.68)** (0.48)** (0.24)* (0.19) (0.08) (0.07) (–0.74)** (0.36)** Quadratic — — — — –0.01 0.01 –0.01 –0.01 0.07 –0.08 –0.01 0.01 –0.01 –0.01 0.07 –0.03 (–0.24)* (0.02) (–0.02) (–0.02) (0.40)** (–0.37)** (–0.24)* (–0.02) (–0.04) (–0.04) (0.43)** (–0.31)** Note: SBP = skill-based pay. N = 2,105. Conditional Model I corresponds to Hypothesis 1, Conditional Model II corresponds to Hypothesis 2, and Conditional Model III corresponds to Hypothesis 3. Standardized estimates are shown in parentheses. *p < .05. **p < .01. the initial earning of SBP was significantly and positively associated with ensuing skill change (see Table 4). These findings offer support for Hypothesis 1 and indicate that individuals who receive SBP on their first attempt will demonstrate greater rates of skill acquisition. Comparing the variance parameters for the linear slope factors in an unconditional LGM model (Years 2 to 5) and the current conditional model indicated that about 5% of the variance in linear change in skill proficiency was accounted for by whether or not individuals earned SBP at their initial attempts. Hypothesis 2 predicted that the frequency with which individuals receive SBP would be positively associated with their rates of skill change and maintenance. This hypothesis was tested by Conditional Model II, which provided adequate fit to the data (RMSEA = .03, GFI = .99, CFI = .99, NFI = .99). The frequency of earning SBP was significantly associated with both forms of skill proficiency change. Frequency of SBP was positively associated with linear change in skill proficiency, which suggests that the more often an individual receives SBP, the greater that individual’s skill change, regardless of the amount of wage increase. Thus, this finding provides supportive evidence for Hypothesis 2. Frequency of SBP was negatively related to quadratic change in skill proficiency, which indicates that individuals receiving SBP more often show less slowdown (deceleration) in their skill change. Comparing the slope factor variances from the conditional and unconditional Downloaded from jom.sagepub.com by guest on July 26, 2010 736 Journal of Management / August 2008 models showed that frequency of SBP accounted for about 42% of variance in linear skill change across individuals ([3.66 – 2.57] ÷ 2.57 = 0.42) and roughly 13% of variance in quadratic skill change across individuals. Hypothesis 3 predicted the amount of SBP earned to be positively associated with rates of skill change. To test this hypothesis, a third LGM model was posited (Conditional Model III) and provided adequate fit (RMSEA = .03, GFI = .99, CFI = .99, NFI = .99). The amount of SBP was associated with linear and quadratic changes in skill proficiency. These results suggest that individuals receiving larger amounts of SBP display more rapid skill change (i.e., steeper positive linear slope) and less deceleration in skill proficiency. These associations are similar in direction to frequency of SBP, but results show smaller parameter estimates indicating that the amount of SBP individuals earn may account for less variance in skill change differences than how often they earn SBP. Comparison of the variance estimates in the conditional and unconditional models shows that the amount of SBP accounted for about 35% and 21% of the variance in linear and quadratic skill change across individuals, respectively.4 Discussion The current study investigated the extent to which SBP influences skill change across a 5-year period. Perhaps the broadest conclusion from our findings is that SBP is indeed associated with skill change among individuals working under such a compensation system. It is important to note that these results represent the first empirical evidence confirming that SBP is related to individual skill change and maintenance. Thus, this evidence lends support to the commonly held but untested supposition that linking pay to organizationally desired skills will subsequently increase the learning of these skills. In particular, individuals who receive SBP in their first attempt at qualifying for such incentive pay have higher rates of ensuing skill change. These results are congruent with previous research showing that belief in one’s behavioral capability is more salient to performance during early attempts at behavior (Mitchell et al., 1994). Our findings suggest that individuals with initial success in earning SBP are also those that more rapidly increase their skill proficiency, even after controlling for several other salient factors (e.g., the difficulty of the focal skill, general cognitive ability, different job contexts, etc.). These results are relevant to compensation practice in that they depict the criticality of early skill demonstration to later skill development. In other words, an individual’s initial success or failure under an SBP system appears to have a lasting influence on subsequent skill change regardless of the difficulty of the skill, his or her cognitive capacity for learning, and particular job responsibilities. Our findings also suggest that the frequency with which individuals receive SBP is associated with higher rates of skill change. Results show that individuals who more often qualified for SBP had steeper slopes of linear change, representing more rapid skill growth. It may be that these individuals perceived clearer links between their own skill proficiency and SBP wage increases. Such linkages, or “lines of sight,” are often discussed in the compensation literature as important characteristics of effective reward systems (Lawler, 2000). Our results further indicate that the amount of SBP individuals earn relative to their skill proficiency also accounts for higher rates of skill growth (i.e., steeper learning curves). This Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 737 demonstrates that utilizing pay differentials requisite with greater levels of skill proficiency can affect the degree to which individuals pursue further skill mastery. It should be noted that the influences of SBP on skill changes are evident despite the fact that monthly increases were the same dollar amounts for all participants (i.e., $100 to $200), even for those at higher base pay levels. Thus, it appears that the SBP incentives were meaningful enough to make what compensation researchers term, a “just noticeable difference.” Study results further indicate that SBP is related to the rate at which individuals experience attenuation of skill change. As previously discussed, the significant negative quadratic trajectory found in this study depicts skill change as a deceleration beyond steady skill growth over time (i.e., a flattening of the upward growth trend). The influence of SBP on this deceleration is evident in the significant associations of SBP with the quadratic change factors. Here, results show that SBP is negatively related to the deceleration of skill change, suggesting that individuals who more frequently receive SBP, and in larger amounts, tend to not experience as much deceleration in skill growth. The interesting implication is that in our sample using SBP appears to “slow the slowdown” in skill development. This finding may have ramifications for an often-cited concern regarding SBP, namely, that it will lose motivating potential because employees may too quickly “top out” in terms of skill and contingent incentive pay. However, several compensation researchers (e.g., Heneman et al., 2000; Lawler & Ledford, 1987) have argued motivation loss to be unlikely, as most individuals will realize that they are better off financially under an SBP system than a traditional job-based system. The idea that employees may top out and become disgruntled under SBP systems notwithstanding, organizations should be concerned with how SBP affects skill growth and skill maintenance. Our results present evidence to suggest that SBP may be associated with skill maintenance in addition to skill change, as earning SBP appears to lessen skill decay (i.e., deceleration of skill growth). In this respect, SBP appears to influence the maintenance of skill proficiency. It is important to note that although the period examined in the current study was substantial (5 years of skill assessments), the effectiveness of SBP in slowing deceleration of skill development may not extend to longer periods. Moreover, the organization used in our study clearly engaged in system monitoring, wherein skill assessments were required on an annual basis. Continuous monitoring of SBP systems is an important component of the effectiveness of this compensation strategy (Lawler, 2000). The extent to which SBP does indeed promote the maintenance of skill proficiency and the impact of actively monitoring SBP systems remain areas in need of additional empirical research. It is important to reiterate that substantial individual differences in rates of skill change were found, as evidenced in the significant variances of both the linear and quadratic slope factors. The current study demonstrates that basic SBP-related variables (frequency, amount, etc.) can account for substantial variance in skill change between individuals. However, much variance still remains to be accounted for by variables not included in the present research. In this sense, the findings are best characterized as providing empirical confirmation of the efficacy of the relationship between SBP and skill change rather than offering explanations of underlying motivational processes. A potentially fruitful starting point in the investigation of additional individual-level variables is the skill-seeking model recently proffered by Murray and Gerhart (2000). Using the theory of reasoned action, their model posits several classes of variables, including attitudes, subjective norms, and perceptions of behavioral Downloaded from jom.sagepub.com by guest on July 26, 2010 738 Journal of Management / August 2008 control, that are presented as influential factors on both skill-seeking intentions and subsequent skill acquisition under SBP systems. Given the substantial individual differences in skill change shown in this study, future systematic examination of these types of individuallevel factors should prove valuable. Previous research indicates that using SBP can lead to desirable organizational-level outcomes. These outcomes are presumably achieved through motivating skill development among the organization’s members. Our study provides confirming evidence that SBP can motivate skill change. However, what remains to be examined is exactly how aspects of SBP influence the broader process of skill seeking, skill change, and subsequent job performance. To date, there has been no research that directly investigated how and if SBP affects individual-level job performance. In addition, recent research has suggested that contextual factors and design features of SBP systems impact their success and sustainability (Shaw et al., 2005). Future research that attempts to incorporate the full breadth of factors affecting SBP effectiveness is clearly necessary. Empirical work that uses a multilevel perspective to incorporate various individual and organizational level factors into data analysis could well serve this research need. Limitations It should be emphasized that the SBP plan used in the current study primarily sought to reward the depth of a particular skill (i.e., foreign language proficiency). However, it should also be noted that individuals were indeed free to pursue greater skill proficiency and earn SBP in multiple languages. Nonetheless, many SBP plans seek to facilitate the acquisition of a repertoire or breadth of skills (Gupta & Shaw, 2001; Heneman et al., 2000; Shaw et al., 2005). Thus, our results may not generalize to SBP systems that attempt to motivate the learning of broader sets of skills or other nonlanguage skills. Future research that longitudinally examines the extent to which skill breadth moderates skill change under SBP systems and research attempting to replicate the current results with other samples and skills would be valuable. It is important to mention that foreign language proficiency is rapidly emerging as a critical need for many U.S. organizations because of immigration, globalization, and issues of national security (Lynch, 2006; Rovira, 2003; U.S. Department of Defense, 2005; Weber, 2004). Thus, although foreign language proficiency may be seen as more specific when compared to skills typically included in SBP systems, such proficiency can be expected to underlie a large proportion of an individual’s job performance. That is, to the extent that using foreign language is necessary to one’s job, increasing an individual’s language proficiency should promote more effective overall performance. As mentioned earlier, what is needed is additional research that investigates the relationships between SBP and job performance to address whether developing particular skills or skill sets improves subsequent on-the-job performance. As with all research, characteristics of the participant sample and organization may affect the generalizability of results. For example, participants used in our study were members of a military organization composed of highly specialized, elite units. As such, the participants may differ from other military personnel in that they underwent a more rigorous selection and assessment process that screens for characteristics such as cognitive ability and physical Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 739 fitness. In addition, the participants’ work roles frequently entailed duties not typically performed by other more mainstream jobs. Several of these sample characteristics could foster greater motivation for skill change, especially if health and well-being are contingent on skill proficiency. However, there are many nonmilitary occupations for which foreign language skill is equally pertinent to well-being (e.g., paramedics, critical care nurses, emergency room surgeons, police officers, etc.), and there are other types of organizations beyond corporations that use SBP systems (e.g., state boards of education for schoolteacher compensation). Even so, the distinctive context in which our research was conducted could limit the generalizability of our findings. In addition to sample characteristics, two other potential limitations are important to note. First, although the multiple regression analyses allay concerns of temporal precedence, these results do not completely negate the temporal overlap in the SBP-related variables (frequency and amount of SBP) used to test Hypotheses 2 and 3. Second, criterion scores for language skill proficiency were operationalized as manifest variables, and thus measurement invariance could not be assessed. A lack of measurement invariance is an essential assumption when interpreting LGM results (Chan, 1998). Because the standardized test used to measure language proficiency in this study does not assess a traditional, multifaceted psychological construct (i.e., the test measures listening and speaking skills), we feel this threat is likely minimal. However, our findings should be interpreted with respect to both of these measurement conditions. Conclusion Our study provides the first empirical evidence that SBP can motivate changes in organizationally relevant skills. Thus, our results offer confirmation for the conventional wisdom that if you pay for skills, individuals will actually learn these skills. These are important findings to both compensation research and practice, as prior SBP research has been scarce, while at the same time the use of SBP has become increasingly prevalent. It should be noted that although the use of SBP has grown in popularity, such compensation strategies may not be utilized for the majority of occupations within the U.S. workforce. Nonetheless, our results provide general support for increases in individual-level skill change under SBP systems. However, many questions remain to be addressed by future research. We hope that the current study encourages new empirical efforts that seek to gain a better understanding of exactly how, why, and when SBP will be an effective component of an organization’s compensation strategy. Notes 1. The Department of Defense has recently changed compensation policy relevant to skill-based pay (SBP) for foreign language skills. These policy changes are effective as of fiscal year 2007. All of our descriptions are based on the prior system, and thus data and results reflect the previous policy. 2. The Department of Defense has been in the process of developing and implementing new computer-based versions of the Defense Language Proficiency Test across all languages. The new versions would not produce the same standardized scores as in the present study. Downloaded from jom.sagepub.com by guest on July 26, 2010 740 Journal of Management / August 2008 3. Because Hypothesis 1 deals with the impact of initial (Year 1) earning of SBP on subsequent skill change, a separate unconditional model was needed to accurately assess longitudinal change in skill proficiency from Years 2 to 5. In this unconditional model, the quadratic component was not significant (p ≥ .05) and thus is not included in the conditional model testing Hypothesis 1. 4. Conditional Models II and III used time-invariant SBP variables to account for skill proficiency change. To examine if results were simply because of these operationalizations, we conducted two additional latent growth modeling analyses using time-variant SBP variables (e.g., changes in application of SBP and changes in amount of SBP over time). Results for these supplemental models were congruent with findings supporting Hypothesis 2 and 3 (from Conditional Models II and III), as longitudinal changes in both the application of SBP and the amount of SBP were positively related to linear skill change and negatively related to quadratic skill change (p ≤ .01). References Ackerman, P. L. 1987. Individual differences in skill learning: An integration of psychometric and information processing perspectives. Psychological Bulletin, 102: 3-27. Baldwin, T. T., & Ford, J. K. 1988. Transfer of training: A review and directions for future research. Personnel Psychology, 41: 63-103. Bentler, P. 1990. Comparative fit indexes in structural models. Psychological Bulletin, 107: 238-246. Bentler, P., & Bonett, D. G. 1980. Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88: 588-600. Browne, M. W., & Cudeck, R. 1993. Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models: 136-162. Newbury Park, CA: Sage. Bunning, R. L. 1989. Skill-based pay: Restoring incentives to the workplace. Personnel Administrator, 34: 65-70. Bunning, R. L. 1992. Models for skill-based pay plans. HR Magazine, 37: 62-64. Carretta, T. R., & Doub, T. W. 1998. Group differences in the role of g and prior job knowledge in the acquisition of subsequent job knowledge. Personality and Individual Differences, 24: 585-593. Chan, D. 1998. The conceptualization and analysis of change over time: An integrative approach incorporating longitudinal mean and covariance structures analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1: 421-483. Chan, D. 2002. Latent growth modeling. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations: Advances in measurement and data analysis: 302-349. San Francisco: Jossey-Bass. Duncan, T. E., Duncan, S. C., Strycker, L. A., Li, F., & Alpert, A. 1999. An introduction to latent variable growth curve modeling: Concepts, issues, and applications. Mahwah, NJ: Lawrence Erlbaum. Farr, J. L. 1976. Task characteristics, reward contingency, and intrinsic motivation. Organizational Behavior and Human Performance, 16: 294-307. Ford, J. K., & Kraiger, K. 1995. The application of cognitive constructs and principles to the instructional systems design model of training: Implications for needs assessment, design, and transfer. International Review of Industrial and Organizational Psychology, 10: 1-48. Gerhart, B., & Milkovich, G. T. 1992. Employee compensation: Research and practice. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology: 481-570. Palo Alto, CA: Consulting Psychologists Press. Goldstein, I. L. 1986. Training in organizations: Needs assessment, development and evaluation. Monterey, CA: Brooks/Cole. Goldstein, I. L., & Ford, J. K. 2002. Training in organizations: Needs assessment, development, and evaluation (4th ed.). Cincinnati, OH: Thomson/Wadsworth. Gross, L., & Griffith, S. 2003. Charting a course toward growth: U.S. compensation strategies at the helm. Perspective, October: 27. Gupta, N., Jenkins, G. D., & Curington, W. P. 1986. Paying for knowledge: Myths and realities. National Productivity Review, 5: 107-123. Gupta, N., & Shaw, J. D. 2001. Successful skill-based pay plans. In C. H. Fay, D. Knight, & M. A. Thompson (Eds.), The executive handbook on compensation: Linking strategic rewards to business performance: 513-526. New York: Free Press. Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 741 Heneman, R. L., & Ledford, G. E., Jr. 1998. Competency pay for professionals and managers in business: A review and implications for teachers. Journal of Personnel Evaluation in Education, 12: 103-121. Heneman, R. L., Ledford, G. E., & Gresham, M. T. 2000. The changing nature of work and its effects on compensation and delivery. In S. L. Rynes & B. Gerhart (Eds.), Compensation in organizations: 195-240. San Francisco: Jossey-Bass. Hills, F. S. 1989. Internal pay relationships. In L. R. Gomez-Mejia (Ed.), Compensation and benefits: 29-62. Washington, DC: BNA Books. Hofmann, D. A., Jacobs, R., & Barrata, J. E. 1993. Dynamic criteria and the measurement of change. Journal of Applied Psychology, 78: 194-204. Hofmann, D. A., Jacobs, R., & Gerras, S. J. 1992. Mapping individual performance over time. Journal of Applied Psychology, 77: 185-195. Hu, L., & Bentler, P. M. 1998. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3: 424-453. Hu, L., & Bentler, P. M. 1999. Cutoff criteria for fit indexes in covariance structure modeling: Conventional criteria versus new alternatives. Structural Equation Modeling, 6: 1-55. Hunter, J. E., & Schmidt, F. L. 1998. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124: 262-274. Jenkins, G. D., Jr., Ledford, G. E., Gupta, N., & Doty, D. H. 1992. Skill-based pay: Practices, payoffs, pitfalls, and prescriptions. Phoenix, AZ: American Compensation Association. Johnson, B. A., & Ray, H. H. 1993. Employee developed pay system increases productivity. Personnel Journal, 72: 112-118. Jöreskog, K. G., & Sörbom, D. 1996. LISREL 8 user’s reference guide. Chicago: Scientific Software. Judge, T. A., & Piccolo, R. F. 2004. Transformational and transactional leadership: A meta-analytic test of their relative weights. Journal of Applied Psychology, 89: 755-768. Kanfer, R. 1990. Motivation theory and industrial/organizational psychology. In M. D. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology, Volume 1: Theory in industrial and organizational psychology: 75-170. Palo Alto, CA: Consulting Psychologists Press. Lawler, E. E., III. 1971. Pay and organizational effectiveness: A psychological view. New York: McGraw-Hill. Lawler, E. E., III. 2000. Rewarding excellence: Pay strategies for the new economy. San Francisco: Jossey-Bass. Lawler, E. E., III, & Jenkins, G. D., Jr. 1992. Strategic reward systems. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology: 1009-1055. Palo Alto, CA: Consulting Psychologists Press. Lawler, E. E., III, & Ledford, G. E., Jr. 1987. Skill-based pay: A concept that’s catching on. Management Review, 76: 46-51. Lawler, E. E., III, Mohrman, S. A., & Ledford, G. E., Jr. 1998. Strategies for high performance organizations. San Francisco: Jossey-Bass. Ledford, G. E., & Bergel, G. 1991. Skill-based pay case number 1: General Mills. Compensation and Benefits Review, 23: 24-38. Lee, C., Law, K., & Bobko, P. 1999. The importance of justice perceptions on pay effectiveness: A two-year study of a skill-based pay plan. Journal of Management, 25: 851-873. Locke, E. A., & Latham, G. P. 1990. A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall. Luthans, F., & Kreitner, R. 1975. Organizational behavior modification. Glenview, IL: Scott, Foresman. Luthans, F., & Kreitner, R. 1985. Organizational behavior modification and beyond. Glenview, IL: Scott, Foresman. Lynch, D. J. 2006, February 8. U.S. firms becoming tongue-tied. USA Today, p. 6B. MacCallum, R. C., & Austin, J. T. 2000. Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51: 201-226. Mahoney, T. A. 1989. Employment compensation planning and strategy. In L. R. Gomez-Mejia (Ed.), Compensation and benefits: 1-25. Washington, DC: BNA Books. Marsh, H. W., Balla, J. R., & McDonald, R. P. 1988. Goodness-of-fit indices in confirmatory factor analysis. Psychological Bulletin, 103: 391-410. Martocchio, J. J. 2006. Strategic compensation: A human resource management approach (4th ed.). Upper Saddle River, NJ: Prentice Hall. Downloaded from jom.sagepub.com by guest on July 26, 2010 742 Journal of Management / August 2008 Medsker, G. J., Williams, L. J., & Holahan, P. J. 1994. A review of current practices for evaluating causal models in organizational behavior and human resources management research. Journal of Management, 20: 439-464. Mitchell, T. R., Hopper, H., Daniels, D., George-Falvy, J., & James, L. R. 1994. Predicting self-efficacy and performance during skill acquisition. Journal of Applied Psychology, 79: 506-517. Murray, B., & Gerhart, B. 1998. An empirical analysis of a skill-based pay program and plant performance outcomes. Academy of Management Journal, 41: 68-78. Murray, B., & Gerhart, B. 2000. Skill-based pay and skill seeking. Human Resource Management Review, 10: 271-287. Muthen, B. O. 1991. Analysis of longitudinal data using latent variable models with varying parameters. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: Recent advance, unanswered questions, future directions: 1-17. Washington, DC: American Psychological Association. Pfeffer, J. 1995. Competitive advantage through people: Unleashing the power of the workforce. Boston: Harvard Business School Press. Podsakoff, P. M., Bommer, W. H., Podsakoff, N. P., & MacKenzie, S. B. 2006. Relationships between leader reward and punishment behavior and subordinate attitudes, perceptions, and behaviors: A meta-analytic review of existing and new research. Organizational Behavior and Human Decision Processes, 99: 113-142. Podsakoff, P. M., Todor, W. D., & Skov, R. 1982. Effects of leader performance contingent and non-contingent reward and punishment behaviors on subordinate performance and satisfaction. Academy of Management Journal, 25: 812-821. Ree, M. J., & Carretta, T. R. 1994. Factor analysis of the ASVAB: Confirming a Vernon-like structure. Educational and Psychological Measurement, 54: 459-463. Ree, M. J., Carretta, T. R., & Teachout, M. S. 1995. Role of ability and prior knowledge in complex training performance. Journal of Applied Psychology, 80: 721-730. Ree, M. J., & Earles, J. A. 1991. Predicting training success: Not much more than g. Personnel Psychology, 44: 321-332. Ree, M. J., Earles, J. A., & Teachout, M. S. 1994. Predicting job performance: Not much more than g. Journal of Applied Psychology, 79: 518-524. Rouiller, J. Z., & Goldstein, I. L. 1993. The relationship between organizational transfer climate and positive transfer of training. Human Resource Development Quarterly, 4: 377-390. Rovira, J. D. 2003. Importance of foreign language capabilities (ALMAR 072/03). Retrieved September 9, 2005, from http://www.marines.mil/almars/almar2000.nsf Rynes, S. L., & Gerhart, B. 2000. Compensation in organizations: Current research and practice. San Francisco: Jossey-Bass. Shareef, R. 1994. Skill based pay in the public. Review of Public Personnel Administration, 14: 60-74. Shaw, J. D., Gupta, N., Mitra, A., & Ledford, G. E., Jr. 2005. Success and survival or skill-based pay plans. Journal of Management, 31: 28-49. Silva, J. M., & White, L. A. 1993. Relation of cognitive aptitudes to success in foreign language training. Military Psychology, 5: 79-93. Stajkovic, A. D., & Luthans, F. 1997. A meta-analysis of the effects of organizational behavior modification on task performance, 1975-1995. Academy of Management Journal, 40: 1122-1149. Stajkovic, A. D., & Luthans, F. 2001. Differential engagement effects of incentive motivators on work performance. Academy of Management Journal, 44: 580-590. Stajkovic, A. D., & Luthans, F. 2003. Behavioral management and task performance: Conceptual background, meta-analysis, and test of alternative models. Personnel Psychology, 56: 155-194. Surface, E. A., & Dierdorff, E. C. 2003. Reliability and the ACTFL Oral Proficiency Interview: Reporting indices of interrater consistency and agreement for 19 languages. Foreign Language Annals, 36: 507-519. Surface, E. A., Dierdorff, E. C., & Donnelly, J. 2004. Modeling change in second language proficiency for U.S. Special Operations Forces soldiers. Paper presented at the 19th annual conference of the Society for Industrial and Organizational Psychologists, Chicago. Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. 1995. Applying trained skills on the job: The importance of the work environment. Journal of Applied Psychology, 80: 239-252. U.S. Department of Defense. 2005. A call to action for national foreign language capabilities. Retrieved April 1, 2005, from http://www.nlconference.org/docs/White_Paper.pdf Downloaded from jom.sagepub.com by guest on July 26, 2010 Dierdorff, Surface / Skill-Based Pay and Skill Change 743 U.S. Department of Defense. 2006. Quadrennial defense review report. Washington, DC: Government Printing Office. Weber, G. 2004. English RULES. Workforce Management, 83: 47-51. Wexley, K. N., & Latham, G. P. 1991. Developing and training human resources in organizations (2nd ed.). New York: HarperCollins. Willett, J. B., & Sayer, A. G. 1994. Using covariance structure analysis to detect correlates and predictors of individual change over time. Psychological Bulletin, 116: 363-381. Biographical Notes Erich C. Dierdorff is an assistant professor of management at DePaul University’s Kellstadt Graduate School of Business. He received his PhD in industrial/organizational psychology from NC State University. His current research interests include understanding how individuals assess requirements of their work roles and examining the interplay between person and contextual influences on work-related inferences, such as those used in performance evaluation, work analysis, and training needs assessment. Eric A. Surface earned his PhD in industrial/organizational psychology from NC State University. He is president of SWA Consulting Inc. (Raleigh, NC; formerly Surface, Ward, & Associates) and is an adjunct assistant professor of psychology at NC State University. His research interests include training effectiveness, the influence of context on training and work criteria, foreign language proficiency testing and training, the use of technology for training, and survey methodology. Downloaded from jom.sagepub.com by guest on July 26, 2010