Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Matt C Howard

    Matt C Howard

    Mediation and conditional process analyses have become popular approaches for examining the mechanisms by which effects operate and the factors that influence them. To estimate mediation models, researchers often augment their structural... more
    Mediation and conditional process analyses have become popular approaches for examining the mechanisms by which effects operate and the factors that influence them. To estimate mediation models, researchers often augment their structural equation modeling (SEM) analyses with additional regression analyses using the PROCESS macro. This duality is surprising considering that research has long acknowledged the limitations of regression analyses when estimating models with latent variables. In this article, we argue that much of the confusion regarding SEM’s efficacy for mediation analyses results from a singular focus on factor-based methods, and there is no need for a tandem use of SEM and PROCESS. Specifically, we highlight that composite-based SEM methods overcome the limitations of both regression and factor-based SEM analyses when estimating even highly complex mediation models. We further conclude that composite-based SEM methods such as partial least squares (PLS-SEM) are the preferred and superior approach when estimating mediation and conditional process models, and that the PROCESS approach is not needed when mediation is examined with PLS-SEM.
    Recent years have seen an increase in the number of researchers and practitioners applying virtual reality (VR) to develop social skills, but varying levels of success have been observed when applying these VR training programs. Due... more
    Recent years have seen an  increase in  the  number of  researchers and  practitioners applying virtual reality (VR) to develop social skills, but varying levels of success have been observed when applying these VR training programs. Due  to these disparities, review and summary work is needed to determine (a) whether these programs are effective and (b) the attributes of these programs that lead to success. In the current article, we perform a meta-analysis to determine the overall effectiveness of VR training programs for developing social skills, and we also study the effect of several moderating variables that may influence the effectiveness of these programs. We test whether certain aspects of the applied hardware (e.g. input devices, output devices), applied software (e.g. game elements, Second Life), participant population (e.g. general/specialized), and study design (e.g. type of control group, type of measure, others) influence the success of these programs. Sources were identified using EBSCO and Google Scholar, and, after our inclusion criteria were applied, 23 samples were included in our meta-analysis. Using a random-effects approach, we  show that  VR  training programs, on  average, perform better than alternative training programs for developing social skills, but almost all the other findings contradict current notions regarding these programs. Of note, gamified programs produced slightly worse outcomes than non-gamified programs, and programs utilizing immersive technologies (e.g. head-mounted displays) produced slightly worse outcomes than programs utilizing non-immersive displays (e.g. monitors). We provide suggestions regarding the effectiveness of VR training for social skills in comparison to alternative training approaches, the attributes of VR training programs that produce better outcomes, and directions for future research and practice.
    We propose that (a) certain aspects of virtual reality (VR) training programs are seductive details, (b) these seductive details prompt trainee distraction and/or cognitive overload , and (c) pre-training interventions can benefit... more
    We propose that (a) certain aspects of virtual reality (VR) training programs are seductive details, (b) these seductive details prompt trainee distraction and/or cognitive overload , and (c) pre-training interventions can benefit learning from VR training programs by targeting these mechanisms. In Study 1, we apply a meta-cognitive strategy pre-training intervention, which targets distraction, and a habituation pre-training intervention, which targets cognitive overload. Habituation had no effect, whereas meta-cognitive strategies worsened learning. Qualitative results indicated the meta-cognitive strategy intervention prompted trainees to become more cognitively engaged in the distracting seductive details of the VR training program. In Study 2, we tested an alternative pre-training intervention, attentional advice, to reduce distraction and increase learning. The attentional advice pre-training intervention was successful, as trainees demonstrated greater learning when provided attentional advice. Together, all proposals were supported. VR can contain seductive details that cause distraction, and attentional advice can improve learning by reducing distraction. K E Y W O R D S pre-intervention, pre-training, seductive details, training, virtual reality training
    Researchers have shown great interest in the antecedents and outcomes of workplace ostracism, which has led to an expansive body of research. In light of this work, the current article fulfills the need for a comprehensive review and... more
    Researchers have shown great interest in the antecedents and outcomes of workplace ostracism, which has led to an expansive body of research. In light of this work, the current article fulfills the need for a comprehensive  review  and  meta-analysis  of  the  antecedents  and  outcomes  associated  with  workplace ostracism.  We  begin  our  review  by  adapting  a  victimization  perspective  to  understand  ostracism  as  a triadic  social  process  between  the  victim,  perpetrator,  and  the  environment.  The  meta-analytic  results then support that leadership characteristics are the strongest related antecedents of workplace ostracism,followed  by  certain  aspects  of  personality  (e.g.,  Big  Five)  and  contextual  characteristics  (e.g.,  social support). The results also show that workplace ostracism very strongly relates to deviance, and it strongly relates  to  other  performance  outcomes  (e.g.,  core-performance,  helping,  voice),  well-being  outcomes(e.g.,  psychological  well-being,  emotions,  self-perceptions),  and  organizational  perceptions  (e.g.,  job satisfaction,  commitment,  justice).  We  also  show  that  the  relationship  of  performance,  well-being,emotions, and self-perceptions when measured after ostracism was comparable to their relationship when measured before ostracism. These results suggest that the outcomes of ostracism are less certain than previously thought, as they may instead be antecedents of ostracism. Finally, we call for future research to investigate this notion, along with further integration of the victimization perspective as well as the study of contextual predictors and moderators.
    Authors have shown a strong inter-est in training self-efficacy (TSE), but two noteworthy concerns are present in the literature. First, existing measures of TSE may contain items that do not gauge their construct of... more
    Authors have shown a strong inter-est in training self-efficacy (TSE), but two noteworthy  concerns  are  present  in  the  literature. First, existing measures of TSE may  contain  items  that  do  not  gauge  their    construct    of    interest.    Second,    although  other  forms  of  self-efficacy  may  better  explain  observed  relation-ships, TSE is often studied in isolation. We address  these  concerns  by  creating  two  measures  in  a  four-study  process.  These  measures are shown to have satisfactory psychometric  properties  and  convergent validity. Additionally, we provide an empirical study that investigates, regarding a computer-based training program, the  impact  of  TSE  beyond  positive  self-  evaluations,  general  self-efficacy,  and  computer-self  efficacy.  The  results  demonstrate that TSE is predictive of trainee reactions  beyond  these  other  predictors,  but  it  is  not  predictive  of  learning.  While  the  specificity  of  TSE  may  cause  these  results,  novel  theoretical  perspectives  may  better  explain  the  observed  relationships.
    Research on task–technology fit (TTF) theory is in need of refinement that is centered around (1) conceptualizations of TTF, (2) operationalizations of TTF, (3) an oversight of“misfit,”and (4) an overemphasis on direct effects. We review... more
    Research on task–technology fit (TTF) theory is in need of refinement that is centered around (1) conceptualizations of TTF, (2) operationalizations of TTF, (3) an oversight of“misfit,”and (4) an overemphasis on direct effects. We review TTF to place it in the broader nomological net of related constructs. We differentiate task–technology misfit (TTM) from TTF, and we distinguish two types of TTM, “Too Little” and “Too Much.” Then, we undergo a four-study process to create two satisfactory scales. Finally, we perform two empirical studies to confirm our three-dimensional conceptualization of TTF and TTM in the larger TTF theory framework.
    Virtual reality (VR) has been repeatedly applied for personal development purposes, ranging from learning and training (cognitive outcomes), to psychological therapies (emotional outcomes), to physical rehabilitation(physical outcomes).... more
    Virtual reality (VR) has been repeatedly applied for personal development purposes, ranging from learning and training (cognitive outcomes), to psychological therapies (emotional outcomes), to physical rehabilitation(physical outcomes). Several factors lead to a successful VR intervention,most notably the hardware and software. In the current article, a meta-analysis is performed to test the effect of specialized input hardware (e.g motion sensors, floor pads, etc.), advanced output hardware (i.e., head-mounted displays, surround-screen displays, etc.), and game elements (i.e.,score, competition, etc.) across and within the three noted applications of VR intervention. When analyzing the overall effects, only game elements had a significant impact on outcomes. When analyzing specific applications, input hardware did not have a notable impact on outcomes for any application; output hardware had a notable impact on cognitive and emotional outcomes but not physical; and game elements had a notable impact on cognitive outcomes but not emotional or physical. From these results, the current article provides direct suggestions for future research and practice. Particularly, certain mediating mechanisms are suggested to explain the impact of output hardware and game elements on VR intervention outcomes, sparking possible new directions for research and practice.
    The current article tests whether task performance influences general self-efficacy without increases in the skills required to achieve success. To do so,an experimental design is applied in which participants predict a random... more
    The current article tests whether task performance influences general  self-efficacy without increases in the skills required to achieve success. To do so,an  experimental  design  is  applied  in  which  participants  predict  a  random future  event,  and  the  relationship  between  prediction  task  performance and  self-efficacy  is  observed.  This  article  also  tests  whether  this  specific performance/self-efficacy  relationship  is  moderated  by  (a)  perceived  illegitimacy  of  predicting  the  future  and  (b)  self-assessed  ability  to  predict  the future. The results show that prediction task performance indeed influences general self-efficacy, and neither of these two moderators has a significant effect on this relationship. Therefore, performance on a low-stakes task does influence general self-efficacy—at least temporarily. More importantly, task performance  is  shown  to  influence  self-efficacy  even  without  increases  in the skills required to achieve success.
    The current article heeds three prior calls to advance the study of courage. First, we replicate prior findings regarding the relationship of social courage and work outcomes, but we also investigate, for the first time, the relationship... more
    The current article heeds three prior calls to advance the study of courage. First, we replicate prior findings regarding the relationship of social courage and work outcomes, but we also investigate, for the first time, the relationship between social courage and well-being outcomes. Second, we apply the approach/avoidance framework to understand how social courage influences these outcomes.Third, approach and avoidance motivations are tested as mediators of the relationships of social courage with outcomes while controlling for established aspects of personality, conscientiousness and neuroticism. Two longitudinal studies show that approach motivation is a robust mediator between courage and both workplace and well-being outcomes, suggesting that courageous individuals achieve positive outcomes because they particularly value the benefits in such actions.Avoidance motivation was a less consistent mediator. Implications are discussed, but it should be emphasized that broader relationships of social courage can now be identified by applying the approach/avoidance framework.
    Storm, Sherwood, Roe, Tressoldi, Rock and Risio (2017) performed a meta-analysis of laboratory studies of ESP in dreams, finding a moderate overall effect (r = .20). Certain statistical decisions, such as primarily... more
    Storm,  Sherwood,  Roe,  Tressoldi,  Rock  and  Risio  (2017)  performed  a  meta-analysis  of  laboratory  studies  of ESP in dreams, finding a moderate overall effect (r = .20).  Certain statistical decisions, such as primarily reporting unweighted effects, may have inflated these observations.  In the current article, I perform an inverse-variance weighted meta-analysis using the database provided by Storm et al., finding a much smaller overall effect (r = .07).  I also find evidence that a significant relationship exists between effect size and sample size, suggesting that the prior results may have been primarily driven by large effects found in small-n studies.  I suggest that future researchers of ESP in dreams should produce more large-n studies, which could alleviate many notable criticisms of the current psi literature.
    The primary goal of the current article is to ‘‘take stock” of the ‘‘other” relationships of self-assessed intelligence (SAI). The current article groups the relationships of SAI into four categories: constructs associated with... more
    The primary goal of the current article is to ‘‘take stock” of the ‘‘other” relationships of self-assessed intelligence (SAI). The current article groups the relationships of SAI into four categories: constructs associated with intelligence (openness, emotional intelligence), tendencies and opportunities to develop intelligence (conscientiousness, education, age, SES, prior IQ test experience), constructs associated with biased self-assessments (extraversion, neuroticism, narcissism, honesty-humility, race), and positive states and life achievements (positive self-regard, psychological well-being, academic achievement). The meta-analytic results demonstrate that almost all variables from these four categories significantly relate to SAI, with the exception of prior IQ test experience. These relationships are also consistent when accounting for psychometric intelligence, and no studied moderator variables consistently influence the magnitude of these results.
    The current article performs the first focused investigation into the construct of perceived self-esteem instability (P-SEI). Four studies investigate the construct's measurement, nomological net, and theoretical dynamics. Study 1... more
    The current article performs the first focused investigation into the construct of perceived self-esteem instability (P-SEI). Four studies investigate the construct's measurement, nomological net, and theoretical dynamics. Study 1 confirms the factor structure of a P-SEI Measure, supporting that P-SEI can be adequately measured. Study 2 identifies an initial nomological net surrounding P-SEI, showing that the construct is strongly related to stable aspects of the self (i.e., neuroticism and core self-evaluations). In Studies 3 and 4, the Conservation of Resources Theory is applied to develop and test five hypotheses. These studies show that P-SEI is predicted by self-esteem level and stressors, and the relationship of certain stressors is moderated by self-esteem contingencies. P-SEI also predicts stress, depression, anxiety , and certain defensive postures. From these studies and the integration of Conservation of Resources Theory, we suggest that P-SEI emerges through an interaction between environmental influences and personal resources, and we provide a theoretical model to better understand the construct of P-SEI. We suggest that this theory-driven model can prompt the initial field of study on P-SEI.
    Scale pretests analyze the suitability of individual scale items for further analysis, whether through judging their face validity, wording concerns, and/or other aspects. The current article reviews scale pretests, separated by... more
    Scale pretests analyze the suitability of individual scale items for further analysis, whether through judging their face validity, wording concerns, and/or other aspects. The current article reviews scale pretests, separated by qualitative and quantitative methods, in order to identify the differences, similarities, and even existence of the various pretests. This review highlights the best practices and objectives of each pretest, resulting in a guide for the ideal applications of each method. This is followed by a discussion of eight questions that can direct future research and practice regarding scale pretests. These questions highlight aspects of scale pretests that are still largely unknown, thereby posing a barrier to their successful application.
    Two notable criticisms have been recently raised against research on retroactiveinfluences (RIs). First, researchers have rarely tested the measurement properties oftheir methods to gauge RIs, and it is still relatively unknown whether... more
    Two notable criticisms have been recently raised against research on retroactiveinfluences (RIs). First, researchers have rarely tested the measurement properties oftheir methods to gauge RIs, and it is still relatively unknown whether any commonlyused method is adequate for gauging RIs. Second, RI research needs more closereplications in addition to conceptual replications, in which all aspects of the originalstudy are reproduced. The current article investigates both of these criticisms. Thecurrent article explores the convergent validity and nomological network of twomethods to measure RIs, and it also provides three studies to test these effects: an initialinvestigation, a close replication, and a conceptual replication. The relationship of thetwo methods did not meet even minimal requirements for convergent validity in allthree studies, failing to provide support for their ability to measure RIs. The overallsample-size weighted correlation was .03 (r, 95% CI [.07, .12]). Also, the twomeasures did not share any significant relationships with any other variables, suggest-ing that they do not share a common nomological net. Together, the current articleemphasizes the need for future research to further test the measurement of RIs, asresearchers of RI cannot convincingly claim that their results are valid without firstsupporting their measurement methods. The current article also highlights the impor-tance of valid measurement across all domains of psychology and beyond. Additionalimplications and directions for future research are discussed.
    Courage has been linked to important workplace outcomes, but little is known about the antecedents of courage. To identify possible methods to develop a more courageous workforce, the current... more
    Courage  has  been  linked  to  important  workplace  outcomes,  but  little  is  known  about  the  antecedents  of  courage.  To  identify  possible  methods  to  develop  a  more  courageous  workforce,  the current article provides initial evidence regarding the antecedents of behavioral social courage from  five  broad  categories:  personality,  job  characteristics,  leadership,  culture,  and  demographic  characteristics.  Three  exploratory  studies  show  that  certain  antecedents  from  these  categories  have a significant relationship with behavioral social courage, including grit, proactive personality (personality),  social  support  (job  characteristics),  empowering  leadership  (leadership),  power  distance  (culture),  and  age  (demographics).  Perceived  courage  benefits  and  risks  did  not  mediate  the  effect  of  most  antecedents  on  behavioral  social  courage.  These  results  suggest  that  certain  antecedents  may  influence  behavioral  social  courage,  but  the  mediators  of  these  relationships  remain unknown. Further implications and directions for future research are discussed.
    The variable-centered approach is favored in management and applied psychology, but the person-centered approach is quickly growing in popularity. A partial cause for this rise is the finer-grained detail that it allows. Many researchers... more
    The variable-centered approach is favored in management and applied psychology, but the person-centered approach is quickly growing in popularity. A partial cause for this rise is the finer-grained detail that it allows. Many researchers may be unaware, however, that another approach may provide even finer-grained detail: the person-specific approach. In the current article, we (a) detail the purpose of each approach, (b) describe how to determine when each approach is most appropriate, and (c) delineate when the approaches diverge to give differing results. Through achieving these goals, we suggest that no single approach is the "best." Instead, the choice of approach should be guided by the research question. To further emphasize this point, we provide illustrative examples using real data to answer three distinct research questions. The results show that each research question can be fully addressed only by the appropriate approach. To conclude, we directly suggest certain research areas that may benefit from the application of person-centered and person-specific approaches. Together, we believe that discussing variable-centered, person-centered, and person-specific approaches together may provide a more thorough understanding of each. Keywords profile analysis, latent profile analysis, factor analysis, quantitative research To describe the differences between variable-centered and person-centered approaches, 1 Morin, Gagne, and Bujacz (2016) state, Whereas variable-centered approaches. .. assume that all individuals from a sample are drawn from a single population for which a single set of "averaged" parameters can be estimated,
    Suicide is the ultimate outcome of poor psychological well-being;however, there is a paucity of research examining the link between occupation and suicide, despite early academic interest and the known importance of work to our everyday... more
    Suicide is the ultimate outcome of poor psychological well-being;however, there is a paucity of research examining the link between occupation and suicide, despite early academic interest and the known importance of work to our everyday lives. We propose that this body of research was abandoned prematurely, and we provide a reanalysis by integrating the Job Characteristics Model and the Conservation of Resources  model  with  extant  suicide  research.  Specifically,  we hypothesize that work design characteristics (job autonomy, task variety, physical demands) and threats to personal resources (absence of viewing work-as-career, work-family conflict, family-work conflict,job dissatisfaction) are linked to suicide attempts via depression and suicidal ideation. Utilizing three measurement occasions and 2,855participants from the AddHealth database, our findings indicate that job autonomy, task variety, work-family conflict, family-work conflict,and job dissatisfaction all indirectly contribute to employees’ suicide attempts  via  depression  and  suicidal  ideation.  Thus,  negative employee perceptions of the workplace environment have much more severe consequences than is typically examined. Based on these results, we provide recommendations for developing a theoretically derived nomological net around suicidal behavior in an organizational context, and offer strategies for managers and employees to construct a work environment that is conducive to employee well-being.
    Core self-evaluations (CSE) and Psychological Capital (PsyCap) each consist of four first-order dimensions, and the two constructs are strong predictors of personal and organizational outcomes. Despite their similarities,much is still... more
    Core self-evaluations (CSE) and Psychological Capital (PsyCap) each consist of four first-order dimensions, and the two constructs are strong predictors of personal and organizational outcomes. Despite their similarities,much is still unknown regarding the manner in which the two constructs are distinct. Also, while both constructs are described as positive attributes, the most popular CSE and PsyCap scales include items that represent negative attributes. We suggest that these items may represent two alternative constructs–Negative CSE and Negative PsyCap. Through a series of three studies, we address these research questions. We show that, although strongly related, the constructs of CSE and PsyCap represent two distinct factors, even when the scale instructions provide a common temporal stability and/or specificity. We also show that the two most popular CSE and PsyCap scales are best represented by four factors that represent CSE, PsyCap, Negative CSE, and Negative PsyCap. As two separate dimensions emerged for the items describing negative attributes, this finding suggests that the factors representing the negative items are more than artifactors alone. We discuss the implications of these findings in regards to the measurement of CSE and PsyCap as well as the future study of Negative CSE and Negative PsyCap
    Recent technological advancements have enabled the widespread application of simulations in organizations, particularly for training contexts. Two important simulation elements, environment and control, have often been shown to improve... more
    Recent technological advancements have enabled the widespread application of simulations in organizations, particularly for training contexts. Two important simulation elements, environment and control, have often been shown to improve trainee outcomes. I argue that environment and control are reliant on each other, and their combined effects are explained by extending the Uncanny Valley Theory. The Uncanny Valley Theory pro-poses that individuals are comfortable with experiences that are very dissimilar or similar to reality, but are uncomfortable with experiences that fall between these conditions. In simulations, perceptions of realism are created through observations (environment) and interactions (control). Users are comfortable with experiences when these elements are in agreement; however, an Uncanny Valley effect may occur when these elements are in disagreement. In the current article, two studies analyze the realism of environment and control in predicting trainee reactions and learning outcomes. Both studies support the extension of the Uncanny Valley Theory to simulations. Simulations with only “low” or only “high” environment and control produce the greatest outcomes, and those with mixed “low” and “high” elements produce the worst outcomes; however, trainees did not differ in reactions to the simulations, indicating that the Uncanny Valley phenomenon in simulations may operate subconsciously.
    A recent advancement in the study of physical rehabilitation is the application of virtual reality rehabilitation (VRR) programs, in which patients perform practice behaviors while interacting with the computer-simulation of an... more
    A recent advancement in the study of physical rehabilitation is the application of virtual reality rehabilitation (VRR) programs, in which patients perform practice behaviors while interacting with the computer-simulation of an environment that imitates a physical presence in real or imagined worlds.Despite enthusiasm, much remains unknown about VRR programs. Particularly, two important research questions have been left unanswered: Are VRR programs effective? And, if so, why are VRR programs effective? A meta-analysis is performed in the current article to determine the efficacy of VRR programs,in general, as well as their ability to develop four specific rehabilitation outcomes: motor control, balance, gait, and strength. A systematic literature review is also performed to determine the mechanisms that may cause VRR program success or failure. The results demonstrate that VRR programs are more effective than traditional rehabilitation programs for physical outcome development. Further, three mechanisms have been proposed to cause these improved outcomes: excitement, physical fidelity, andcognitive fidelity; however, empirical research has yet to show that these mechanisms actually prompt better rehabilitation outcomes. The implications of these results and possible avenues for future research and practice are discussed.
    Studies on self-esteem instability (SEI) must obtain multiple measurements of self-esteem, such as daily for two weeks, to calculate SEI. This measurement method is called the“gold standard.”The intensive nature of the gold standard... more
    Studies on self-esteem instability (SEI) must obtain multiple measurements of self-esteem, such as daily for two weeks, to calculate SEI. This measurement method is called the“gold standard.”The intensive nature of the gold standard forces SEI be studied through multiple-day, controlled, student-sample designs, which restricts generalizability and possible variables studied. We attempt to address these concerns through creating a new single-administration SEI scale, called the SEI Measure (Studies 1 and 2), and analyzing the relationship of the SEI Mea-sure and prior single-administration SEI scales with the gold standard (Studies 3 and 4). The SEI Measure and existing scales only have moderate correlations with the gold standard, suggesting that they do not adequately gauge SEI. As these studies are the most robust investigation into single-administration SEI scales to date, we suggest that the study of such scales should be refocused, and our results also provide direction for new research avenues. Innovative single-administration SEI measures, such as implicit measures, may still adequately gauge SEI,and authors should consider perceived SEI separately from SEI itself. Perceived SEI may be an important aspect of self-perceptions that leads to distinct personal outcomes that are not predicted by SEI or other commonly studied variables.
    Purpose The current article reviews extant knowledge on courage and identifies a dimension of courage relevant to modern organizations, social courage, which is an (a) intentional , (b) deliberate, and (c) altruistic behavior that (d) may... more
    Purpose The current article reviews extant knowledge on courage and identifies a dimension of courage relevant to modern organizations, social courage, which is an (a) intentional , (b) deliberate, and (c) altruistic behavior that (d) may damage the actor's esteem in the eyes of others. Through a multiple-study process, quantitative inferences are derived about social courage, and the Workplace Social Courage Scale (WSCS) is created. Design Four studies using seven samples analyze the WSCS's psychometric properties, internal consistency, method effects, discriminant validity, convergent validity, concurrent validity, and utility. Many of these are investigated or replicated in largely working adult samples. Findings Each aspect of the WSCS approaches or meets specified guidelines. Also, social courage is significantly related to organizational citizenship behaviors, and the construct may relate to many other important workplace outcomes. Implications The current study is among the first to quantitatively demonstrate the existence of courage as a construct, and the discovered relationships are the first statistical inferences about social courage. Future research and practice can now apply the WSCS to better understand the impact of social courage within the workplace. Originality Despite many attempts, no author has created a satisfactory measure of courage, and the current article presents the first successful measure through focusing on a particular courage dimension-social courage. Future research should take interest in the created measure, the WSCS, as its application can derive future inferences about courage and social courage.
    Current methodologies in training evaluation studies largely employ a single method entitled random confirmatory trials, prompting several concerns. First, practitioners and researchers often analyze the effectiveness of their entire... more
    Current methodologies in training evaluation studies largely employ a single method entitled random confirmatory trials, prompting several concerns. First, practitioners and researchers often analyze the effectiveness of their entire omnibus training, rather than the individual elements or identifiable components of the training program. This slows the testing of theory and development of optimal training programs. Second, a common training is typically administered to all employees within an organization or work group; however, certain factors may cause individualized training to be more effective. Given these concerns, the current paper presents two training evaluation methodologies to overcome these problems: the multiphase optimization strategy and sequential multiple assignment randomized trials. The multiphase optimization strategy is a method to evaluate a standard training, which emphasizes the importance of a multi-stage training evaluation process to analyze individual training elements. In contrast, sequential multiple assignment randomized trial is used to evaluate an adaptive training that varies over time and/or trainees. These methodologies jointly overcome the problems noted earlier, and they can be integrated to address several of the key challenges facing training researchers and practitioners.
    Authors within the fields of cyberpsychology and human−computer interaction have demonstrated a particular interest in measurement and scale creation, and exploratory factor analysis (EFA) is an extremely important statistical... more
    Authors within the fields of cyberpsychology and human−computer  interaction  have  demonstrated  a  particular interest in measurement and scale creation, and exploratory factor analysis (EFA) is an extremely important statistical method for these areas of research. Unfortunately, EFA requires several statistical and methodological decisions to which the best choices are  often  unclear.  The  current  article  reviews  five  primary decisions  and  provides  direct  suggestions  for  best  practices.These  decisions  are  (a)  the  data  inspection  techniques,  (b)  the factor  analytic  method,  (c)  the  factor  retention  method,  (d)  the factor  rotation  method,  and  (e)  the  factor  loading  cutoff.  Then the article reviews authors’ choices for these five EFA decisions in  every  relevant  article  within  seven  cyberpsychology  and/or human–computer  interaction  journals.  The  results  demonstrate that authors do not employ the recommended best practices for most  decisions.  Particularly,  most  authors  do  not  inspect  their data  for  violations  of  assumptions,  apply  inappropriate  factor analytic methods, utilize outdated factor retention methods, and omit the justification for their factor rotation methods. Further,many  authors  omit  altogether  their  EFA  decisions.  To  rectify these concerns, the current article provides a step-by-step guide and checklist that authors can reference to ensure the use of recommended best practices. Together, the current article identifies concerns with current research and provides direct solutions to these concerns.
    Purpose An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula... more
    Purpose An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula for item-sort tasks and proposes a new statistical significance formula with greater utility across a wider range of item-sort tasks. Design First, a simulation study compares the two formulas in an array of conditions that vary on sample size and number of assignment choices. Second, an empirical study compares the results of three separate item-sort tasks across the two formulas for statistical significance. Findings In the empirical study, the proposed formula produces more correct retention decisions than the existing formula across all three item-sort tasks. In the simulation study, the proposed formula is more appropriate than the existing formula under most conditions. The two formulas function identically in item-sort tasks with only two assignment choices. Implications Researchers could obtain erroneous results when misapplying the existing item-sort task statistical significance formula to cases with more than two assignment choices. The proposed formula corrects this limitation , ultimately providing accurate results more often than the existing formula. Applying the proposed formula could help future research and practice throughout the scale development process. Originality Despite widespread use, few attempts have been made to improve scale-creation pretest methods, particularly item-sort tasks. The current study demonstrates that even conventional statistical methods are susceptible to misuse and misapplication, and future research could benefit from the reexamination of other common methods.
    Cyberpsychology is a recently emergent field that examines the impact of technology upon human cognition and behavior. Given its infancy, authors have rapidly created new measures to gauge their constructs of interest. Unfortunately, few... more
    Cyberpsychology is a recently emergent field that examines the impact of technology upon human cognition and behavior. Given its infancy, authors have rapidly created new measures to gauge their constructs of interest. Unfortunately, few of these authors have had the opportunity to test their scales' psychometric properties and validity. This is concerning, as many theoretical assumptions may be founded upon scales with inadequate attributes. If this were found to be true, then previous findings in cyberpsychology studies would need to be retested, and future research would need to shift its focus to creating psychometrically sound and valid measures. To provide inferences on this concern, the current study examines the article reporting, scale creation, and scale reliabilities of every article published in Cyberpsychology, Behavior, and Social Networking from its inception to July 2014. The final data set encompassed the coding of 1,478 individual articles, including 921 scales, and spanning 17 years. The results demonstrate that the simple survey methodology has become more popular over time. Authors are gradually applying empirically tested scales. However, self-created measures are still the most popular, leading to concerns about the measures' validity. Also, the use of multi-item measures has increased over time, but many articles still fail to report adequate information to assess the reliability of the applied scales. Lastly, the average scale reliability is 0.81, which barely meets standard cutoffs. Overall, these results are not overly concerning, but suggestions are given on methods to improve the reporting of measures, the creation of scales, and the state of cyberpsychology.
    Courage has seen an explosion of research in all branches of psychology, and the most popular measure is Norton and Weiss's the courage measure (CM). Despite widespread use, limited investigations into its psychometric properties and... more
    Courage has seen an explosion of research in all branches of psychology, and the most popular measure is Norton and Weiss's the courage measure (CM). Despite widespread use, limited investigations into its psychometric properties and validity have been performed. To ensure the strength of findings drawn from the scale, the current study performs a theoretical, psychometric, and empirical analysis of the CM. The results demonstrate that the CM has theoretical concerns stemming from the operational definition of courage used during its creation, and may not actually measure courage. Also, the CM was shown to consist of two dimensions separated by regular and reverse coding. Next, the scale demonstrated slight concerns with method effects, and its construct validity was analyzed. Together, the results demonstrate that the CM would benefit from the removal of reverse coded items, and may actually gage persistence despite fear rather than courage.
    Computer self-efficacy is an often studied construct that has been shown to be related to an array of important individual outcomes. Unfortunately, existing measures of computer self-efficacy suffer from several deficiencies , including... more
    Computer self-efficacy is an often studied construct that has been shown to be related to an array of important individual outcomes. Unfortunately, existing measures of computer self-efficacy suffer from several deficiencies , including criterion contamination, outdated wording, and/or inadequate psychometric properties. For this reason, the current article presents the creation of a new computer self-efficacy measure. In Study 1, an over-representative item list is created and subsequently reduced through exploratory factor analysis to create an initial measure, and the discriminant validity of this initial measure is tested. In Study 2, the unidimensional factor structure of the initial measure is supported through confirmatory factor analysis and further reduced into a final, 12-item measure. In Study 3, the convergent and criterion validity of the 12-item measure is tested. Overall, this three study process demonstrates that the new computer self-efficacy measure has superb psychometric properties and internal reliability, and demonstrates excellent evidence for several aspects of validity. It is hoped that the 12-item computer self-efficacy measure will be utilized in future research on computer self-efficacy, which is discussed in the current article.
    A vast multitude of online groups exist, and authors have been rapidly investigating their dynamics.Extant studies have provided great information on the effects of online group membership, but limitations are often noted in these... more
    A vast multitude of online groups exist, and authors have been rapidly investigating their dynamics.Extant studies have provided great information on the effects of online group membership, but limitations are often noted in these studies. Amongst the most concerning limitations are issues of generalizability. Authors are often unsure whether their results are able to generalize to other online groups, including those that are seemingly similar. For this reason, some researchers have created typologies of online groups, in hopes that online groups that fall within the same category will be generalizable; however, no study has analyzed the merit of an online group typology, and conclusions are based upon speculation. For this reason, the current study analyzed the dynamics of three different online groups, which fall within separate categories of an online group typology: a cancer support forum,a LGBT forum, and a Harry Potter fan forum. The results demonstrate that these groups vary in their properties, including group members’ group identity, well-being, and social support. These results provide support for an online group typology, and precisely demonstrate in what manner these groups differ. Additionally, the results offer valuable information about the individual groups, as some variables were previously unstudied in some group types. The discovery of these previously unknown dynamics leads to the potential of new studies, which is discussed. Therefore, the current study provides important implications for future studies, as well as the interpretation of future research results.
    Online groups have become more popular in recent decades, in both research and practice. Many authors have proposed important outcomes of group membership, and some have even investigated some preliminary dynamics of these online groups.... more
    Online groups have become more popular in recent decades, in both research and practice. Many authors have proposed important outcomes of group membership, and some have even investigated some preliminary dynamics of these online groups. Unfortunately, no validated measure of online group identity exists, causing these researchers to employ measures with poor psychometric properties or concerning construct validity. For these reasons, the current article undergoes a multiple study process to validate a measure of online group identity. In doing so, several aspects of online group identity are discovered,largely based on propositions previously posed for offline groups. Study 1 demonstrates that individuals from a general sample identify with online groups, and the measure has an identifiable factor structure.Using a naturally occurring online group, Study 2 shows the measure’s concurrent, convergent, and diver-gent validity, while simultaneously revealing many novel relationships of online group identity. Lastly,Study 3 investigates the effect of The United States President, Barack Obama, creating an account on the website studied in Study 2 and interacting with group members. The results of Study 3 reveal that online group identity did not change after this historic event, although members had notable emotional responses. In all, the current study illustrates the validity of an online group identity measure, and dis-covers many important relationships previously unknown in regards to online group identity. It is believed that the investigated measure of online group identity will become an important tool in future research, especially when further probing the relationships analyzed in the current study. Further implications and suggestions for future studies are discussed.
    Persistence is often viewed as a behavioral event, whereby a person works through obstacles in the pursuit of a goal. This operationalization does not speak to persistence as an individual difference, but many researchers have observed... more
    Persistence is often viewed as a behavioral event, whereby a person works through obstacles in the pursuit of a goal. This operationalization does not speak to persistence as an individual difference, but many researchers have observed people for which persistence appears habitual and is applied in the pursuit of all goals broadly. We suggest that several closely related constructs, such as grit and perseverance, may partially capture multiple dimensions of the overall construct of persistence. These dimensions are Persistence Despite Difficulties (PDD), Persistence Despite Fear (PDF), and Inappropriate Persistence (IP). In an initial study, we show that these three dimensions emerge in multiple measures for these closely related constructs, along with the construct of Goal Time Preference (GTP). Then, we create the Multidimensional Persistence Scale through a three-study process, showing that the scale has satisfactory psychometric properties and validity, and we support our three-dimensional conceptualization of persistence. Lastly, in two studies, we demonstrate that the three dimensions of persistence have notable and distinct relationships with personal well-being; however, only PDD appears to significantly influence organizational outcomes, supported through its significant relationship with organizational citizenship behaviors.