zyxwvutsrqp zyxw zyxwvut An Integrated Approach for the Analysis of Factors Affecting Journal Citation impact in Clinical Neurology Weiping Yue* The University of New South Wales, School of Information Systems, Technology and Management, Sydney, NSW, 2052 Australia. Email: wp.yue@unsw.edu.au Concepcion S. Wilson The University of New South Wales, School of Information Systems, Technology and Management, Sydney, NSW, 2052 Australia. Email: c.wilson@unsw.edu.au This study continues to develop and test an integrated conceptual model of journal citation impact. The main objective is to obtain a more complete understanding of how and to what extent the external factors affect journal citation impact. Four external factors have been identifiedfrom previous studies: journal properties, journal accessibility, journal internationality and perceivedjournal quality. A conceptual model of journal citation impact is developed and a web-based survey is conducted for obtaining information on the perceived journal quality. Structural equation modeling (SEM) with partial least squares (PLS) is used to test the conceptual models with empirical data from 41 research journals in clinical neurology. Interrelationshipsamong journal citation impact and the four external factors have been successfully explored. The results also suggest that, for an integrated conceptual model of journal citation impact, various citation indicators should be used. zyx zyx Nevertheless, researchers have realized that the citation process is complex and it is affected by certain external factors (Cronin, 1984; Gilbert, 1977). For example, Garfield (1979, p. 148), the founder of the Science Citation Index (SCI) and the journal impact factor, has indicated that “scientific merit is not always the sole reason an author will cite a paper published in a particularjournal ”. Garfield (1 979) further stated that an author’s citation behavior might be influenced to varying degrees by the visibility, prestige, and accessibility of the cited journal. Likewise, Cronin (1984) suggested that social and psychological factors as well as other external factors, such as the target audience or the character and status of a cited journal, affect the citing process. In order to use citation analysis for evaluating journals more objectively and comprehensively, we need a better understanding of what and how external factors influence the journal citation impact. To date, little effort has been made to explore the complex interrelationships among the various external factors with respect to the journal citation impact. Thus, how and to what extent do the external factors influence the journal citation impact in the context of journal evaluation become crucial issues. This study continues to develop and test an integrated conceptual model of journal citation impact (Yue & Wilson, 2003). The main objective of this study is to obtain a more complete understanding of how the factors affect journal citation impact. Although similar issues have been discussed in previous literatures, there is no study attempting to address the research problem at the conceptual level and to integrate all the operational external factors of journal citation impact into one model. In this study, four external factors have been identified from previous studies: journal properties, journal accessibility, journal internationality, and perceived journal quality. We argue that such a conceptual model involves latent variables or constructs that cannot be observed directly. More specifically, the journal citation impact itself and its external factors are regarded as constructs that only can be inferred by other observed zyxwvutsrq Introduction Based on the standard perspective of citation, which views citing as a merit-granting process, it is plausible to assume that the information value of a given publication is reflected by the frequency of citations obtained from other publications. Therefore, citation counts can be used as indicators or measurements of the level of quality, importance, impact, influence or performance of individual publications or aggregations of publications, such as journals (Wilson, 1999). In the past decades, citation analysis has been extensively utilized as a practical evaluation tool to assess journal performance in various disciplines. Journal citation indicators from the Institute for Scientific Information (ISI) have become popular measures for assessing journal quality, in particular, the journal impact factor and the number of total citations. ’ For correspondence. 2004 Proceedings of the 67th ASIS&T Annual Meeting, vol. 41 527 zyx zyxwvutsr zyxwvuts indicators. Therefore, our study not only identifies the external factors that affect journal citation impact but also identifies the observed indicators of the external factors and of the journal citation impact. Based on our prior research findings we developed a web-based survey to obtain the information on the perceived journal quality and incorporated the results in our model. The proposed model addresses the complex interrelationships among these external factors and journal citation impact in one case study, and further explores the extent to which the journal citation impact is influenced by external factors. Methodology Research Question This study identifies five constructs: journal properties, journal accessibility, journal internationality, perceived journal quality, and journal citation impact. It addresses the following interrelated research question: What is the strength of the relationship among the five constructs each in turn, and most importantly the relationship between the journal citation impact and the other four constructs? Models Developed This study uses structural equation modeling (SEM) with partial least squares (PLS) - a relatively new approach for testing multivariate models with empirical data. SEM incorporates both observed and latent variables; usually these variables can be separated into a structural model and several measurement models. The measurement models address the reliability and validity of the indicators in measuring latent variables or hypothetical constructs, while the structural model specifies the direct and indirect relations among the latent variables and describes the extent of explained and unexplained variances in the model. Instead of using co-variance based SEM techniques, PLS was used as the primary strategy for examining the relative effects of external factors on the journal citation impact, as it is suitable for “causal-predictiveanaIysis in situations of high complexity but low theoretical information” (Joreskog & Wold, 1982;p270). In addition, PLS makes no prior distributional assumptions and is applicable to small sample sizes (Sambamurthy & Chin, 1994). developed from the structural model. Figure 1 graphically depicts the propositions to be tested. Figure 1: Structural Model of Journal Citation Impact Journal Citation Impact: Impact is defined as a forceful consequence or strong influence. In this study, journal citation impact is used to refer to the degree of the journal influence as gauged by the uses made of the journals through citations. It is reflected through various journal citation indicators obtained or calculated from IS1 indicators. Journal Properties: This construct is partly an “editor-driven” concept, which describes the basic and historical attributes of a particular journal. Journal properties include, inter alia: age, size, frequency, multidisciplinarity, the number of review articles for a particular year, and the number of societal affiliations. Journal Accessibility: Journal accessibility represents the extent of the availability of a journal when needed by users. It is a multi-dimensional concept encompassing physical and content access of journals. Journal circulation (or subscription), language, and the number of Abstracting and Indexing (A&I) services or databases that choose to include a particular journal are indicators of accessibility, and thus, a mix of “publisher-controlled”, “editor-driven” and “reader- or user-choice” construct. Journal Internationality: Journal internationality refers to the extent of a journal being international or global in scope. It includes the indicators of the nationality (or country affiliation) of the authors, the editor-in-chief and members of the editorial board. Perceived Journal Oualitv: “Quality” refers to a degree of excellence or worth. In this study, perceived journal quality is a concept with respect to the general perception of a journal fiom peer assessment, obtained through a survey questionnaire, and thus purely a “reader- or user-choice” construct. zyxwvu zyxwvu zy Structural Model A structural model is regression-based and consists of the unobservable constructs and the theoretical relationships among them. In this research, the structural model has been developed on the basis of theoretical works of citation analysis and previous empirical fmdings of journal evaluation. Five constructs have been chosen for the structural model and seven propositions were 2004 Measurement Models Correspondingly, five measurement models have been developed for the five constructs in the structural model. Each measurement model consists of the relationships between the observed variables and the constructs Proceedings of the 67th ASIS&T Annual Meeting, vol. 41 528 zyxwv measured. Indicators for each construct are derived from those used in previous studies. When using SEM with PLS, the indicators in the measurement model can be modeled as reflective or formative. Reflective indicators are viewed as affected by the same underlying concept (i.e., the Latent Variable or LV). The change in the underlying LV will result in similar changes in its reflective indicators. Thus, reflective indicators should be correlated. In contrast, formative indicators are measures that form or cause the creation or change in a LV and they are not assumed to be correlated (Chin, 1998). Wold (1980), the developer of the PLS technique, suggested that in a situation of low theoretical knowledge, all independent LVs should be modelled in a formative mode and all dependent LVs in a reflective mode This recommendation is adopted in creating the model ofjournal citation impact for this study. In short, journal properties, journal accessibility, journal internationality and perceived journal quality are modeled formatively; whereas, journal citation impact is modeled reflectively. The five measurement models are shown with all their indicators in Figures 2 to 6. Measurement Model of Journal Citation Impact: The indicators of citation impact used in this study are either obtained or calculated from IS1 data. zyxwvuts zyxwvut zyxwvutsr zyxw Impact factor 1 \ /A Journal Citation Impact lrnrnediacyindex I Journal age Total citations Cited/Citing rate \ I zyxwvu Journal Properties I Multidisciplinarity No. of Reviews Citing author geographic location Figure 2: Measurement model of Journal Citation Impact Impactfactor: The journal impact factor is a measure of the frequency with which the ''average article" in a journal has been cited in a particular year. The IS1 impact factor for the year 2000 was used. Immediacy index: The journal immediacy index is a measure of how quickly the "average article" in a journal is cited and for this study, the 2000 immediacy index was used. Total citations: The indicator for total citations refers to the total number of citations to articles in a journal for the year 2000. CitedCiting rate: In previous studies, citedciting rate was regarded as a quality index of scientific journals (Price, 1981; Zinkhan & Leigh, 1999) or the inputfoutput 2004 ratio reflecting the status of the journal in a given network (Kim, 1992). In our study, this indicator is calculated by dividing the total number of citations that a journal received by the total number of references that the same journal has given; both numbers are for the year 2000. Citing disciplines: Citing disciplines refer to the extent of the inter- or multi- disciplinary impact of a journal. It can be traced through the analysis of subject areas from where citations originate (Wormell, 1998). The measure is the number of citing disciplines; this is obtained by using the Dialog information system to search all the citations to each journal, in our case study for the year 2000, and then ranking the journal subject category (SC) field. Citing Author Geographic Location: Geographic locations (GL) or country affiliations of all citing authors in a particular journal for the year 2000 can be obtained by searching Science Citation Index; further, the GL field can be ranked to produce a list of countries in decreasing order of productivity. Hence, journal citation impact with respect to the citing authors can be determined by the number of country affiliations of the citing authors. Measurement Model of Journal Properties: Journal properties are formed by the journal age, size, publication frequency, multidisciplinarity, number of review articles, and number of societal affiliations. Figure 3: Measurement Model of Journal Properties Journal age: Journal age was calculated from the year the journal was first published regardless of how many times the journal changed its title or altered its scope. Journal size: Journal size was defined as the total number of articles the journal published in the year 2000. It is coIIected from the Journal Citation Reports for 2000, and mainly includes the document types, articles and reviews. Frequency: Frequency of a journal is the number of issues the journal publishes each year. Multidisciplinarity: Each journal in the IS1 database is assigned at least one subject category, indicating a specific area of knowledge. Journals may be included in more than zyxw Proceedings of the 67th ASIS&TAnnual Meeting, vol. 41 529 z zyxwvuts zyxwvuts zyxwvutsr zyxwvut v zyxwvutsr zyxwvu < zyxwvuts one subject category; therefore, the extent or number of subject areas a journal covers can be an indicator of the journal’s multidisciplinarity. Number of Reviews: The total number of review articles in the journal for the year 2000 was regarded as an indicator of journal properties. Number of Aflliations: The number of affiliations a journal has with academic societies is another indicator of a journal’s property. Measurement Model of Journal Accessibility: The formative indicators of journal accessibility include: circulation, publication language of the journal, and the number of A&I services. Circulation Journal Accessibility Language piZFl Figure 4: Measurement model of Journal Accessibility Circulation: Although circulation size may over-represent a popular generalist journal and under-represent a scholarly specialist journal, it can still be used as a measure of a journal’s accessibility in a discipline. In this study circulation size from the Ulrich ’s database is regarded as an indicator of the journal’s accessibility. Language: More than 95% of journal articles in the SCZ for 1998 are in English and the role of English as the major language of scientific communication is ‘striking’ (Van Leeuwen, et al., 2001). Thus, the language of publication becomes an important indicator. It is treated as a ‘dummy’ variable; journals published in English were labeled ‘1’ while ‘2’ was assigned to the journals published in multi-languages. A&Z services: This measure is defined as the total number of A&I services or databases indexing a journal. The more A&I services indexing a journal, the wider the potential audience of readers (Ali et al., 1996). Measurement Model of Journal Internationality: Journal internationality is formed by the number of country affiliations of authors and editorial board members. In previous studies, this construct has been discussed from various aspects: the international scope of the readers (Wormell, 1998) and the editorial board (Zsindely et al., 1982); the geographical source of citations (Ren & Rousseau, 2002; Wormell, 1998); and affiliation data of the authors (Gutikrrez & L6pez-Nieva, 2001). In this paper, due to the difficulty of obtaining readership information from the publishers, we had to delete reader geographic location from the operational model. Alternatively, we could regard the actual users’ (that is, citing authors’) geographic location distribution as a 2004 measure to replace the readership internationality. However, due to the objectivity of our study, the citing authors’ geographic location is regarded as more related to the journal citation impact, and therefore, was considered as a reflective indicator of journal citation impact. Jitorial board GL Figure 5 : Measurement model of Journal Internationality Author Geographic Location: As with the country affiliations of the citing authors, the GLs of authors of documents in each journal for the year 2000 can be obtained. The method of assigning values to this measure for each journal is the same as that of assigning values to the citing authors. Editorial Board Geographic Location: The number of country affiliations of the editors-in-chief and of the editorial board members was also determined for each journal for the year 2000. Measurement Model of Perceived Journal Oualitv: In this study, perceived journal quality is formed by one indicator only - the results from a web-based survey. PmemeJ. Peer Perception Figure 6: Measurement model of Perceived Journal Quality Propositions (1) Journal DroDerties affect ioumal citation imDact: Numerous studies have found that journal properties influence different citation indicators. For instance, the frequency and the size of a journal have a positive effect on the value of the journal impact factor (Rousseau & Hooydonk, 1996). Joumal age has been regarded as the main factor for the number of total citations (Cooper et al, 1993; Dombrowski, 1988). Moreover, the coverage or scope of a journal is also regarded as a measure of citation impact (Dombrowski, 1988). The argument is that the more disciplines the journal content covers, the more people would use it and thus the more citations the journal would attract. The type of articles in a journal often influences the journal citation impact. For example, review articles generally attract more citations due to its special publication type. Further, Smart and Elton (198 1) also pointed out that the citation frequency could be explained by journal structural characteristics, such as journal size, references per article, and disciplinary focus. In short, as indicated by Cozens (1989), there is a need to analyze journal characteristics in terms of citation ‘inflators’ or ‘deflators’. Thus, journal properties are proposed to affect the journal citation impact. Proceedings of the 67th ASIS&T Annual Meeting, vof,41 530 zyxwvutsrq (2) Journal accessibility affects iournal citation impact: Previous studies indicate that one reason for not citing relevant works was the inaccessibility of the journal (Dubios & Reeb, 2000; Gorman, 2000; Van Dalen & Henkens, 1999; Cooper et al., 1993). The unavailability of the journal might be due to physical inaccessibility or other factors that hinder potential citers from using the work. For example, the language of publication can affect the number of citations (Dombrowski, 1988). Moreover, coverage by A&I services may also influence the citation rates of the journal (Cooper et al., 1993). The argument is that the more A&I services indexing the journal, the wider the potential readers (Ali et al., 1996) and the more opportunity for the journal to be cited. As journal accessibility may influence citation behavior, it is proposed in this study that journal accessibility affect journal citation impact. (3) Journal internationality affects iournal citation impact: Although the issue of journal internationality has been discussed, there are few studies investigating the relationship between journal internationality and journal citation impact. Theoretically, if a journal has a larger cohort of international ‘source’ authors and editors, then it is more likely to attract more citations and perhaps be more ‘influential’. However, with respect to the editorial board, Nisonger (2002) found no correlation between the international composition of the editorial board and two citation measures: the impact factor and total citations. Nevertheless, it is necessary to explore this issue further. Thus, it is proposed that journal internationality could affect journal citation impact. (4) Perceived iournal qualitv affects iournal citation impact: According to previous studies, perception of journal quality is also an external factor, which influence the number of citations the journal receives. For example, Dieks and Chang (1 976) stated that the ‘impact’ of a paper was not only to be determined by its scientific significance, but also by the prestige of the journal in which the paper is published. Likewise, Johnson (1997) pointed out that a ‘good’ journal increased the rate of citations. Anderson and Goldstein (1981) also indicated that the perceived reputation of a journal could be related to the citation counts to that journal from other journals. Thus, in this study, perceived journal quality is proposed to affect journal citation impact. ( 5 ) Journal prouerties affect perceived iournal quality: These indicators forming journal properties were thought to be potentially important determinants of journal quality (Bioscience, 1973; Brooks et al, 199 1 ;Zinkhan and Leigh, 1999). For instance, it is generally assumed that a journal would have an established reputation if it has been published for a substantial number of years (Ali et al., 1996). In addition, affiliation with professional associations was also proposed to be one of the factors in assessing journal quality (McCracken & Coffey, 1996). 2004 Normally a journal affiliated with a major society would have higher visibility than a journal not sponsored by a society (Bioscience, 1973). Hence, journal properties are proposed to have an effect on the perceived journal quality. (6) Journal accessibility affects perceived iournal auality A review of the relevant literature suggests that perceived journal quality is affected directly by journal accessibility (BioScience, 1973; McCracken & Coffey, 1996; Van Dalen & Henkens, 1999; Gorman 2000). Dubois and Reeb (2000) also indicate that there might be a relation between journal circulation and journal perception ratings. Further, the abstracting and indexing of a journal by secondary publication services has been regarded as a possible journal quality criterion (Zwemer, 1970; BioScience, 1973; Gorman; 2000). Based on prior research, it is proposed in this study that journal accessibility affect perceived journal quality. 17) Journal internationalitv affects uerceived iournal aualitv: Previous studies discussed journal internationality with respect to author geographic location and readership scope (Wormell, 1998; Vastag & Montabon, 2002). However, the relationship between journal internationality and perceived journal quality was not explored; if this relationship had been explored, then, we propose that journal internationality would have a positive relationship to perceived journal quality. zyxw zyxwvuts zyx zyxwvuts zyxwv Data Collection All the observed variables were collected from the following sources: Journal Citation Reports (JCR); Ulrich ’s International Periodicals Directory; Science Citation Index; National Libraty of Medicine’s LOCATORplus; the World Wide Web; printed versions of journals, and a survey of perceived journal quality. In order to produce a set of core journals in clinical neurology, we invited a clinical neurologist (also a journal editor of the European Journal ofNeurologv)to identify a set of journals for clinical neurology used by specialists like himself. Among 137 journals in the subject category of clinical neurology in ISI’s Journal Citation Reports for the year 2000,41 journals were identified and confirmed to be journals largely on clinical neurology rather than on neurosurgery, neurosciences, or psychiatry - closely related disciplines. Data collection is based on the above 41 journals from the clinical neurology journal subject category for the year 2000. In order to obtain a complete profile for each journal, those that have changed titles, merged with another journal or split into two or more journals are regarded as the same journal. Different data on journal age from different sources have been resolved by tracing the journal itself back in time. Proceedings of the 67th ASIS&T Annual Meeting, vol. 41 53 1 zyxwvuts A web-based survey of the perceived quality of journals in clinical neurology was designed to obtain information on peer assessment (see URL: This http://www.sistm.unsw.edu.au/people/wpyue). survey is the first attempt, to our knowledge, to identify and evaluate the perceived journal quality in clinical neurology. In the survey, respondents were asked to rate the journals on a five-point Likert scale that ranged from ‘Excellent’ to ‘Poor’. If they were unfamiliar or otherwise cannot rate the journal, they were asked to choose ‘Cannot Rate’. Respondents were also asked to provide demographic or general information about themselves, such as: research areas, current positions, the number of recent publications, etc. There were also some open-ended questions for respondents to write comments or to add and rate journals that they thought important in clinical neurology but not included in the list. Although respondents were encouraged to use the web to complete and submit the survey, we also provided other options: completing a Word-version of the survey and returning the result via E-mail, fax, or postal mail. With the support of the World Federation of Neurology (WFN), an e-mail message was sent to members of WFN (approximately 1500 members) inviting them to participate in the survey. The average values of the ratings for all journals were used as indicators of perceived journal quality. Over three-quarters of respondents indicated that they are academics, engaged in teaching, research, and clinical practice and their current positions are shown in Table 2. In addition, of the 196 (77.8%) respondents who have published at least one paper in a peer-reviewed journal between 1998 and 2003, 42 (21.4%) indicated that they have published more than 20 papers over the same period of time. Table 2. Current Positions CurrentPosition 1 No. (n=252) I Clinician Professor Associate Professor Assistant Professor Senior Lecturer Research Fellow Retried Sr. Research Fellow Emeritus Professor Lecturer Associate Lecturer Other No response Yo I 9.9 zyxw 4 4 3 3 2 2 2 9 1.6 1.6 1.2 1.2 zyxwvuts zyxwvut Results and Discussion Survey Results Within three months of the survey distribution, 252 validated and non-duplicate responses were collected for a response rate of ca. 17%. Most (238) of the responses were via the web, 9 via e-mail as Word attachments, 4 via fax, and 1 via postal mail. The responses were globally distributed and came from 49 countries; respondents from the USA ranked first with 19%. The 49 countries were grouped into major geographical areas and the distribution of respondents is shown in Table 1. Table 1. Distribution of Geographic Locations t I I 0.8 0.8 0.8 I 3.6 As expected, most (91.3%) respondents indicated that their principal focus of research is in clinical neurology; however, most were also engaged in other research areas such as neuroimmunology, neuropsychology, and neuroimaging, etc. The distribution of the research areas of the respondents is shown in Table 3. Respondents were able to nominate as many research areas as were applicable. The category ‘Other’ in Table 3 refers to other disciplines in neurosciences and more specific research areas in clinical neurology (eg. stroke, epilepsy, movement disorders). In the survey, we asked respondents to ‘position’ themselves into one of the ten-year age groups. Interestingly, about 42.1% of respondents were in the age group, 46-55. Respondents in this age group might well be in their most productive phase. Not surprisingly, the next two age groups are up to ten years younger or up to ten years older. Table 4 shows the age group distribution of the respondents in ranked order. zyxwvutsrq zyxwvut Europe Latin America North America Asia Middle East 95 56 50 I 17 16 2004 I I 37.7 22.2 19.8 6.7 6.3 Proceedings of the 67th ASIScETAnnual Meeting, vol. 41 532 zyxwvutsrq Table 3. Distribution of Research Disciplines Neurosurgery Psychiatry Neuroepidemiology Neiirorehahilitation 14 12 10 5 Other 101 nonparametric” (Chin, 1998, p3 16). Hence, the multiple R2 values given for the dependent construct is used to assess the predictiveness. It is equivalent to the value derived from the regression, which indicates the fraction of total variance in the dependent construct accounted for by those independent constructs (Chin, 1998; Mathieson et al., 2001). In our study, the R2 of the journal citation impact is 0.798, showing that the four external factors explained 79.8% of the variance of the journal citation impact. zyxwvu zyxwvuts 5.6 4.8 4 .O 2.0 I 40.1 Table 4. Age Distribution (*Significant at 0.05 level; **Significantat 0.01 level) ._ __ _.. 36 - 45 65 56 - 65 Over 65 Under 26 No response 51 25.8 20.2 13 1 2 5.2 0.4 0.8 zyxwvutsr zyxwvut zyxwvut Table 5 . Distribution of Highest Degree ~ Degree M.D. M.D.+Ph.D. M.A./M.Sc. DM Other No response I No. (n=252) YO 130 81 51.6 32.1 10 4 I 1 0.8 4.0 1.6 Structural Model PLS-Graph (version 3 .OO) was used to analyse the data. PLS has no requirement of the distributional assumption, and “evaluation of the PLS models should appl) prediction-oriented measures that are also 2004 Figure 7: Results of Structural Model Moreover, we chose “bootstrapping” to estimate the precision of our PLS estimates. Overall, four path coefficients were found to be significant at the 0.05 level; T-test for these four path coefficients are all >1.65, thus providing support for the corresponding research propositions 2, 3, 4, and 5. However, no significant influence was found for journal properties to journal citation impact, nor for journal accessibility and journal internationality to perceived journal quality. All the path coefficients between the research constructs are expressed in a standardized form to permit comparison of the relative strengths. The results of this study support the assumption that certain external factors influence the journal citation impact. In particular, journal accessibility (path coefficient of 0.499) has an important influence on the journal citation impact while journal internationality (0.3 11) and perceived journal quality (0.243) have only moderate influences on the journal citation impact. However, our study shows that journal properties have no direct influence (0.013 and not significant) on journal citation impact. Nevertheless, journal properties with a path coefficient of 0.464 demonstrate a strong influence on perceived journal quality. Measurement Models As to the measurement models, PLS also enables the assessment of measurement components by providing loadings and weights of indicators. In general, weights are appropriate for interpreting the effects for formative indicators while loadings are more suitable for examination of reflective indicators. Additionally, Proceedings of the 67th ASIS&T Annual Meeting, voi. 41 533 zy zyx zyx zyxwvutsrq zyx zyxwvuts bootstrap resampling also examines the significance of the weights and the loadings (Chin, 1998). Formative indicators are weighted according to their relative importance in forming the construct. Thus, the weights allow us to determine the extent to which each indicator contributed to the development of the construct. The general approach is to compare the weights of different indicators and to see if the weights are significant (Sambamurthy & Chin, 1994). Table 6 shows the weights for all formative indicators in the model, thus, showing extents of the reliability of the indicators for all the formative constructs. However, when some weights include the necessary constant to complete the PLS estimation procedure, weights can be > I . In our case, author internationality has a stronger weight. I Table 6. Formative constructs, indicators, and weights Construct 1 Indicator Weight Journal size 0.727** 0.088 Journal age Frequency 0.290 Journal Properties Number of affiliations 0.020 Number of reviews 0.013 Multidisciplinary 0.247* Language 0.108 Journal 0.777** Circulation Accessibility No. of A&I services 0.532** Author internationality 1.046** Journal Editorial Internationality 0.367** internationality Perceived Peer Assessment 1.ooo Journal Quality I I ** Sygnificant at the 0.01 level'(T-Statistic is above 2). On the other hand, each reflective construct is viewed as an existing entity. The extent to which each indicator reflects the underlying construct can be determined by examining the loadings. The rule of thumb is that the standardized loadings should be >0.707 (Chin, 1998). Moreover, for reflective constructs, we need to assess the internal consistency and discriminant validity. Composite reliability calculated by PLS is suitable for assessing the internal consistency while Average Variance Extracted (AVE) is appropriate for checking discriminant validity. It is suggested that the AVEs of LVs should be >0.500 and the composite reliability should be >0.700 (Chin, 1998). Overall, Table 7 shows that all loadings of the indicators for journal citation impact are >0.800 with a composite reliability of 0.947 and an AVE of 0.749, thus confirming our construct design ofjournal citation impact. The results also suggest that, for an integrated conceptual model of journal citation impact, various citation indicators should be used. 2004 Table 7. Reflective construct, indicators and loadings I Journal n:*-r: _- Loading Indicator Construct I Impact factor Immediacv index Total citations 0.822** 0.850** I I 0.885** ** Significant at the 0.01 level (T-Statistic is >2). Overall Model Discussion Our finding that journal accessibility has a significant influence on journal citation impact is consistent with discussions in prior research indicating that physical inaccessibility to journals would hinder their use and subsequent citings to them (Dubios & Reeb, 2000; Gorman, 2000; Van Dalen & Henkens, 1999; Cooper et al., 1993; Dombrowski, 1988). It also supports statements that the number of A&I services indexing a journal could be an indicator for an increase number of citations (Cooper et al., 1993). Among the formative indicators of journal accessibility, data in Table 6 c o n f m that circulation and number of A&I services are influential factors in forming journal accessibility. However, language contributes little to the formation of journal accessibility. The reason for this result might be due to the fact that most journals covered in our research are in English. Journal internationality had a medium influence on journal citation impact, thus confirming our proposition 3; in particular, the author internationality contributed greatly (1.046) to the journal internationality construct. This finding confirms a previous study showing that internationalization indicators correlated moderately with journal impact (Zitt & Bassecoulard, 1998). Likewise, editorial internationality with a weight of 0.367 also contributed significantly in forming the construct of journal internationality. Overall, in terms of geographic location, our study suggests that the broader the distribution of authors and editorial board members, the higher the citation impact of the journal. Another significant relationship in our model is between perceived journal quality and journal citation impact; this is consistent with discussions in prior research indicating that perceived reputation or prestige of a journal positively affects the journal citation impact (Dieks & Chang, 1976; Anderson & Goldstein, 1981; Johnson, 1997). However, it is surprising to see that perceivedjournal quality is not the most significant factor influencingjournal citation impact. zyxwvut Proceedings of the 67th ASIS&TAnnual Meeting, vol. 41 534 zyxwvuts zyxwv zyxwvuts zyxwvutsrqp zyxwvuts zyxwvu In this situation, there may exist two possible scenarios. Firstly, it is possible that a few high ‘impact’ articles in some less prestigious journals attracted a large number of citations and subsequently inflated their journal citation impact; however, the peer perception of these journals remained the same. Secondly, we found that respondents from different geographical regions rated certain journals very differently. Thus, the variation among respondents due to their geographical location and the subjectivity of the survey results may account for the relatively low influence from perceived journal quality to journal citation impact. Although, there is no statistical significance between journal properties and journal citation impact, we found that journal properties have a strong influence on perceived journal quality. Thus, through perceived journal quality, journal properties might have an indirect effect on journal citation impact. Among all the indicators of journal properties, journal size (0.727) certainly contributed greatly to the formation of the construct. The number of disciplines a journal covers also significantly contributed to the journal properties construct. However, as shown in Table 6,journal age, fiequency, the number of affiliations and the number of review articles did not contribute significantly to the development of the journal properties construct. Therefore, the most significant indicators influencing perceived journal quality are journal size and journal multidisplinarity. As we move into an electronic environment without ‘walls’ or ‘boundaries’, it is not surprising to find that journal accessibility and internationality do not affect the perception of journal quality. Our findings are not consistent with prior speculations that accessibility could influence perception (McCracken & Coffey, 1996; Todorov & Glanzel, 1988; Van Dalen & Henkens, 1999; Dubois & Reeb, 2000; Gorman 2000). However, our research goes beyond speculation; we have used empirical data to investigate the relationships. Additionally, our research also found that the international scope of authors and editors does not affect perception of journal quality. Conclusion In conclusion, inter-relationships among journal citation impact and four external journal factors have been successfully explored. Nevertheless, there are some limitations related to measurement reliability and research validity. Firstly, there are missing values in the circulation variable. Also, we need to bear in mind that circulation figures obtained from Ulrichs may not completely reflect the actual number of readers or users, particularly in the case of organizational subscriptions; nor are the figures always updated (upwards or downwards) from year to year. Secondly, the survey itself has limitation in having subjectivity and measurement errors. Further, like any statistical tool, PLS itself also has its bias in obtaining 2004 parameter estimates. The bias can be minimized by increasing the sample size and at the same time the number of indicators (Chin, 1998). Finally, other potential equivalent models are possible. Hence, it is suggested that issues concerning cultural and disciplinary biases need to be considered before generalizing our research fmdings. We suggest that future research be extended to other disciplines and to more journal sets. ACKNOWLEDGMENTS We thank Dr. Frangois Boller for his valuable comments on our survey design. We would also like to thank the World Federation of Neurology and its members for their support and cooperation. Finally we thank the two anonymous reviewers for their constructive comments. REFERENCES Ali, S.N., Young, H.C., & Ali, N.M. (1996). Determining the quality of publications and research for tenure or promotion decisions: a preliminary checklist to assist. Library Review, 45, 39-53. Anderson, P.J., & Goldstein, R.K. (1981). Criteria of journal quality. Journal of Research Communication Studies, 3, 99-110. BioScience Information Service. (1973). Biological Abstracts, 56. Brooks, C.H., Walker, L.R., & Szorady, R. (1991). Rating journals in health-care administration - the perceptions of program chairpersons. Medical Care, 29,755-765. Chin, W. W. (1998). The partial least squares approach for structural equation modelling. In George A. Marcoulides (Ed.), Modern Methods for Business Research (pp. 295-336). New Jersey: Lawrence Erlbaum Associates. Cooper, R.B., Blair, D., & Pao, M. (1993). Communicating MIS research - a citation study of journal influence. Information Processing & Management, 29, 113-127. Couens, S.E. (1989). What do citations count? the rhetoric first model. Scientometrics, 15,437-447. Cronin, B. (1984). The citation process: the role and significance of citations in scientific communication. London: Taylor Graham. Dieks, D., & Chang, H. (1976). Differences in impact of scientific publications some indexes derived from a citation analysis. Social Studies of Science, 6, 247-267. Dombrowski, T. (1988). Journal evaluation using Journal Citation Reports as a collection development tool. Collection Management, 10, 175-180. Dubois, F.L., & Reeb, D. (2000). Ranking the international business journals. Journal of International Business Studies, 3 1, - 689-704. zyxw Garfield, E. (1979). Citation indexing - its theory and application in science, technology, and humanities. New York: Wiley. Gilbert, G.N. (1977). Referencing as persuasion. Social Studies of Science, 7, 113-122. zyxwvut Proceedings of the 67th ASRXTAnnual Meeting, vol. 41 53 5 zyxwvutsr zyxwvuts zyxwvutsrqp zy Gorman, GE. (2000). Journal quality in the Asian region: Results of a pilot study for the IFLA round table of library and information science journals. Retrieved January 15, 2004 from http://www.ifla.org/IV/ifla66/papers/156-125e. htm. GutiCrrez, J., & L6pez-Nieva, P. (2001). Are international journals of human geography really international? Progress in Human Geography, 25,53-69. Johnson, D. (1997). Getting noticed in economics: The determinants of academic citations. American Economist, 4 1, 43-52. JBreskog, K.G., & Wold, H. (1982). The ML and PLS techniques for modeling with latent variables - Historical and comparative aspects. In: Jlireskog, K.G. and Wold, H., (Eds.), Systems under indirect observation: causality, structure, prediction Part Amsterdam; New York; Oxford: I (pp. 263-270). North-Holland publishing company. Kim, M.T. (1992). A comparison of three measures of journal status: influence weight, importance index, and measure of standing. Library and Information Science Research, 14, 75-96. Mathieson, K., Peacock, E., & Chin, W.W. (2001). Extending the technology acceptance model: The influence of perceived user resources. Data Base for Advances in Information Systems, 32, 86-112. McCracken, M.J., & Coffey, B.S. (1996). An empirical assessment of health care management journals a business perspective. Medical Care Research and Review, 53,48-70. Nisonger, T.E. (2002), The relationship between international editorial board composition and citation measures in political science, business, and genetics journals. Scientometrics, 54 : 257-268. Price, D.J.D.S. (1981). The analysis of square matrices of scientometric transactions. Scientometrics, 3, 55-63. Ren, S.L., & Rousseau, R. (2002). International visibility of Chinese scientific journals. Scientometrics, 53, 389-405. Rousseau, R., & Hooydonk, GV. (1996).Journal production and journal impact factors. Journal of the American Society for Information Science, 47,775-80. Sambamurthy, V., & Chin, W.W. (1994). The effects of group attitudes toward alternative GDSS designs on the decision-making performance of computer-supported groups. Decision Sciences, 25,215-241. Smart, J.C., & Elton, C.F. (1981).Structural characteristics and citation rates of education journals. American Educational Research Journal, 18,399-413. - 2004 - Todorov, R. and Glanzel, W. (1988). Journal citation measures a concise review. Journal of Information Science, 14,47-56. Van Dalen, H.P., & Henkens, K. (1999). How influential are demography journals? Population and Development Review, 25,229-251. Van Leeuwen, T.N., Moed, H.F., Tijssen, R.J. W., Visser, M.S., & Van Raan A.F.J. (2001). Language biases in the coverage of the Science Citation Iindex and its consequences for international comparisons of national research performance. Scientometrics, 51,335-346. Vastag G.,& Montabon F. (2002). Journal characteristics, rankings and social acculturation in operations management. Omega International Journal of Management Science, 30, 109-126. Wilson, C.S. (1999). Informetrics. Annual Review of Information Science and Technology, 34, 107-247. Wold, H. (1980). Model construction and evaluation when theoretical knowledge is scarce - Theory and applications of Partial Least Squares. In: Kmenta, J. & Ramsey, J.B., (Eds.), Evaluation of econometric models (pp. 47-74). New York: Academic Press. Wormell, I. (1998). Informetric analysis of the international impact of scientific journals: how 'international' are the international journals? Journal of Documentation, 54, 584-605. Yue, W.P., & Wilson, C.S. (2003). Measuring the citation impact of research journals in clinical neurology: a structural equation model analysis. In G Jiang, R. Rousseau, & Y. Wu (Eds.), Proceedings of the 9th International Conference on Scientometrics and Informetrics (pp. 362-368).Dalian: Dalian University of Technology Press. Zinkhan, GM., & Leigh, T.W. (1999). Assessing the quality ranking of the journal of advertising, 1986-1997. Journal of Advertising, 28, 5 1-70. Zitt, M., & Bassecoulard, E. (1998). Internationalization of scientific journals: a measurement based on publication and citation scope. Scientometrics, 41, 255-271. Zsindely, S., Schubert, A., & Braun, T. (1982). Editorial gatekeeping patterns in international science journals. A new science indicator. Scientometrics, 4,57-68. Zwemer, R.L. (1970). Identification of journal characteristics useful in improving input and output of a retrieval system. Federation Proceedings, 29, 1595-1604. - zyxwvu zyxwv zyxwvut Proceedings of the 67th ASIS&TAnnual Meeting, vol. 41 536