INFO 532: Scholarly communication

Chapter Four: Evaluative Bibliometrics

Instructor:
DR. LAWRENCE ABRAHAM GOJEH
(ASSOCIATE PROFESSOR)
 TEST = 5%
• What does Collaboration refer to?
• List without explanation eight most
important patterns of collaborations as
learnt from your assigned reading in
chapter three.

• Time allowed = 5 minutes

 Evaluative Bibliometrics
• Citation Analysis: A Short History of the Use of
Citations as a Measure of the Impact of
Scientific and Scholarly Work
• Scientometrics: The Complementarities of
Scientometrics and Economics
• Patents of Bibliometrics.
 Chapter Objectives
• To explain Citation Analysis and discuss a
Short History of the Use of Citations as a
Measure of the Impact of Scientific and
Scholarly Work
• Explain Scientometrics and discuss the
Complementarities of Scientometrics and
Economics, and
• Explain Patents of Bibliometrics.
 Evaluative bibliometrics
• Evanuative bibliometric as introduced by Narin in 1976 is an
application of bibliometric that focuses on the evaluation of scientific
activity, and more in particular on quality aspects of scientific
performance.
• In general, evaluation in itself is focused on the control of quality, so
that, more specifically focusing research on the safeguard of
scientific quality.
• Winking at the tradition of library studies, the term “bibliometrics,”
coined by Alan Pritchard in the late 1960s, stresses the material
aspect of the undertaking:
– counting books,
– articles,
– publications,
– Citations.
 Citation analysis
• Citations
• Citations are becoming a major type of raw data for the study of
information.
• Citations are defined as the references placed at the end of any
scholarly paper, to those articles previously published that the author
has made use of in his own paper.
• Purpose or reasons for citation
• The purpose or reasons for citation are:
– 1. To pay homage to pioneers on the field.
– 2. To give credit as to methodology or equipment used in the
earlier work.
– 3. To provide background reading.
– 4. To correct ones work.
– 5. To correct others work.
 Citation analysis (cont.)
– 6. To criticize previous work.
– 7. To substantiate a claim.
– 8. To alert researchers to forth coming work.
– 9. To provide leads to poorly disseminated indexed or incited
works
– 10. To dispute priority claims of others.
• Nature of citation
• The terms ‘citation’ and ‘reference’ are often used interchangeably
but there is a subtle and important difference between their
connotations:
– a source document (or citing document) has reference (or
footnotes) usually listed at the end of the document, and
– appropriately linked to the relevant portions of the text.
 Citation analysis (cont.)
• Thus each reference identifies a bibliographic entry cited in the
source document.
• The term citation is used when one wishes to discuss the references
made to a cited document in one or more citing documents.
• Thus, the description of a bibliographic entity may be thought of
either as a reference (from the point of view of the citing document)
or as a citation (from the point of view of the cited document)
• Theory of citation
• Garfield said that:
– “high citation counts reflect impact but may not reflect intrinsic
worth. The data obtained from citation analysis are always
relative rather than absolute!"
 Citation analysis (cont.)
• Price noted that:
– "clearly the citing of a paper does not necessarily imply it has
been read and even more certainly that all papers read are cited,
but there is some significance with such citation".
• The explanation to Garfield's quotation shows that citation is
equated to use made of materials as evidence to the references
cited.
• The statistics here is the frequency count of number of journals of
how many times it is cited.
• A journal that is cited once reflects its insignificance but when cited
several times, then it shows its significance.
• High citation of journals calls for librarians to think of acquiring them
because of their importance on the field.
 Citation analysis (cont.)
• Price's quotation has highlighted materials; while the lack of citing
could be due to unavailability, deliberate, etc.
• Limitations of citation
• 1. Not all papers cited are read and not all papers read are cited. It
may be due to the unavailability of the relevant papers,
forgetting to record the source from which information was
obtained and failure to acknowledge writings had been used.
• 2. Its restriction to documents examined particularly to journals.
This obviously does not provide a clear picture of the total
communication system in the field. Although sampling is
essential in research, the strength of the citation analysis rests on
the size of the sample. This calls for the analysis of a large part
of the literature if valid conclusions are to be made.
 Citation analysis (cont.)
• 3. Despite these limitations, citation analysis is one of the
bibliometric methods that have been used extensively; particularly
in the sciences to studying communication patterns and
identified the most useful literature.
• Citation analysis
• Another major area of bibliometric research uses various methods of
citation analysis in order to establish relationships between authors
or their work.
• Citation analysis takes place, when one author cites another author,
a relationship is established.
• Citation analysis uses citations in scholarly works to establish links.
• Many different links can be ascertained, such as links between
authors, between scholarly works, between journals, between fields,
or even between countries.
 Citation analysis (cont.)
• Citations both from and to a certain document may be studied.
• One very common use of citation analysis is to determine the impact
of a single author on a given field by counting the number of times the
author has been cited by others.
• One possible drawback of this approach is that authors may be citing
the single author in a negative context (saying that the author doesn't
know what s/he's talking about, for instance)
• Citation analysis is a generic term for a set of well-known techniques
that have a long history in bibliometric studies of scholarly
communication. 
• As artifacts of the scholarly communication process, citations can
reveal formal communication patterns.
• Methods of citation analysis are unobtrusive and can be highly
reliable.
 Citation analysis (cont.)
• Citation analysis is generally regarded as a valuable tool for
determining the impact of scholarly works.
• Citation analysis allows for choosing institutional frameworks as
another perspective for aggregation. 
• In other words, the unit of analysis can be varied. 
• Citation analyses are normally conducted on the references of
thousands of print (and a few electronic) journals.
• Different rules of aggregation, however, refer to different dynamics,
and therefore the specification of different theoretical reflections is
expected. 
• Note that one has a dual problem in citation analysis: the unit of
analysis can be either text or author, and the level of aggregation
may vary. 
 Citation analysis (cont.)
• Dimensions of citation analysis at the micro-level using the following
cross-tabling:
– Citing Author
– Citing Text
– Cited Author
– Professional Relation
– Reward
– Cited Text
– Cognitive Resource
– Discursive Relation
• The specifications at the micro-level refer to (aggregated) systems that
generate specific hierarchies and dynamics by operating in a
distributed mode, that is, in terms of micro-events. 
 Citation analysis (cont.)
• The various (sub‑)systems participate in each citation to a greater or
lesser extent by interacting. 
• Rewards, for example, tend to help stratify the system into social
hierarchies, journals help to structure the communication, while
cognitive resources are aggregated in terms of more abstract
categories like theories. 
• Each event is a micro-operation, but the distributions of these
interactions can be organized along different axes. 
• This can be reflected using corresponding theories.
• Some reasons adduced for the popularity of this method are:
– i. Citation analysis does not require the active cooperation and
participation of the individual scholar.
•  
 Citation analysis (cont.)
– ii. Citation analysis does not limit investigator to subjects as they
exist within limited geographical areas or institutional settings,
since what is actually being analyzed are artifacts of research and
development.
– iii. Citation analysis can be accomplished without the assistance of
a highly formed subject specialist (there is no need for subject
back ground on the field). The limitation of citation is that it only
provides titles without the subjects treated in the titles.
• Advantages of citation analysis
• The advantages of citation are that it has an unobtrusive measure,
which means that writers are not aware that what they cited is going
to be subjected to study.
• The unobtrusive measure is a very popular method being used in the
area of bibliometrics.
 Citation analysis (cont.)
• Other methods are difficult to employ, e.g. Bradford's law, Lotka’s,
etc. because they require a database.
• But for citation analysis, the data base is already in existence, that is
to say that the materials used for citing citation analysis requires a
short period of citation, e.g five years and so on.
• Research topics
– 1. To what extent do reference librarians look outwardly from other related
fields to answer reference questions?
– 2. To what extent do information scientists make use of computers?
– 3. To what extent do library administrators, contributors make use of materials
outside library administration?
• NB: Recency of citation is very important in any research.
 Citation analysis (cont.)
• Variables to be analyzed
– 1. Author analysis: This is to discover the impact of individuals'
works on the field of his writing.
– 2. Subject analysis: For the impact of a subject, each subject will
be scored through providing two columns for scoring, to
determine the various subjects cited. High citation to that subject
of about 90%; suggests that the breath or scope of the subject is
narrow whereas high citation to several subjects would suggest
wider breadth to that subject. This has acquisition implication of
cutting across several fields in the acquiring of materials.
– 3. Language analysis: They are aimed at determining the
languages cited, so as to direct acquisition procedures towards the
most frequently cited languages. In most citation studies English is
the dominant language. This can be attributed to the sources used
for the citation studies.
 Citation analysis (cont.)
– 4. Journal citations: The analysis of journal citations are aimed at
identifying the most and least cited journals through the
application of the Bradford’s law. Cited journals can be grouped
according to zones of decreasing utility (Comment: when
analyzing citation of journals, you use cards to record the titles of
journals on each card. The cards would serve as a database as
well as being flexible; since they are alphabetically arranged from
A-Z). Scientist, make use of current journals because of currency
of information needed on the field.
– 5. Age of cited items: Analysis of citations by their dates of
publication is aimed at establishing whether the literature cited in
a subject is largely made up of current or retrospective materials.
 Citation analysis (cont.)
– For instance, it has been established that whereas
scientists make more use of current literature, humanities
scholars make more use of retrospective literature while
in between humanities and sciences the social science
scholars want not only retrospective materials but also
current materials for their data analysis. Such field as
History; require retrospective information and also current
data.
• For example, take age analysis of citation has an acquisition
implication for the librarian in meeting the needs of his
clienteles.
 Citation analysis (cont.)
• e.g: Age analysis: The period is indicated in backward in
time.
• 1986 = 2
• 1985 = 7
• 1984 = 5
• 1983 = 6
• etc. Total 20
•  Concept of Age analysis:
• a. Half-life: This is the number of years in which half the
total citation to a subject was made. For example, if the
total citation is 94; then half the figure of 94 is 47.
 Citation analysis (cont.)
• Therefore 47 stipulates half-life of citation which
corresponds to the period on the age analysis,
• e,g
• 1986 = 20
• 1985 = 12
• 1985 to 1971 = 15
• 1970 to1966 = 47
• Total = 94
 Citation analysis (cont.)
• Therefore half-life from the analysis is between 1986 and 1971, which is 15 years
• Half-life is a measure of whether a subject field as characterized as current or
retrospective.
• It is current if it is cited within the last five years.
• It is considered to be a subject in which people make use of current materials.
• This measure has been challenged that it is not scientific enough.
• That, that subject has the attribute of a scientific subject because science subjects
are characterized as having current materials and not based on half-life analysis
• b. Price Index: Price suggested that an index which he named after himself as
price index can be employed to advantage in in establishing the scientific nature of
a subject this index is based on the proportion of all cited references which years.

•  
 Citation analysis (cont.)
• The citations of a subject can be placed on a scale which ranges from
what he termed hard (sciences) through soft (science and technology)
and finally to what he called non-sciences
• According to Price if a subject has 42% or more of its citations dated
within the previous the previous five years, it is Hard science; if
between 42% and 20%, it is a soft-science, and a subject that has less
than 21% is a more non-science subject.
• The validity of the Price Index lies in the fact that it is based on
citation studies of 154 journals from various subjects and dates.
• The Index level for certain subject fields tends to fall consistently
within a certain range.
• Subjects in the humanities tend to have a low Price Index (i.e. they
have a low percentage of citation to publications written within the
previous five years), whereas, the natural science and technology
have a high-index.
 Citation analysis (cont.)
• The terms “hard”, “soft”, “non-science” were used by Price in the
context of the use of publications and not on the basis of how difficult
a subject is.
• Price also introduced a graph which is based on the index.
• He plotted a graph of references to papers on the vertical axis; against
the percentage of references entered in the previous five-years on
the horizontal axis.
• He then divided the graph into three columns.
• Papers with an index below 21% were described as “archival”, i.e.
based largely upon old papers.
• Those papers with an index between 21% and 42% were described as
“normal”.
• Those with 42% and above were described as “research fronts”.
 Citation analysis (cont.)
• These categorizations correspond with those that Price established
for subject disciplines:
– A Hard-science subject would be at the “research front” column;
– A “soft-science” would be located in the column labeled “normal”;
– A non-science subject would lie in the “archival” column.
• “Price was able to classify science into “hard”, and “soft” on the basis
of the number of references of each paper. He considered a paper as
scholarly only if it had more than 10 references. However, his
concentration seemed to be on the social linkage aspect of the
citations.”
•  KEY
• Y-axis = Literature of a given year
• X-axis = Total number of citation
• Note: The horizontal axis cannot be more than 100%
 Citation analysis (cont.)
• Co-citation/Bibliographic coupling:
• If two authors; jointly cited a work can be indicated to be a co-
citation.
• This suggests significance and relatedness of the paper cited. It is also
called “Bibliographic Coupling”.
• For example, Document “A” cited document “C” document “B” also
cited document “C”. Co-citation is said to have taken place.
• Figure 1 shows the co-citation ship.
•   A
•  
• C
• B
• Figure1: Co-citation of document “C”
 Citation analysis (cont.)
• Implications:
– 1. Document ‘A’ and ‘B’ are said to be somewhat related.
– 2. There is element of significance of document ‘C’.
– 3. Co-citation leads to document clustering or groups of document.
•  Citation Indexes
• A citation index is the indexes of cited works. In Nigeria, it cannot be
used as a database for Bibliometrics studies.
• Suppose an article by an author ‘X’ library history in Journal of
Librarianship volume 20 (2) of 1964 pp.3-25 (cited work).
• Beneath it will be those who have cited the above article.
• Under it, are different authors in different years citing the work,
 Citation analysis (cont.)
• e.g.
• Citing article∑ 1. ------------------------------------------
• 2. ------------------------------------------
• 3. ----------------------------------------
• 4. ---------------------------------------
• These are citations to support a claim.
• The whole citation index was borrowed from the legal field.
• The science citation index (SCI) was published Eugene Garfield being
a lover of citation index that can be used to reveal many things.
• For example, it is significant because it enables one to know whether
a work has been cited or not.
• In general, citation reveals usage and selection or choice of journals
for inclusion in library collection.
 Citation analysis (cont.)
• According to Garfield (the President of the publishers’ of science
citation index) that it is significant if the listing is many that it also
reveals the measures of epidemic theory.
• Some Bibliometricians, use SCI as their database.
• In this case, in Ethiopia, it is not advisable to use the SCI because the
indexes are not up-to-date due to omission of journals published in
Ethiopia.
• But where it is possible to get a complete index, then it is a potential
data source or base.
• Bibliographic coupling
• Bibliographic coupling occurs when two works reference a common
third work in their bibliographies.
 Citation analysis (cont.)
• The coupling strength is higher; the more citations the two bodies
have in common, and this coupling is used to extrapolate how similar
the subject matter of the two works is. Bibliographic coupling is
invaluable in all fields of research since it helps the researcher to find
related research done in the past.
• A closely-related notion is the "co-citation index," which refers to
the number of times two works are cited together in subsequent
literature.
• The term "bibliographic coupling" was first introduced by MM Kessler
in a paper published in 1963.
• Others have questioned the usefulness of the concept, pointing out
that the two works may reference completely unrelated subject
matter in the third.
 Scientometrics
• Scientometrics is the science of measuring and analysing
science.
• In practice, scientometrics is often done using bibliometrics
that is measurement of (scientific) publications.
• While bibliometrics is a statistical significant , which
manifest in any recorded information, regardless of
disciplinary bounds.
• Scientometrics emphasizes and encompasses all
quantitative aspects and models related to the production
and dissemination of scientific and technological
knowledge.
 Scientometrics (cont.)
• Provided some preliminary assumptions about what
science actually is and how a true scientific achievement
is to be recognized,
• It ultimately addresses the quantitative and comparative
evaluation of:
– scientists’,
– groups’,
– institutions’, and
– countries’ contribution to the advancement of
knowledge.
 Scientometrics (cont.)
• Modern scientometrics is mostly based on the work of Derek J. de
Solla Price and Eugene Garfield.
• The latter founded the Institute for Scientific Information, which is
heavily used for scientometric analysis.
• One significant finding in the field is a principle of cost escalation to
the effect that achieving further findings at a given level of
importance grow exponentially more costly in the expenditure of
effort and resources.
• Economists and historians of science, for example, use bibliometric
indicators to measure productivity and eminence.
• Patents indicate a transfer of knowledge to industrial innovation and
a transformation into something of economic and social value; for
this reason they constitute an indicator of the tangible benefits of an
intellectual and economic investment.
 Patents of Bibliometrics
• A patent is a legal document issued by a governmental agency that, in
exchange for the public disclosure of the technical details of an
invention, grants the inventor, or any person or organization to whom
the inventor’s prerogatives have been transferred, the monopoly on
its production and commercial exploitation.
• The right holds, as long as certain fees are paid, within the boundaries
of the issuing agency’s country.
• For example, the U.S. patents are granted by the United States Patent
and Trademark Office (USPTO) for a period of time that begins at the
date of issue and ends twenty years from the date the application was
filed.
• Since most inventions are, in large part, enhancements built upon
previous objects or techniques, the ultimate verification of
patentability calls for an in-depth analysis of the invention’s technical
specifications by a skilled examiner.
 Patents (cont.)
• The patents are vehicle of protection of intellectual
property rights emanated from scientific projects or
scientific discoveries.
• A new product or process or technique derived from a
scientific research work, which has certain applications for
the betterment of human life, is patentable and inventors
can claim it as their intellectual property by registering it
with patenting authorities by following certain legal
procedures.
• All these channels of scholarly communication are
popularly known as primary sources or original sources or
primary literature.
 Patents (cont.)
• Prior art disclosed in scientific literature and in earlier subject-related
patents is especially relevant to the examination process insofar as
the occurrence of a similar idea or invention, anywhere and at any
prior time, is the basis for the final judgment on either accepting or
rejecting the applicant’s claims.
• A typical U.S. patent is composed of three basic sections:
– 1. A title page, containing bibliographic data and practical information useful
to identify the document unambiguously: title, abstract, classification number,
name and address of both the inventor and the assignee, date, application
number, and patent number.
• When a U.S. patent is granted, the title page also contains a list of bibliographic
references supplied by the patent examiner; they are the building blocks of
most patent citation analyses.
 Patents (cont.)
– 2. The description of the invention, explaining how to make and use it. This
includes drawings, technical specifications, and, scattered throughout the text,
the references to prior relevant literature supplied by the inventor.
– 3. The claims defining the scope or boundaries of the patent, that is, the
specific features of the invention for which legal protection is being requested.
• Patents, like many other human artifacts, have long been of interest
to economists concerned with the output of scientific and
technological research and its correlation with standard indicators of
economic performance, such as the Gross Domestic Product.
• Indicators based on output measures, however, are of limited use for
the assessment of the actual value of patented inventions: simply
counting and classifying patents doesn’t tell anything about the
weight of each patent’s contribution to economic and technological
advancement.
 Patents (cont.)
• So, following the same trajec-Impact Factor and the Evaluation of
Scientists history of scientific performance indicators, traditional
econometric statistics based on output measures were
supplemented, at a certain point, by patent citation analysis.
• To be accomplished at best, such extension should have been driven
by a catalyst for raw data harvesting comparable to the SCI, with
patents as source documents instead of scientific papers.
• The initial lack of a standard tool, by contrast, caused patent citation
analysis to split its empirical base into different local data files
occasionally growing out of the two parallel, seemingly unrelated
research traditions of bibliometrics and econometrics:
– the former’s initial input came from information retrieval, but soon found its
way to an evaluative arena firmly rooted in the long-established conceptual
and methodological framework of scientometrics,
 Patents (cont.)
– whereas the latter simply took for granted what many bibliometricians still
consider a puzzle to be solved, namely the significance of citations as quality
indicators.
• The idea to use citations as an aid to effective patent searches
alternative (or complementary) to subject-based classification codes
was circulating among American patent attorneys as early as the
1940s.
• A decade later it was put into concrete form by Garfield who,
inspired by a proposal of Arthur H. Seidel, tested a patent citation
index to 4,000 chemical patents in 1957.
• Its official version, published in the 1964 and 1965 editions of the SCI,
included as sources all U.S. patents, but was soon dropped for lack of
financial support.