Alternative assessment metrics initiateve meeting 1 opening

Do We Need
Standards for
Alternative Metrics?
What are we hoping to
accomplish?
Todd Carpenter, Executive Director, NISO
NISO Alternative Assessment Initiative
Discussion Forum #1
San Francisco, CA | October 9, 2013
Friday, October 11, 13

• Non-proﬁt industry trade association accredited
by ANSI with 150+ members
• Mission of developing and maintaining technical
standards related to information, documentation,
discovery and distribution of published materials
and media
• Volunteer driven organization: 400+ spread out
across the world
• Responsible in various wasy for standards like
ISSN, DOI, Dublin Core metadata, DAISY digital
talking books, OpenURL, MARC records, and ISBN
About

We stand on the
shoulders of giants...

The Mighty Citation

Impact factor

Thomson-Reuters
IMPACT FACTOR

Two Stanford Students

They wrote a paper

They started a company in
someone’s garage
Photo:AP

Citations are awesome, aren’t they?
A perfect solution to all the world’s
problems, no?
A simple conclusion
can be drawn

“What Des-Cartes did was a
good step.You have added
much several ways, &
especially in taking ye
colours of thin plates into
philosophical consideration. If
I have seen further it is by
standing on ye shoulders of
Giants."
Back to Sir Isaac Newton

Citations
aren’t
always what
we think
they are
Caveat emptor

Another problem with citations

So where to from here?

Deﬁne altmetrics,
please

Usage Based Metrics

Clickstream Analysis

Social Media Metrics
Image: Danny Brown,The State of Social Media Marketing 2012

Clickstream data yields high-
resolution maps of science
Johan Bollen, HerbertVan de Sompel, et al.
PLoS One, February 2009
Clickstream network analysis

Behavioral Metrics

Multivariate analysis

Sentiment Analysis

Image: Domenico, Caron, Davis, et al.
Citation of non-traditional content
Citations to new
forms of
communication
aren’t as simple
as one might
think.
Deﬁne a use of
this?

We’ve got
what, now how?

Image: Flickr user karindalziel
What can be measured?

How do we measure
things?

At what granularity do
we measure things?

Which measurements
are valuable?

How do we ensure the
validity of those
measurements?

How to
(or even if we should)
deal with other issues
such as gaming?

Issues we face
with alternative
metrics

Basic Deﬁnitions
(So we are all talking
about the same thing)

Comparison across providers
Source:
Scott
Chamberlain,
Consuming
Article-‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,

Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.

Consistency in what is counted
Source:
Scott
Chamberlain,
Consuming
Article-‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,

Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.

Authorship
-----
Disambiguation
&
Contributor Roles

Open exchange of
component data

What is NISO trying to do?

Alterna(ve
Assessment

Ini(a(ve

Phase
1
Mee(ngs
October
9,
2013

-‐
San
Francisco,
CA
December
11,
2013
-‐
Washington,
DC
January
23-‐24
-‐
Philadelphia,
PA
Phase
1
report
expected
in
May
2014

Alterna(ve
Assessment
Ini(a(ve

Phase
2
Presenta(ons
of
report
(June
2014)
Priori(za(on
Eﬀort
(June
-‐
Aug,
2014)
Project
approval
(Sept
2014)
Working
group
forma(on
(Oct
2014)
Consensus
Development
(Nov
2014
-‐
Dec
2015)
Trial
Use
Period
(Dec
15
-‐
Mar
16)
Publica(on
of
ﬁnal
recommenda(ons
(Jun
16)

Criticism

Why now?
Isn’t it too soon?
Can NISO to do this?

How soon is now?

These things take time

It will be hard to change

“All I see now is
blonde, brunette,
redhead”

One Very Last Thing…
Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/

Thank you!
Todd Carpenter, Executive Director
tcarpenter@niso.org
National Information Standards Organization (NISO)
3600 Clipper Mill Road, Suite 302
Baltimore, MD 21211 USA
+1 (301) 654-2512
www.niso.org

Alternative assessment metrics initiateve meeting 1 opening

More Related Content

Alternative assessment metrics initiateve meeting 1 opening