No - HKUST Institutional Repository

Analyze and Improve
Your Citation Counts
Library-VPRDO Joint Workshop for
Faculty & Research Staff
Presenters:
Mr. Ken Cheng (Program Manager, Knowledge Transfer)
Ms. Victoria Caplan (Subject Librarian for SS)
Ms. Diana Chan (Associate University Librarian)
Date: December 8, 2009
Venue: HKUST Library
Analyze & Improve Your Citation
Counts
December 8, 2009
Presented by
2
Victoria Caplan
[email protected]
Diana Chan
[email protected]
Learning Objectives
1. Learn about New Search Tools & Techniques
Scopus
GoogleScholar & Publish or Perish™ open source software
GoogleBooks
Proquest
Web of Science (Social Science Citation Index)
2. Recent Developments in HSS Bibliometrics
3
Scopus - Search
Scopus (1996-) = ~18,000 titles
• Has ~ 2,760 Social Science titles (all of ISI’s)
• As of 2009 ~3,500 Arts & Humanities titles
Author Search- Lee, James Z
Affiliations
CalTech
Univ. Michigan
Results
5 docs & 19 citations
Citation report
4
Scopus Demo (2)
5
Google Scholar – Advanced Search
Author Search
 Author search for “JZ Lee” & Limit to HSS
Contents: = Mystery box
“…articles, theses, books,
abstracts, court opinions and
other scholarly literature
from all broad areas of
research. …from a wide
variety of academic publishers
and professional societies, as
well as scholarly articles
available across the web.”
6
~ 27 docs, cited ~ 320
 Author search for “James Z Lee” & no limit
~ 40 docs, cited ~ 370
 Author search for “李中清”
~ 35 docs, cited 55 times
 Author search for ZL Lu –
~3,000+ science papers in many disciplines
 Author search for 呂宗力
= 5 documents, cited 80 times
Publish or Perish® with GoogleScholar
http://www.harzing.com/pop.htm
Designed to help individual academics present
their case for research impact
Better for Social Sciences
than for Humanities
&
At moment, no 中文
7
Uses Google Scholar to obtain raw citations,
then analyzes them & presents statistics:
•Total number of papers
• Total number of citations
• Average number of citations per paper
• Average number of citations per author
• Average number of citations per year
• Etc., etc., etc.
Google Scholar Advanced Search Phrase Search
“Lee, James Z” as phrase = 81 docs
“Lee JZ” as phrase = ~100 docs
- but some are the urologist’s
李中清 as a phrase = 184 docs
“Lu Zongli” as a phrase = 17
Searched “Lu ZL” as a phrase = ~3,670 docs
almost all wrong
Searched 呂宗力 as phrase = 27 docs
8
GoogleBooks - Advanced Search
Phrase Search
Searching Lee, James Z. as a phrase = 169
Searched李中清 as a phrase = 417
Searching Lu Zongli in GoogleBooks as a phrase = 71
呂宗力 in GoogleBooks as a phrase = 305
9
Warning: Can be Slow & Tedious to sort through
Proquest - Search name as full-text, to see if it
appears in bibliography
Lee, James Z as a phrase search
= 20 documents
mostly to PhD theses
10
“Lu, Zongli” as a phrase search
= 6 docs (5 OK)
all citations in PhD theses
This “mining” is also slow and not quite as rewarding as GoogleBooks
Web of Science - ISI Indexes: SSCI & A&H CI
Social Science Citation Index (1991-) (covers 2,759 journals)
Mediocre for some Social Sciences Faculty members, & poor for others
Quite bad in the Humanities
Example:
Only 2 citations for Dr. James Z. Lee
Some senior members of HSS had NO Listings
Arts & Humanities Citation Index
(covers 1,516 journals)
About the same (if not worse)
Example:
Only 1 article found for Dr. James Z Lee
Other senior members had results like this:
4 articles found, no cites listed
1 article found, no cites
0 articles, 0 cites
11
Refer to hand-out for sampling of journal coverage
Social Science Citation Index
Author Search
Lee, JZ
Found 3
Only 2 correct
12
Social Science Citation Index
Cited Reference Search
Cited Ref search for Lee, JZ
• 37 docs
• Cited 100 times
• 50 FALSE hits
Biggest false hit generator (41 citations):
13
Lee SD, Sohn DW, Lee JZ, Park NC, Chung MK (2000). An epidemiological study of
enuresis in Korean children. British Journal of Urology International, 85(7), pp. 869-873.
Searching Comparisons
Author
Web of Science
Scopus
Proquest
Google Scholar
Google Books
Author A
2 docs,
cited 5
5 docs,
cited 19
20 hits
144 articles,
cited 427
170 hits
Author B
0
0
5 hits
Chinese name
12 docs, 77
cites
Chinese name
306 hits
English name
4 docs, 3 cites
English name
= ~70 hits
107 docs
cited 965
~343 hits
Author C
14
26 docs,
cited 71
23 docs,
cited 158
~ 80 hits
Database Availability at HKUST
15
Major Databases
Here at HKUST Useful ?
Social Sciences Citation Index
(1991-) - 1,900 journals
Yes
OK for scholars in Economics or
Psychology, less so for historians,
anthropologists, etc.
Arts & Humanities Index
No
Not very useful
Scopus (1996-)
Yes
OK for Econ & Psych; increasing
coverage in the humanities
GoogleScholar
Open Access
Yes, but must be mined manually
GoogleBooks
Open Access
Yes, but must be mined manually
Proquest
Yes
Yes, but must be mined
Chinese Citation Database
(中國引文數據庫)
No
Have not evaluated
2) Recent Developments in HSS Bibliometrics
The 4 Literatures of HSS
- International Peer Reviewed Articles
- Books
- National Literatures (non-English)
- Enlightenment literature
Other tools and methods
- LibCitations
- Linmans’ Proposals
16
Part 2 - The Issue: “Just” Publishing Isn’t Enough Anymore
17
Mischa Richter. Licensed from the www.catoonbank.com Aug 14, 2009
Citation Counting Concerns & Issues in HSS
“Bibliometric evaluation in the social science is possible. However,
if done correctly, it is messy, complex, and expensive”
- Hicks (2006)
“[G]iven the specific nature of the Humanities, the indicators…
will most of the time not be a mere copy of existing science
indicators.
- Linmans (2009)
18
The Four Literatures of Social Science*
1. International Journal Articles
2. Books
3. National Journals (non-English language)
4. Enlightenment Publications (”popular” press & media)
- Hicks (2004)
*
19
And Humanities?
International Peer Reviewed articles
1st literature of social sciences (less so in Humanities)
• Internationally oriented
• Peer reviewed journal articles
• English language (most often)
Need a decade-long window to examine Social Science Literature
(Glanzel, cited in Hicks 2006).
Interdisciplinary work makes scattered citation patters
79% of Science papers referenced other ISI journal papers
Only 45% of the Social science papers referenced other ISI papers
- Leydesdorff ( cited in Hicks 2006)
20
Books = High Impact
Books are more cited than journal articles
3:1 or as high as 6:1
20 most highly cited articles averaged ~55 citations
20 most cited books ranged up to 512 citations
Sociology Example:
Most cited books received 79% of citations from OUTSIDE of sociology
Most cited articles received 55% of citations from OUTSIDE Sociology
(Hicks 2006)
Neither books nor articles can be ignored
21
Books often = Tenure & Promotion
“So, fair or not, the basic standard for tenure on my campus is two books.”
- (Schmidt 2007)
“The book is clearly required for promotion and tenure in the humanities
and social science disciplines “
- (Estabrook 2004) from a study done with faculty members in 18
institutions of CIC
22
Books – a study at HKU
Physics dept.’s output: 99%
Music Dept’s output: 11%
= journal articles,
= journal articles
Books, monographs, & book chapters
• Sociology,
• Education
• Philosophy
• Linguistics
• Comparative Literature
> journal articles in these depts.
No Natural Science department produced a book in those years
23
- (Huang & Chang 2008)
Non-English Language material “National Literatures”
“Each national literature is a world unto itself ”.”
English language sociology = ~ 70% of literature in the area
English & US sociologists cited English language material 99% of the time
German & French language sociology = 10% of literature in the area
German & French sociologists cited own language material 60% of the time
Things are changing: Slow Convergence:
1980-2000
Norwegian social scientists citation of English language journals
increased from 33% to 50%
24
Enlightenment Literature
“Popular or “Enlightenment” writing & media presentations
Contributions to “public debate”
= Newspapers, magazines, news shows, etc.
In Norway, twice as many social scientists do it than scientists
English Linguistics, Education, & sociology lead in share of popular
literature
Some think that these publications might also need to be counted to show
a true picture of a scholar’s productivity.
25
Other possible tools: LibCitations
 New method (2009 article)
 Count the no. of libraries holding a given book as reported in a national
or international union catalog (WorldCat)
 Count its holdings pattern in comparison to other books in class no.
 Measures can be then derived to compare different departments (in
different universities) for cultural impact
 Political Advantage = makes Humanities & Social Sciences authors look
better, especially those who don’t show up well in ISI Thomson or Scopus
databases
 Problem = Can make other departments look bad
26
Another Idea – Linmans’ Proposals
3- Level approach: citations, library holdings, & productivity
 Citations = Not citation per publication, but citation per author
 Looking at Library holdings of books (including WorldCat) – help
overcome Anglo-American bias of WOS
 Look at lifetime productivity instead of shorter window sometimes
suggested by Hirsch
Linmans (2009)
27
Conclusion - One Source of Citations Is Not Enough
International Peer Reviewed Articles
• Use SSCI, Scopus & Google Scholar
• Use other commercial Databases like Proquest, Ebsco, etc.
Books
• Do manual searching of SSCI to for books
• Do manual searching of GoogleBooks, GoogleScholar
• Do manual searching of citation databases like Proquest & Ebsco,
National Literatures
Try searching names of authors as “full text” in National databases (e.g. CAJ)
Newspapers, Magazines, & other Enlightenment literature
Consider listing your contributions to newspapers and magazines & videos
Or consider list them as “service” activities
28
Bibliometrics in HSS: Fun, but….
“Humanists, who are quite used to merciless criticism in essay
form, may someday have to decide whether it is worse to have
their hearts broken qualitatively or quantitatively.”
-White, et al. (2009)
29
References
Estabrook, Leigh. (2004). Book Publication as the Gold Standard for Tenure & Promotion, a presentation given at the 144th Membership Meeting
of the Association for Research Libraries, Tucson Arizona, May 14, 2004. Accessed 3 December 2009 from
http://www.arl.org/arldocs/resources/pubs/mmproceedings/144/estabrook_files/estabrook.ppt.
Harzing, A.K.and Wal, R.D. (2008) “Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, Vol.8:
p. 61-73.
Hicks, Diana (2004). The Four Literatures of Social Science. In: H.F. Moed et al. (eds.), Handbook of Quantitative Science and Technology
Research, Chapter 21, p. , 473-496.
---- (2006) The Dangers of Partial Bibliometric Evaluation in the Social Sciences, Economia Politica23 (2) August, pp. 145-162. Accessed
12 nov 2009 from http://works.bepress.com/cgi/viewcontent.cgi?article=1008&context=diana_hicks
Hicks, Diana; Wang, Jian. (2009). Towards a Bibliometric Database for the Social Sciences and Humanities. Accessed 12 November 2009
at http://works.bepress.com/cgi/viewcontent.cgi?article=1017&context=diana_hicks
Huang, Mu-hsuan; Chang, Yu-wei. 2008. Characteristics of Research Output in Social Sciences and Humanities: from a Research
Evaluation Perspective. Journal of the American Society for Information Science and Technology, 59(11), p. 1819-1828.
Kousha, Kyavan; Thelwall, Mike. 2009. Google Book Search: Citation Analysis for Social Science and the Humanities. Journal of the
American Society for Information Science and Technology, 60 (8):1537-1549.
Linmans, A. J.M. (2009). Why with bibliometrics the Humanities does not need to be the weakest link. Scientometrics, published online 13
August 2009. Accessed 15 November from SpringerLink database.
Moed, Henk. F. , Glänzel, Wolfgang, Schmoch, Ulrich (eds.). (2004) Handbook of Quantitative Science and Technology Research. Dordrecht:
Springer, doi 10.1007/s11192-009-0088-9
Moed, Henk F. (2005). “Differences Between Science, Social Sciences and Humanities” in Chapter 9, Citation Analysis in Research Evaluation.
Dordrecht: Springer.
Schmidt, Thomas, (2007). How Not to Write a Second Book. Chronicle of Higher Education, March 30, 2007. Retrieved from Ebsco
Database 3 December 2009.
White, Howard D., Boell, Sebastian K.;Yu, Hairong; Davis, Mari; Wilson, Concepcion S.; Cole, Fletcher, T.H. (2009). LibCitations: A
Measure for Comparative Assessment of Book Publications in the Humanities and Social Sciences. Journal of the American Society for
Information Science and Technology, 60(6):1083-1096.
30