Volume 5 Issue 5 - August 15, 2008
H-index: A New Trend in Evaluating Research Quality
Article Digest
Ching-Rong Wen
Continuous-time photoelectron spectroscopy for monitoring monochromatic soft x-ray photodissociation of CF3Cl adsorbed on Si(111)-7×7
Ching-Ting Lee
AlGaN/GaN Metal-Oxide-Semiconductor High-Electron Mobility Transistors Using Oxide Insulator Grown by Photoelectrochmical Oxidation Method
Ciann-Dong Yang
Nonlinear Dynamics Governing Quantum Transition Behavior
Hui-Tzu Min
EFL Vocabulary Acquisition and Retention: Reading Plus Vocabulary Enhancement Activities and Narrow Reading
Article Digest
Jinn-Kong Sheu
Modeling and Analysis Strategies for Failure Amplification Method
News Release
TNR Brings Salient Results at NCKU
Wastewater Emulsified Fuel Oil - Dream Technology for Tomorrow
Editorial Group
H-index: A New Trend in Evaluating Research Quality

Dr. Peter Jasco from the University of Hawaii analyzed the stengths and weaknesses of various citation databases
In order to enhance both the quality and quantity of research output at NCKU, the Office of Research and Development and Thomson Reuters invited Dr. Peter Jasco, an established expert in library and information science at University of Hawaii, to analyze and review various citation databases and the h index, a new trend in evaluating research impact on July 30th, 2008. The seminars were co-organized by Chin Shan Information Service Co., Ltd.

Part I. The Dimensions of Cited Reference Enhanced Databases

Being described as “passionate and adamant”, Dr. Jasco emphasized the importance of understanding the breadth and depth of cited reference enhanced subset of databases. Using empirical evidence to reveal the performance of many citation databases, he concluded that “citations are mismanaged in many databases”, and “few academic online services and publishers treat cited references well, if at all”.

Speaking of the usefulness of citation databases, a common sense  concern would be how big it is. However, to evaluate a database, Dr. Jasco pointed out that the absolute database size was not everything, and the biggest was not always the best. Also, when comparing two databases, one should consider how database A is bigger than database B, in what shape and form, i.e., it is bigger horizontally (wider) or vertically (taller), etc. Understanding the qualities and features of different databases, researchers can choose the appropriate database to use according to his/her research needs.

According to Dr. Jasco, the Google Scholar, with a dominating logic of “more records for shorter time span”, is not a savvy choice.

A message, published in 2008 and already cited 435 times, concludes the points Dr. Jasco emphasizes about the professional and academic online information services, “every academic and professional online service should have a software that would at least try to a) understand mispronounced or misspelled words, b) make sense out of simplistic or garbled queries, c) guide the users through choosing the right databases, right search words, right synonyms, best qualifiers and filters for refining the search results, d) provide clues through adding novel and/or mashed-up facts, factoids, tidbits and snippets and e) facilitate the refinement of the query in an intuitive way or cherry-pick the ones from the final results most pertinent to the users.”
Dr. Jasco perceived the intrinsic human desire behind many ranking pursuits pertaining to various spheres of human endeavors (Ranking is universal obsession: Guiness Book of World Records mentality, the desire for the biggest, largest, greatest, tallest, and the fastest. For academia, the most productive, most cited journal, author, and institution are the norm.)

Part II. Cited-based Measures of Scholarly Productivity and Impact

Another important issue explored by Dr. Jasco was the reasonable way to evaluate academic research performance. He introduced the h index, a new trend in evaluating research impact proposed by Jorge E. Hirsch, professor of physics at the University of California at San Diego, in 2005.

In order to introduce the h index with authentic material, we now use direct quotes from Jorge E. Hirsch’s paper “An index to quantify an individual’s scientific research output”. 

The Index h
Jorge E. Hirsch, advocator of the h index as a simple and valid way to evaluate research impact

In September, 2005, Jorge E. Hirsch, Professor of Physics, University of California at San Diego, proposed the index h, defined as number of papers with citation number ≧h, as a useful index to characterize the scientific output of a researcher.

Reasons to invent the index h
In this historical paper, Hirsch wrote that “For the few scientists who earn a Nobel prize, the impact and relevance of their research is unquestionable. Among the rest of us, how does one quantify the cumulative impact and relevance of an individual’s scientific research output? In a world of limited resources, such quantification (even if potentially distasteful) is often needed for evaluation and comparison purposes (e.g., for university faculty recruitment and advancement, award of grants, etc.).”

Application of the index h
Hirsch suggested that this index should be useful not only for physics, but also for other scientific disciplines as well.

Interpretation of the value of index h
Based on empirical calculation, Hirsch proposed that “I suggest (with large error bars) that for faculty at major research universities, h ≒ 12 might be a typical value for advancement to tenure (associate professor) and that h ≒ 18 might be a typical value for advancement to full professor. Fellowship in the American Physical Society might occur typically for h ≒ 15–20. Membership in the National Academy of Sciences of the United States of America may typically be associated with h ≒ 45 and higher, except in exceptional circumstances.”

However, the h index should be interpreted with cautions due to the complexity of real life circumstances.

Illustrating the h index with familiar examples

Dr. Jasco then used many familiar examples to count their h value to make this concept understood by the audience. He concluded that understanding the background of the h-index, we could understand the limitations of the h-index. Though h-index was extremely popular, extremely embraced, it was not perfect. There had already been some variants. To some extent, they could overcome the shortcomings of the h-index. But nothing could overcome the deficiencies caused by the shortcomings of databases.

The sessions ended with a group of more informed audience realizing the trend of evaluation of their career.
Dr. Jasco did not hasitate to communicate his judgement of citation databases by showing self-evidant examples.
Dr. Jasco used examples to illustrate the h index.

Summarized by Helen Chang
The Banyan Editorial Office

Copyright National Cheng Kung University