Social Stratification and Library Use
In two papers Bensman (1982; 1985b) analyzed the social bases of bibliometric laws and library use particularly as they related to scholarly journals. During this analysis he demonstrated that the concentration of use on a relatively small proportion of a library's serials holdings was a function of a series of sociobibliometric laws based on the principle of cumulative advantage or, more specifically, the double-edged Matthew Effect. Bensman argued that these laws were operative not only in library use but also in the social stratification of scholarship, and he developed the hypothesis that the concentration of journal use in academic libraries was partially a reflection of the process of formation of scholarly elites. Bensman found peer ratings and citations to be virtually equivalent measures of scientific value, and he theorized that a logical result of the double-edged Matthew Effect should be distributions stable over time with large zero or random classes. In his opinion, citations represented a measure of the formation of scholarly elites, the highly stratified and relatively stable social system of scholarship, as well as of those journals that research scholars regard as important.
A unique opportunity to investigate this hypothesis further occurred when, through the mediation of the university's Dean of Graduate School and subsequent Provost, LSU Libraries became a test site for the database developed by the National Research Council (NRC) during its 1993 survey of U.S. research- doctorate programs (Goldberger, Maher, and Flattau 1995). Because a major study of the structure of the library market for scientific journals in chemistry had just been completed, with data collected in the SRP pilot project at the LSU Department of Chemistry, it was decided to utilize the NRC database to investigate the interrelationship of the scientific social stratification system with the scientific journal system in this field. The NRC database is a massive one, containing not only the 1993 peer ratings of academic departments but also data developed by the Institute for Scientific Information (ISI) on the publication and citation rates of departments in the sciences, engineering, and the social sciences. An extremely valuable feature of the NRC database is that it contains statistical measures developed not only for the 1993 survey of U.S. research-doctorate programs but also for the three major such surveys immediately preceding it, making it possible to investigate stratification patterns over time.
History of Research-Doctorate Program Ratings in the United States
The traditional method of evaluating university graduate programs is peer rating. This method was pioneered in the early 20th century by the noted psychologist Cattell (1906; 1910), who statistically constructed a list of the 1,000 most-eminent American scientists through a survey of the leading representatives of 12 scientific disciplines. He first ranked universities and then academic departments by the number of these eminent scientists at them. This shift of focus from university to academic department or discipline became a standard feature in all later evaluations of graduate education.
Cattell's work was further developed by Hughes (1925), who while president of Miami University in Ohio conducted a study of U.S. graduate schools in 1924 as an aid in hiring new faculty and in advising students where to obtain advanced degrees. Hughes had Miami University faculty members in 20 disciplines draw up a list of major doctorate institutions, select 40 to 60 professors in each field throughout the U.S. to serve as raters, and on the basis of the responses construct a statistical ranking of the institutions offering the doctorate in the 20 disciplines. The 1924 rating represented an evolution from Cattell's work in two major respects: (1) it directly evaluated universities in each discipline instead of indirectly rating them by first ranking the persons working in the various fields and then locating the most eminent at specific institutions; and (2) it extended the process of academic evaluation from primarily the sciences to also the humanities and social sciences.
Hughes (1934) led a second study of U.S. graduate schools, this time for the Committee on Graduate Education of the American Council on Education. In this study graduate programs were not ranked but merely classified as "adequate" or "distinguished." The next peer rating was conducted in 1957 by Keniston (1959, 115–50) as part of a comprehensive analysis of the University of Pennsylvania Graduate School. However, because of its special purpose, the study was limited to only 25 universities deemed similar to Pennsylvania.
The evaluation of U.S. graduate programs conducted by Cartter (1966) under the auspices of American Council on Education in 1964 represented a milestone in that its methodology for obtaining and presenting peer ratings of the quality of university faculty was adopted by all future such studies. In the questionnaire, raters were asked to judge "the quality of the graduate faculty" (underlining in original), taking into consideration only their "scholarly competence and achievements," and to assign grades from 1 to 6 to the programs. In addition, the raters were given the option of not evaluating the programs by marking their questionnaire "Insufficient information" in the appropriate box. The grades were then assigned the following numerical weights: Distinguished–5; Strong–4; Good–3; Adequate–2; Marginal–1; Not sufficient to provide acceptable doctoral training–0. These numerical weights were averaged to obtain a score for each program.
This methodology for obtaining and presenting peer ratings of the scholarly quality of university faculty was essentially replicated by the second American Council on Education evaluation of graduate education done in 1969 by Roose and Andersen (1970); the 1981 Assessment of U.S. research-doctorate programs done under the auspices of the American Council of Learned Societies, American Council on Education, National Research Council, and Social Science Research Council (Jones, Lindzey, and Coggeshall 1982); and the 1993 evaluation of U.S. research-doctorate programs sponsored by the National Research Council (NRC) (Goldberger, Maher, and Flattau 1995). The latter two studies represented an advance over the preceding ones in that not only were the peer ratings of the research-doctorate programs given in them but also other measures of these programs, including publication and citation measures derived from ISI data for disciplines in the sciences, engineering, and the social sciences.
A notable feature of these peer ratings is the remarkable stability they exhibit at the top over time both at the institutional and the program level. In their book on elite U.S. research universities and their challengers Graham and Diamond (1997) identify 16 preeminent institutions that dominated American research prior to World War II: California at Berkeley, Cal Tech, Chicago, Columbia, Cornell, Harvard, Illinois at Urbana–Champaign (UIUC), Johns Hopkins, Michigan, Minnesota, MIT, Pennsylvania, Princeton, Stanford, Wisconsin at Madison, and Yale. The work of Graham and Diamond is not based on peer ratings, and, even though their central argument is that the traditional elite was successfully challenged after 1945 by rising new research universities, their statistical measures of research performance—institutionally based and controlled for institutional category and size—manifest the continued dominance of the old elite. Graham and Diamond point to the Matthew Effect as a causal process in this.
The traditional elite institutions identified by Graham and Diamond have always scored high in peer ratings no matter how these ratings are constructed or aggregated. For example, 14 of these 16 institutions appeared among the top 15 universities in Cattell's (1906, 739) first such ranking based on peer ratings of individual scientists. Moreover, when Webster and Skinner (1996, 26–27) ranked 104 universities with 15 or more programs evaluated by the 1993 NRC study according to the mean peer rating of the scholarly quality of all their programs, the traditional 16 were all among the top 23 institutions. As a further test of the relative stability of the academic stratification system, the NRC database was used to rank universities by aggregating the 1993 scholarly quality peer ratings of all their evaluated programs into one total score, and 14 of the traditional elite universities came in the top 19 out of 274 institutions. Only Cal Tech and MIT were not in the top 19 due to their narrower subject focus and the resulting smaller number of rated programs.
The stability of the overall institutional ratings is a function of the underlying stability of the program ratings. This question was analyzed in the recent NRC evaluation of U.S. research-doctorate programs (Goldberger, Maher, and Flattau 1995, 42–43). In this study the relative ranking of programs rated in both 1981 and 1993 were compared. These programs were distributed over quality quarters ranked in descending order by the mean peer rating of the scholarly quality of the faculty. It was found that those programs in the top quarter in 1981 tended to remain there in 1993 (399 of 468) and those in the bottom quarter in 1981 also tended to stay there in 1993 (363 of 487). The top was most stable in the social sciences (89% of the 1981 top quarter programs remained there in 1993) and lowest in the arts and humanities (80% of the 1981 top quarter programs stayed there in 1993). Programs rated for the first time in 1993 generally fell into the bottom half of the quality groupings.