Description of the Serials Evaluator
The Serials Evaluator is a computer program designed to put the scholarly, scientific, or technical value of journals in relationship to their price, thus taking advantage of the skewed distributions in these areas and maximizing the amount of value purchased. The program does this mainly by set definitions and comparison of sets. The Evaluator first defines serials by subject; this step is considered so basic that the Evaluator was intentionally structured so as to make it impossible to proceed further until this task has been performed. Not only is set definition by subject important in the statistical sense, it is also politically necessary to protect the smaller disciplines from drowning in the global mass of numbers as a result of fewer persons generating data for the measures of value. Within these subject sets the Evaluator then ranks serials in descending order by both price and value. Value can be measured in three ways—expert ratings, total ISI citations, and library use—and each of these methods has its strengths and weaknesses, which will be discussed below. With the subject sets defined and the serials measured in terms of cost and value, the Evaluator can then be used to define subsets that contain the maximum amounts of cost and value. The user is given the capability to set the amounts of cost and value within the subsets. Having performed these functions, the Evaluator can then merge the cost and value subsets and print out a list of serials high in cost but low in value for either cancellation or nonpurchase. The Evaluator was used to process data gathered by the SRP, and the procedures and results are discussed in the following sections.
Set Definition, Academic Units, and the LC Classification Schedules
The 392 LSU faculty members responding to the SRP named 3,231 serial titles as useful to their research and teaching. Although these faculty members were concentrated in science and technology, their selections ranged over all 21 main subject groups of the LC class schedules from A (General Works) to Z (Bibliography; Library Science). However, of these 21 subject groups, 6 (28.6%) contained 2,906 (89.9%) of the titles selected by the faculty. These subject groups were as follows: Q (Science)–1,498 titles (46.4%); T (Technology)–469 (14.5%); H (Social Sciences)–316 (9.8%); S (Agriculture)–295 (9.1%); G (Geography; Anthropology; Recreation)–176 (5.5%); and R (Medicine)–152 (4.7%).
Three basic factors account for the broad subject spread of the serials named by the LSU faculty in the SRP. First, there is the nature of the LC class schedules. The LC classification system tends to split subjects according to their different facets. For example, fishery materials are located in HD9450–9472 (Fishery Product Industry; Fish Trade Aquaculture Industry; Seafood Industry; Shellfish Industry), QL401–432 (Mollusca), QL435–448 (Crustacea), QL614–638.8 (Fishes), or SH1–400.8 (Aquaculture; Fisheries) in accordance with their economic, scientific, or technical aspects.
Second, there is the character of the academic units surveyed in the initial phase of the SRP and the relationship of these units to the LC classification system. Although organizationally located in the ST part of the university, some of these units (such as the Department of Agricultural Economics & Agribusiness and the School of Vocational Education) are actually engaged in the social sciences, and their materials are largely classed in HD (Economic History and Conditions) in the former case and L (Education) in the latter case. Moreover, there is the problem of what can be termed "segmented" units (units that encompass different disciplines). A quintessential example of such a unit is the Department of Geography & Anthropology. Not only does this unit encompass two academic disciplines—anthropology and geography—but geography itself is segmented into the mapping sciences (cartography and remote sensing), human geography, and physical geography (climatology, geomorphology, and coastal, as well as biogeography and environment) with interests ranging from the social sciences to the sciences. Another example of a segmented unit is the School of Human Ecology, which gives courses in the following areas ranging from the social sciences to technology: apparel, textiles, and merchandising; family, child, and consumer sciences; as well as human nutrition and food. The third factor that accounted for the broad subject spread of the serials selected was the operation of Bradford's law of scattering, which states that articles on a given subject will be found in journals dedicated to other subjects.
There are two basic ways to handle faculty survey data on serials such as those generated by the SRP. One can deal with the data academic unit by academic unit. In this case, one encounters Bradford's law of scattering and finds a core of journals on which everyone is agreed together with a long tail of other journals on which there is little agreement. It is almost impossible to evaluate the importance of the journals in the long tail. The other way is to take advantage of Garfield's law of concentration. To do this, one defines the academic units or their segments as "disciplines" in the sociological, Kuhnian sense of groups of scholars or scientists studying given "subjects" defined by some classification scheme, such as the LC schedules. In this case, one should find cores of journals on which the various disciplines agree, letting the various disciplinary tails interlock with each other over these cores. Such a method not only handles the disciplinary tails but improves the evaluation of the journals in the various subject cores by measuring their importance to other disciplines. Due to these advantages, it was decided to opt for the latter system, using the LC schedules to define the subject cores.
The LC classification system suffers from all the faults listed above for any library classification system. In addition, it is a system that was not intended to be universalistic but was designed starting around the turn of the twentieth century specifically for the collections at the Library of Congress, and it is somewhat incoherent due to the separate development of the various subject schedules by different groups of persons working more or less independently from each other (Chan 1990, 5–22). Nevertheless, LC class numbers are among the most readily available subject codes for thousands of serials, and it was deemed important to test the applicability of the LC schedules to define sets in the statistical sense.
The first step in the process of defining subject sets was to narrow the focus of the analysis to the three major LC classes—Q (Science), S (Agriculture), and T (Technology)—corresponding to the three main academic units surveyed in the SRP: College of Basic Sciences, College of Agriculture, and College of Engineering. These three LC classes accounted not only for the bulk of the titles selected by the faculty—2,262 of 3,231 (70.0%)— but, more importantly, the vast bulk of the cost of these serials—$1,356,863 of $1,569,658 (86.4%). Serials in the Q class alone accounted for $1,055,486 (67.2%) of the costs. The other three of the six major LC classes containing the bulk of the titles named by the faculty—G (Geography; Anthropology; Recreation), H (Social Sciences), and R (Medicine)—were not analyzed, because it was thought that the serials in these classes had to await the results of the forthcoming survey of LSU's social science units to be properly evaluated. With the analysis thus focused, the serials were identified that were on current subscription at LSU Libraries in LC classes Q, S, and T but not named by the faculty surveyed during the SRP. This zero class consisted of 207 titles costing $70,917.
The method used to define subject sets for the Evaluator was the one pioneered by McGrath and Durand (1969) at the University of Southwestern Louisiana, i.e., classifying the academic units surveyed by the initial phase of the SRP with the LC schedules Q, S, and T. For this purpose, the Louisiana State University General Catalog for 1995–96 was used, and LC class groups were assigned to the various academic units based on the department purposes and course descriptions given in this catalog. As a result of this process, 33 "curriculum cores" were defined and given computer codes for manipulation by the Evaluator. Serials were assigned to these curriculum cores on the basis of their LC call numbers.
A detailed description of the process of assigning LC class groups to the various academic units will not be given, for the problems encountered here were basically the same as those faced by any catalog librarian assigning class numbers to library materials. We will limit ourselves merely to outlining the more general problems and the solutions we adopted for them.
First, there was the problem of segmented units such as the School of Forestry, Wildlife, & Fisheries, the School of Human Ecology, the Department of Physics & Astronomy, the Department of Geography & Anthropology, and the Department of Electrical and Computer Engineering. The solution was to treat the segments as disciplines and not the units as a whole. For example, forestry, wildlife, and fisheries were treated separately; the three curricula of the School of Human Ecology concerning apparel, family, and human nutrition were handled individually; climatology was broken out of the Department of Geography & Anthropology; physics and astronomy were isolated from each other; and a differentiation was made between computer engineering and electrical engineering.
Second, there was the problem of the low response rate to the SRP survey on the part of faculty in certain units such as the Departments of Horticulture and Computer Science. Horticulture's LC class groups were merged into the LC class groups that covered agronomy, and the LC class groups on computer engineering and computer science were combined—a procedure that appeared justified by the heavy overlap in the serials selections by the Departments of Computer Science and Electrical & Computer Engineering.
Third, faculty members selected materials that were classed in those parts of the LC schedules for subjects not taught at LSU. This problem mainly concerned 12 titles in class group TN (Mining Engineering; Metallurgy), and, due to their high average cost, it was decided to subject these 12 titles to analysis by allocating them to other curriculum cores on the basis of the departmental origins of the majority of the faculty selecting them.
Fourth, there was the problem of those parts of schedules Q, S, and T, that related to units not surveyed during the SRP, such as those on agricultural economics, vocational education, and landscape architecture. It was decided to exclude these parts from analysis pending further survey data from the social science units. This rule, though, was violated for two such class groups—QA273-280 (Probabilities; Mathematical Statistics) and T57-57.97 (Applied Mathematics; Quantitative Methods)—which were affected by the lack of survey data from the Department of Quantitative Business Analysis in the College of Business Administration. Due to their importance, it was decided to include these groups in the analysis to obtain a preliminary evaluation of the serials contained in them.
Fifth, there was the problem of small LC class groups forming curriculum cores such as those for nuclear science, dairy science, and poultry science. There were not enough serials in these cores for statistical analysis, and the problem was handled by collapsing these small curriculum cores into larger ones with nuclear science being merged into physics and dairy science and poultry science going into animal science.
Sixth, there was the problem that the LC classification system makes a clear distinction between science and applied technology with science materials being classed in Q and applied technology being classed in S and T. As the example of fishery materials above illustrates, this distinction causes materials on the same subject to be classed in different areas, but, because this distinction conformed to the distinction between the academic units surveyed, it was decided to maintain it by allocating the Q classes to the College of Basic Science curriculum cores, the S materials to the College of Agriculture curriculum cores, and the T materials to the College of Engineering curriculum cores. This rule was broken only with respect to entomology, where the Q and S groups were combined into one curriculum core.
Finally, certain sections of LC schedules Q, S, and T pertain to many different academic units and cannot be allocated to any particular one. This happens not only at the beginning of these schedules, which are dedicated to the general aspects of their respective subjects, but also in biology. In the latter case, the main LC class group QH covers the general aspects of biology, before the classification system splits along the lines of botany (QK) and zoology (QL, QM, and QP), whereas the LSU organizational structure maintained a strict division along the botany and zoology lines. As a result of this, it was deemed necessary to create general curriculum cores not specifically related to any LSU academic unit not only for science, agriculture, and technology, but also for biology.
The final product of the above effort was the creation from LC schedules Q, S, and T of 33 curriculum cores related to the LSU academic units (see table 2).
Throughout the process of assigning serials to the various curriculum cores, cases of apparently misclassified titles were observed, but, except for a few egregious examples, the temptation to reclassify them was resisted, because part of the purpose of the project was to verify whether the LC classification system could be utilized to define statistical sets.
Three curriculum cores—Mechanical Engineering, Chemistry, and Agronomy—were selected to test the efficacy of the above method in capturing the subject interests of the faculty in the various academic units surveyed in the SRP. These cores were deliberately selected as representative of the three major academic units covered by the survey—the Colleges of Basic Sciences, Agriculture, and Engineering. The cores were created by merging LC class groups in the following manner: Chemistry—QD1–412.5 and QD450–999; Agronomy—S589.7–599.9, S602.5–604.37, S631–667, S950–954, and SB1–467.8; and Mechanical Engineering—TJ1–1475, TJ1501–1570, and TL1–4050. In general, the tests can be regarded as a success, and the results not only demonstrated the truth of Bradford's law of scattering but replicated the findings of Narin, Pinski, and Gee (1976) with biomedical citations as well as of Metz (1983, 32–41, 85–87) in VPI library use that while technology is dependent upon the literature of science, the relationship is not reciprocated.
The results of the tests are shown in table 3. Of the 191 serials named by the Department of Chemistry, 99 or 51.8% were classed in the Chemistry core, with the next highest number being the 22 or 11.5% located in the Physics core. Only 5 (2.6%) of the Department of Chemistry's selections were classed in the Chemical Engineering core. In contrast, of the 183 serials named by the Department of Agronomy, 38 or 20.8% were classed in the Agronomy core and 33 or 18.0% were in the Plant Biology core. All told, 49 or 26.8% of the Department of Agronomy's selections were in the College of Agriculture cores, but 58 or 31.7% were in the College of Basic Sciences cores. A similar phenomenon was observed in the selections of the Department of Mechanical Engineering. Of the 108 serials named by this department, 32 or 29.6% were classed in the Mechanical Engineering core with the next highest number being the 17 titles or 15.7% found in the Physics core. However, unlike the Department of Agronomy, the Department of Mechanical Engineering concentrated its selections more in the cores of its own college with 48 or 44.4% of its choices in these cores as against 23 or 21.3% in those of the College of Basic Sciences.