The main objective of this analysis is to illustrate trends over the past two decades in the aggregate number of academic scientific publications in relation to funding for academic R&D in S&E and the number of U.S.-trained, doctorate-level academic researchers in science, engineering, or health. Additional objectives are to determine whether the aggregate findings are replicated at various types of universities and colleges and across fields of science and engineering. It is hoped that this analysis will spur further research.
After presenting aggregate trends, this paper compares trends for the following types of universities and colleges: (1) publicly versus privately controlled institutions and (2) Research I institutions versus less research-extensive universities and colleges. Researchers from publicly controlled universities authored about two-thirds of total academic sector articles. [5] Similarly, researchers from Research I institutions authored about two-thirds of total academic sector articles.
This paper also covers the fields in which academically employed doctorate holders earned their first research doctorate in science, engineering, or health. [6]
To account for the passage of time between funding, conducting, and publishing results of the research, this analysis compares R&D expenditures in a given year with counts of academic research personnel from the following year and with publication counts from two years after the year of R&D expenditures. [7] For example, publication counts in 2011 are compared with inflation-adjusted academic R&D expenditures in 2009 and estimates of doctoral researchers in 2010. Whenever a year is noted in the text, figures, or tables, it refers to the publication year. Further analysis could explore the use of different lag times, no lag times, or field-specific lag times.
Funding is measured as the annual S&E R&D expenditures of universities and colleges.[8] NSF surveys these institutions annually via its Higher Education Research and Development Survey (HERD) (called the Survey of R&D Expenditures at Universities and Colleges before 2010). For the years covered in this analysis, the survey was a census of academic institutions that granted a bachelor's degree or higher in S&E fields and reported at least $150,000 in separately budgeted S&E R&D expenditures in the previous fiscal year. The survey defined R&D as projects that are separately budgeted and fall under the Office of Management and Budget's (OMB's) A-21 definition of organized research. It further specified that R&D includes (1) R&D sponsored by federal and non-federal agencies and organizations and (2) R&D that is separately budgeted under an internal application of institutional funds.[9] About two-thirds of academic R&D is basic research. Applied research and development together constitute the remaining one-third.
In this analysis, academic R&D expenditures—whether for basic research, applied research, or development—are treated as if their sole purpose was to fund research that leads to publications. However, spending on academic R&D results in many other outcomes. It supports training for the next generation of science, engineering, and mathematics professionals in research practices and other advanced skills. It also funds the development of new processes or technologies that may result in patents, the curation of databases, and the purchase and maintenance of research equipment. Academic R&D funds are also spent on a wide range of administrative and regulatory activities associated with conducting research. It is beyond the scope of this paper to analyze the effects of changing priorities for the use of academic R&D funds. For example, the paper does not attempt to measure whether a growing emphasis on patenting or interdisciplinary research could affect academic R&D publication output.
Salaries, wages, and benefits are the largest component of academic R&D expenditures, constituting about 40% of these funds.[10] Other components include non-salary-related direct R&D costs (roughly 20% of total academic R&D expenditures), indirect costs[11] (around 25%), funds passed through to subrecipients (about 8%), and equipment and software costs (around 5%). Although research projects may span multiple years, the Survey of R&D Expenditures at Universities and Colleges defines current fund expenditures as operating funds actually spent by a school during its fiscal year (typically from 1 July of the preceding calendar year through 30 June of the current calendar year).
Academic R&D personnel are persons who received their first doctorate degrees in science, engineering, or health [12] from a U.S. institution; work in 4-year universities, medical schools, or university research institutes; and report that research is their primary or secondary work activity (i.e., the activity that occupies the most or second-most hours of their work time during a typical work week).[13] Estimates of this population come from NSF's biennial SDR.
There are several limitations associated with using the SDR to measure trends over time among academic research personnel. First, the survey excludes foreign-trained doctoral researchers, as well as researchers without a doctoral degree. Foreign-trained doctorate holders made up just under 20% of overall academic doctoral employees in 2010; they also figure very prominently in academic R&D. [14],[15] As a result, estimates in this analysis of the publications-to-researcher ratio are somewhat overstated, particularly in recent years. Furthermore, many students who have not yet received a doctorate conduct research that contributes to publications. In 2010, graduate research assistantships were the primary means of support for 27% of graduate students, a percentage that has remained fairly stable since the early 1990s.[16]
In defining academic R&D personnel as U.S.-trained, academically employed doctorate holders who report that research is their primary or secondary work activity, this analysis attempts to include persons who are most likely to be publishing articles. Further research could focus only on primary academic researchers—those who reported that research is their primary work activity. From 1993 to 2010, primary academic researchers represented a gradually growing share of academic researchers.[17] The SDR does not provide estimates of the number of hours researchers spend in their various work activities or how this number has changed over time; this could vary substantially over the years.
The fields component of this analysis assumes that academic researchers are conducting research in the area of their first doctorate in science, engineering, or health. Although this may be true for most academic researchers, it is not always the case. Additional analysis could explore field shifts from first doctoral degree for academic researchers.
Publication output, also referred to as "articles" or "article output," is measured by counts of S&E articles, notes, and reviews published in a set of scientific and technical journals tracked by Thomson Scientific in the Science Citation Index and Social Sciences Citation Index (http://www.thomsonreuters.com/business_units/scientific/).[18] Credit for articles is assigned to the institutions of the authors, not to the individual authors. Throughout the period of our analysis, roughly three-quarters of US publication output was produced by the academic sector.
Article output is allocated primarily by using "whole counts," which assign full credit for a publication to each category of institution that appears in an article's author list. For example, an article with one or more co-authors from both a public and a private university is counted once in the article count for public universities and once in the article count for private universities. Summing the counts for these two groups to arrive at an article count for all universities would result in the double-counting of articles collaborated between the groups. To improve upon whole counting, rather than summing articles this analysis counts each article only once in each aggregated group, regardless of collaboration. Because universities of different types often collaborate (e.g., Research I institutions collaborate with less research-extensive institutions and public universities collaborate with private universities), the whole counts of articles for universities as a group in these data are less than the sum of the whole counts for public universities and private universities. Similarly, whole counts of articles for universities as a group are also less than the sum of the whole counts for Research I and less research-extensive institutions.
Fractional counting, used to a lesser extent in this analysis, is an alternative method of measuring publication output.[19] In fractional counting, all publications receive a single credit. For collaborative publications, this credit is divided equally among the institutions credited in the list of the publication's authors. Because of the robust growth of academic R&D collaboration over the past 25 years, growth in article counts over time is more moderate when articles are counted fractionally than when they are counted using whole counts. The advantage to fractional counting is that groups can be aggregated by simple summing and no article will be counted more than once. For example, using fractional counting for an article that listed four institutional addresses—two different private U.S. universities, a French university, and a U.S. nonprofit institution—one-fourth of the publication would be attributed to each of the U.S. universities, one-fourth to the French university, and one-fourth to the U.S. nonprofit. With this method, the category of private universities would receive credit for one-half of an article.
As estimates, both whole and fractional counts are distortions. Whole counts give credit for the entire article to each participating institutional group under analysis as though that group had sole authorship, and fractional counts discount the extra effort that goes into combining contributions from different sources. Despite these differing distortions, both methods demonstrate the same basic pattern of an increasing expenditures-to-articles ratio over time and relatively flat trends over time in the ratio of articles to academic researchers. Thus, except where noted, all data presented is computed using whole counts.
Finally, conference proceedings, which are not well represented by Thomson Scientific in the Science Citation Index, play a significant role in certain fields, such as engineering and computer science. Thus, article counts for these fields may not reflect output as well as counts for other fields. The spending-to-publications ratio appears higher and the publications-to-researcher ratio appears lower than they would have been if conference proceedings were well represented in the publications database.
This analysis builds on previous NSF analyses of the publication output of the U.S. academic sector. Those previous analyses explored trends over a 15-year period from 1988 through 2003 in the total number of S&E publications whose authorship was credited to the U.S. academic sector.[20],[21],[22] Those analyses reported a leveling off of academic publication counts during the 1990s and early 2000s and examined possible contributing factors. Unlike those earlier studies, this paper is centrally concerned with trends in the relationship between resource inputs (R&D expenditures and academic researchers) and publication outputs. In addition, this paper covers a more recent period in which output again began to rise.
[5] This is unsurprising given that about two-thirds of the nation's top 100 institutions in S&E R&D expenditures are publicly controlled. See National Science Board (NSB). 2014. Science and Engineering Indicators 2014, appendix table 5-6. NSB 14-01. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/statistics/seind14/.
[6] This paper grouped publications and expenditures data as closely as possible to the fields of researchers' first doctorate in science, engineering, or health. Included in the overall totals but not separately analyzed are computer sciences, mathematics, and psychology.
[7] This method was selected because an earlier NSF analysis generally showed that publications in a given year were most closely linked to personnel counts from the previous year and to expenditures from two years before the publication year; an alternate way yielding very similar results would be to have a two-year lag between both expenditures and publications and personnel counts and publications. Whenever a year is noted in the text, figures, or tables, it refers to the publication year. See Javitz H, Grimes T, Hill D, Rapoport A, Bell R, Fecso R, and Lehming R. 2010. U.S. Academic Scientific Publishing. Working Paper SRS 11-201. Arlington, VA: National Science Foundation, Division of Science Resources Statistics. Available at http://www.nsf.gov/statistics/srs11201/.
[8] This analysis does not count spending by non-U.S. academic co-authoring institutions. If included, estimates of spending per publication would be somewhat higher, especially in recent years.
[9] See National Science Foundation, National Center for Science and Engineering Statistics. 2011. Academic Research and Development Expenditures: Fiscal Year 2009. Detailed Statistical Tables NSF 11-313. Arlington, VA. Available at http://www.nsf.gov/statistics/nsf11313/.
[10] Britt R. 2012. Overview of FY 2010 findings from NSF's Higher Education R&D Survey. Presentation at the 2012 American Institutes for Research (AIR) Annual Forum, Washington, DC, 5 June.
[11] Indirect costs are general expenses that cannot be associated with specific research projects but pay for things that are used collectively by many research projects at an academic institution. Two major components of indirect costs exist: (1) facilities-related costs, such as the construction, maintenance, and operation of facilities used for research and (2) administrative costs, including expenses associated with financial management, institutional review boards, and environment, health, and safety management.
[12] In this analysis, health doctorates are grouped with doctorates in biological, agricultural, and environmental life sciences to create the broad field of life sciences.
[13] In this analysis, research includes basic research, applied research, development, and design.
[14] In 2010, the U.S.-trained academic workforce with doctorates in science, engineering, or health numbered about 295,000, whereas their foreign-trained counterparts numbered about 64,000. About 90% of the nation's foreign-trained academic doctoral personnel classified research as their primary or secondary work activity as compared to about 70% of their U.S.-trained counterparts.
[15] National Science Board (NSB). 2014. Science and Engineering Indicators 2014, chapter 5. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/statistics/seind14/. See pages 5-24 and 5-61.
[16] National Science Board (NSB). 2014. Science and Engineering Indicators 2014, chapter 5. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/statistics/seind14/. See page 5-32.
[17] National Science Board (NSB). 2014. Science and Engineering Indicators 2014, chapter 5. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/statistics/seind14/. See page 5-30 and appendix table 5-19.
[18] The changing set of journals reflects the current mix of the world's influential journals. Article count excludes letters to the editor, news stories, editorials, and other material whose purpose was not the presentation or discussion of scientific data, theory, methods, apparatus, or experiments. Thomson Reuters selects journals each year as described at http://www.thomsonreuters.com/products_services/science/free/essays/journal_selection_process/. The journals selected are notable for their relatively high citation rank within their corresponding S&E subfields; journals of only regional interest are excluded. The set of journals covered in this working paper—covering the analyzed fields—grew at an average annual rate of 1%, from 3,587 in 1988 to 4,437 in 2011.
[19] Publication counts reported in the National Science Board's biennial Science and Engineering Indicators reports are typically counted on a fractional basis, allowing for an approximation of the share of the contribution of each country (or institutional sector) toward the whole paper because every individual institutional address on a publication is given an equal share. For example, a paper with three French and one U.S. institutional address is attributed three-fourths to France and one-fourth to the U.S. when counting fractionally, but it is attributed one count to each country when using whole counts. Fractional counting minimizes over-crediting to a country a paper where the majority of the coauthors are from another country (or institutional sector).
[20] Javitz H, Hill D, Rapoport A, Bell R, Fecso R, and Lehming R. 2010. U.S. Academic Scientific Publishing. Working Paper SRS 11-201. Arlington, VA: National Science Foundation, Division of Science Resources Statistics. Available at http://www.nsf.gov/statistics/srs11201/. This working paper analyzes the covariates of publication trends in the nation's top 200 R&D performing universities and includes an analysis of the effects of different assumptions about the lag times between research activity and publications.
[21] Hill D, Rapoport AI, Lehming RF, and Bell RK. 2007. Changing U.S. Output of Scientific Articles: 1988–2003. Working Paper NSF 07-320. Arlington, VA: National Science Foundation, Division of Science Resources Statistics. Available at http://www.nsf.gov/statistics/nsf07320/. This special report delineates national and international trends in publication and coauthorship. In a section on methodological issues it discusses the journal database and alternate methods for counting publications.
[22] Bell RK, with Hill D and Lehming RF. 2007. The Changing Research and Publication Environment in American Research Universities. Working Paper SRS 07-204. Arlington, VA: National Science Foundation, Division of Science Resources Statistics. Available at http://www.nsf.gov/statistics/srs07204/. This working paper records the perspectives of academic scientists and administrators on the changing research and publication environment in U.S. universities.