Uncovering Trends in Public Policy-Making
Given the proliferation of data in these information-rich times and the technological ability to analyze that data, one would think reliable answers to such questions would be easy to find. Not so.
According to NSF Principal Investigator Bryan D. Jones, Director of the Policy Agendas Project at the University of Washington, the datasets traditionally available to the public for study of policy-making processes suffer from "massive reliability problems."
"Systematic policy comparisons have eluded students of public policy change because a valid, reliable set of categories of policy activity did not exist," Jones explains. "Researchers have made a lot of mistakes using inconsistent categories."
The problem arises when the creators and custodians of datasets, and the researchers who use them, don't respect the difference between "information retrieval systems" and systems whereby trends can be analyzed over time. Meaningful longitudinal studies require diligence in keeping dataset categories consistently updated as new information is added over time.
"Our datasets are what is called backward compatible," adds Jones. "If we add a new category we have to reclassify everything. And we do."
The Policy Agendas Project was born in 1994 when Jones and Co-PI Frank R. Baumgartner, of Pennsylvania State University decided to trace congressional attention to all policy issues over the entire post-World War II period. Using an unprecedented and broad approach, their Project set out to capture data on all congressional hearings, all laws, all stories in the Congressional Quarterly Almanac, a sample of stories in the New York Times Index, and the entire federal budget. Recently added are all presidential executive orders and selected public opinion data.
Supported by a series of grants from NSF's Political Science Program and funding from the University of Washington and Penn State, a 10-person team works continuously on the datasets at the Center for American Politics and Public Policy directed by Jones. The team is comprised of graduate students, work/study participants, and Jones and his partners, Baumgartner and Wilkerson. Over the next few years, they will be releasing additional datasets that include all congressional bills, all State of the Union addresses since 1946, and interest group activity, congressional roll-call votes, and Supreme Court decisions.
The Policy Agendas Project website currently has between 200 and 400 unique visitors per week. Students and researchers find the recently redesigned site very easy to use. Graphing and data tools include tutorials and example graphs. Extensive information on coding schemes, additional data-related resources and teaching resources are provided. And visitors are able to immediately access full texts of source policy documents from linked websites.
Scholars from smaller universities are thrilled that the site makes the datasets publicly available at no charge, since commercially available database services can be expensive. A key goal of the project is to broaden the base of the social science infrastructure, and the website provides a truly interactive application system for delivery of their comprehensive databases to public policy scholars, students and policy analysts.
And, indeed, scholars from around the nation have begun to use data from the Policy Agendas Project to investigate institutions and policymaking. Numerous publications and papers using Project datasets, as well as Policy Dynamics (written by Baumgartner, Jones, and Wilkerson) are available for review on the website.
In one study using datasets from the project, Jones and his colleagues found that there are only three times in the post-WW II period where major changes in U.S. government spending took place. These budget jumps, or "macro punctuations," occurred in:
- 1956 under Eisenhower, when spending increased to enable major buildup of infrastructure, science, technology, housing, etc.
- 1975 under Nixon, when a major downward shift in spending occurred in response to new budget rules instituted in Congress that balanced spending with taxes.
- 2001 under Bush, when an across-the-board budget punctuation occurred, probably related to the removal of the Pay/Go rules.
The bottom line, according to Jones, is that the Policy Agendas budget datasets track spending reliably from 1945, and the studies utilizing that data confirm that institutional rules such as Pay/Go do work.
As an important analysis tool, the Policy Agendas Project not only improves the ability of social scientists to study policy dynamics, it enables researchers to link those dynamics to government processes that form the core of democracy: representation, public decision-making, political communication, and the operation of democratic institutions.