File: J:\pubs\1996\nsf9652\chapter5.pdf
Pages:1 to 6 of 6
Document Body Page Navigation Panel Document Outline

Document Outline


Document Body

Chapter 5
Indicators in the current volume .........................................................106
Indicators for elementary and secondary education .......................106
Educating Americans for the 21st century ...................................................................106
Indicator systems ...........................................................................................................107
Improving indicators .....................................................................................................108

Future directions....................................................................................109
Concluding remarks ................................................................................109
References ...............................................................................................110


1 0 6 I N D I C AT O R S O F S C I E N C E A N D M AT H E M AT I C S E D U C AT I O N 1 9 9 5
T his chapter reflects on the process of developing statistical indicators of science and mathematics education. These indicators, which are statistics
that shed light on important policy issues, inform policy

makers about the state of current science and mathematics
education and recommend areas for future investigation.
The chapter reviews the frameworks for choosing
the statistical indicators in this report. It also reflects on
the status of data sources for the chosen indicators and
suggests areas that require greater attention in future volumes
as well as additional research.
The selection of topics for this report reflects the major
issues that are important to the Directorate for Education
and Human Resources
(EHR) of the National Science
Foundation (NSF). EHR is actively involved with conducting
research and supporting projects that lead to
improvements in student achievement in science and
mathematics for all students in the United States. Thus,
the indicators in this volume reflect the concerns about
how the current efforts to establish standards for science
and mathematics education are understood and implemented.
They also reflect the changes that are occurring to
establish greater equity among male and female students
and students of all races and ethnic groups.

Indicators in the Current Volume
The indicators presented in this volume are a synthesis
of available statistics about science and mathematics education.
Authors selected them from existing national surveys.
The authors of this report attempted to select indicators
of evidence of change in the Nation’s mathematics
and science education system. They examined sources of
data on trends in student achievement, teacher knowledge
and practices, content of curriculum, high school
coursetaking, and changes in characteristics of postsecondary
students, graduates, and faculty.
The authors of this volume selected indicators of shifts
toward the standards of excellence and equity of the education
system during the past 2 decades. Moreover, they
selected indicators to monitor U.S. efforts to reform the
entire education system by setting high expectations for
all students’ performance and obtaining a greater alignment
among components of the education system. For
elementary and secondary education, the selection of indicators
monitors curriculum coverage, teacher practices,
and student achievement. This selection was influenced
by national standards, which were developed by profes-

sional education associations. For postsecondary education,
the selection of indicators monitors the extent of
access to science and engineering postsecondary education
by underrepresented minorities and females.
Many of these indicators were informed by three commissioned
reports about science and mathematics education.
These reports followed the Commission on
Excellence report of 1983 that brought renewed national
attention to the need to reform the elementary and secondary
school system. The following section reviews the
recommendations of three of those reports in light of the
topics addressed by this volume.

Indicators for Elementary and
Secondary Education

Three major reports were prepared during the 1980s to
define issues of concern to NSF—

u Educating Americans for the 21st Century: A Plan of
Action for Improving Mathematics, Science and
Technology Education for All American Elementary and
Secondary Students so that Their Achievement Is the Best
in the World by 1995,
u Indicator Systems for Monitoring Mathematics and Science

Education, and
u Improving Indicators of the Quality of Science and

Mathematics Education in Grades K– 12.

These reports suggested means for developing indicators
of science and mathematics education.

Educating Americans for the 21st Century
The Commission on Precollege Education in
Mathematics, Science and Technology prepared a plan to
improve science, mathematics, and technology education
and presented it to the National Science Board in
September 1984. This report said that objective measurement
of achievement and participation in mathematics
and science was necessary and should be performed. It
recommended that the National Assessment of
Educational Progress (NAEP), which began in 1968, be
modified to include assessment of states and the Nation
in order to monitor progress using the most up-to-date
testing techniques. The report writers’ assumption was
that the Nation’s best students ranked equally with those
of any other nation, but that the average American stu-

Postscript


P O S T S C R I P T 1 0 7
dent’s achievement was low compared with other
advanced countries of the world. Thus, the report
encouraged efforts to measure the progress of all students.
It recommended that certain skills be monitored, including
the ability to write for a purpose and apply high-level
problem-solving skills to analyze and draw conclusions.
Many of the policies that were implemented after the
report was released were consistent with these recommendations.
For example, in 1990, the NAEP began to
include state-by-state comparisons on a trial basis. These
comparisons became a regular part of the survey in 1992.
In addition, new forms of national testing were developed
to measure high-order thinking skills. Indicators’ development
also extended into monitoring the changes in
student achievement levels for students at high and low
levels of achievement.
No specific indicators have been developed for this volume
to measure writing skills or the ability to analyze
information. Current performance assessment scoring
methods recognize these goals, but in practice, national
data collection strategies were still under development at
the time of writing the current report. Future reports
should be able to include reliable indicators of trends in
student performance of thinking and writing skills consistent
with the recommendations by the 1984 NSF
Commission from new survey information that is currently
being collected by the NAEP. The future analyses should
maintain a focus on how such new indicators would measure
progress toward good practices, such as toward adopting
the standards of mathematics and science and achieving
greater equity of performance among students.

Indicator Systems
In 1987, the
RAND Corporation released Indicator
Systems for
Monitoring Mathematics and Science Education.
This report sought to identify for NSF a set of indicator
systems that would allow monitoring of precollege science
and mathematics education. The report focused on
science and mathematics indicators since other agencies,
such as the National Center for Education Statistics,
have major responsibility for collecting information on
all aspects of education. RAND’s report suggested that a
“patchwork” of existing indicators be constructed from
existing data sources (such as NAEP) and that developmental
research be undertaken to create better indicators
that could be used constructively by policy makers and
educators. Specific recommendations were that
u an indicator system be developed both to describe and

to relate essential elements of the mathematics and science
education system;
u key indicators be developed for both the national and

state levels;

u critical gaps in existing indicators and analytical methods
be identified;
u the amount and quality of data available on mathematics

and science education be expanded;
u studies be conducted to analyze the causes of observed

changes and suggest alternative policy implications;
u new measures on student achievement be developed to

measure knowledge such as the ability to think critically
and apply knowledge in solving problems;
u new measures be developed on teacher quality, depth of

coverage, and scientific accuracy of the curriculum; and
u procedures be developed to analyze and report indicators

that ensure that results are well reviewed and disseminated
appropriately.
The report suggested a general model for the specific
elements of an indicator system. This model was organized
around inputs, processes, and outputs of school systems.
The system identified student achievement, attitudes,
and aspirations as the main outcomes of schooling.
Process indicators were divided into curriculum, instruction,
teaching, and school quality. Inputs included fiscal
resources, student background, and teacher quality. For
each of these areas, the report recommended 99 specific
indicators that should be monitored, and it linked them
to existing data sources. Additionally, RAND recommended
that NSF develop state-level information and
create comparisons of different natures (across time, with
normative standards, with other countries, and with different
populations). The report recommended that NSF
not develop its own expensive surveys, but that it develop
new measures for existing data collection efforts and
develop new measures from them.
Since the report’s publication, many of the recommendations
have been implemented. For instance, NSF
adopted this biennial indicators report with a defined
review process. Also, cooperation between the
Department of Education and NSF has continued. This
has increased the amount of information available about
science and mathematics. In addition, the number of
state-level indicators has increased. However, these indicators
are not yet as well developed as the national indicators
because the sample sizes are too small to permit
comparison of change over time for each state. Therefore,
they cannot be reported regularly as evidence for change
within particular states. Finally, as the RAND report recommended,
researchers have performed some studies that
explore the causes of student achievement (DeAngelis &
Talbert, 1995, April; Miller, 1992, April; Schiller, 1995,
May 30; Schneider, Plank, & Wang, 1994, August; SuiChu
& Willms, 1995, April). These studies provide alternative
strategies for education policies. They are written
for the purpose of describing correlations between schooling
experiences and student achievement.


1 0 8 I N D I C AT O R S O F S C I E N C E A N D M AT H E M AT I C S E D U C AT I O N 1 9 9 5
Since 1987, the amount of survey data for elementary
curriculum, instructional techniques, and assessments
increased substantially.
Iris Weiss’ (1987) surveys of
teachers have increased the amount of information available
on what teachers know and how they present materials
in class. (See also Weiss, Matti, & Smith, 1994.)
Additionally, surveys of students and teachers in several
longitudinal studies have increased the information available
about the science and mathematics topics that students
are exposed to in school. The general model suggested
in RAND’s report encouraged investigative models
of the effects of changes in curriculum and teacher experience
to student performance.

Improving Indicators
Recommendations in Improving Indicators of the Quality
of Science and Mathematics Education in Grades K– 12
were
based on the premises that
u all students need to leave school with adequate knowledge

and reasoning skills to be able to renew their
knowledge of science and mathematics throughout
their lives; and
u student learning is determined by what teachers and

students do in schools.
The report, written by the Committee on Indicators of
Precollege Mathematics and Science for the National
Academy of Sciences, reviewed the needs of the education
system and recommended a series of topics that
required further development and monitoring. It recommended
seven key indicators and a set of supplementary
indicators to expand the issues.
The key indicators involved
u learning among students,

u literacy among adults,
u enrollment in science and mathematics courses,
u nature of classroom instruction,
u teachers’ knowledge,
u salaries of college graduates, and
u quality of curriculum content in state guidelines and
materials.
The supplementary indicators involved
u amount of time spent on science and mathematics

homework,
u college courses completed by teachers,

u teachers’ use of time outside the classroom for activities
related to teaching,
u materials used for instruction,

u Federal financial support, and
u resources committed by scientific bodies for school
improvement.
The report recommended that these 13 indicators
cover five areas: student learning, student behavior,

teaching quality, curriculum quality, and financial support.
It recommended that
u an accelerated program of research and development

be carried out to construct free-response materials and
techniques that measure skills not measured with multiple-
choice tests— these materials would help in the
development of indicators of learning in science and
mathematics;
u information be gathered on the number of minutes per

week that elementary students devote to science and
mathematics, as well as the number of semesters of science
and mathematics that secondary students take, to
develop indicators of student behavior;
u teachers be tested on the same content and skills that

their students are expected to master and that information
be gathered on teacher preparation, such as undergraduate
and graduate college coursework, in order to
develop indicators of teaching quality;
u exemplary frameworks of science and mathematics

content coverage be constructed for elementary and
secondary grades, with the highest priority given to
early elementary and middle schools, in order to develop
indicators of curriculum quality— the frameworks
would be used to match textbooks, state guidelines,
and materials, such as tests and exercises, to analyze
the content of the implemented curriculum for indicators
of content coverage; and
u a set of accounts be developed on the expenditures of

science and mathematics education from departments
and agencies of the Federal Government for indicators
of financial and leadership support.
This volume incorporated many of the recommendations
in Improving Indicators of the Quality of Science and
Mathematics Education in Grades K– 12,
including using
indicators on learning among students, coursetaking,
nature of instruction, amount of time spent on science and
mathematics, courses completed by teachers, teachers’ use
of time outside the classroom, materials used for instruction,
and Federal financial support. However, some recommendations
have not yet been developed into indicators.
The areas involve adult literacy, teachers’ knowledge, and
the number of resources committed by scientific bodies.
Some work is being done to remedy this situation. For
instance, measures of adult literacy currently are being
developed by NSF. However, attempts to develop measures
of teachers’ knowledge have been difficult because
of objections by the teaching community that teacher
assessment should not be a concern of policy makers.
Studies of resources committed to science and mathematics
education by government and nongovernment
sources have not yet been conducted. This is an important
area for future study. Other areas for future study are
discussed below.


P O S T S C R I P T 1 0 9
Future Directions
The publication of indicators of science and mathematics
education require reliable sources of data from elementary,
secondary, and postsecondary institutions.
Typically, special data collection efforts are required for
specific subject areas of science and mathematics because
these subjects represent small components of a large education
system. The reports discussed in this postscript have
suggested mechanisms for integrating the existing surveys
of students and teachers into a systematic collection of
data. Such efforts could enhance the amount of information
available for science and mathematics education.
This indicators report has shown the value of integrating
subject area topics into existing surveys, such as the
assessment of student learning, the measure of teacher
practices in the classroom, and the relationships between
secondary school coursetaking with field choice in higher
education. However, the development of indicators that
measure the issues raised by reform efforts will require
new efforts with new types of survey techniques. For
instance, monitoring of systemic reform efforts will
require indicators on
u alignment among parts of the education system;

u changes in governance;
u number of community, business, and school partnerships;
and
u integration of elementary and secondary school systems

with postsecondary education.
In addition, new indicators will be required to
u measure changes in student achievement, coursetaking,

and teaching practices for states;
u show the relationship between planned and implemented

changes to elementary and secondary science
and mathematics curriculum, including adoption of
technology, as reform efforts continue;
u measure coursetaking and course content within postsecondary

institutions;
u monitor the science and mathematics literacy of college

graduates; and
u monitor the transition of graduates into the workforce.

Indicators will need to be developed specifically for
postsecondary education, because the sources of data concerning
students and faculty in undergraduate institutions
are limited. The issues of the quality of teaching and
learning in colleges and universities have been infrequently
addressed in national reports that review the
condition of science and mathematics education. Few
data sources inquire about teaching practices or the content
covered by students. Also, the quality of teaching is
infrequently covered in national surveys of higher education
or faculty.
NSF has asked the Grants Board of the American

Educational Research Association (AERA) to map out a
strategy for developing indicators of undergraduate mathematics
education. The project is developing a conceptual
framework for indicators that will be useful in monitoring
the status of undergraduate mathematics education, especially
with respect to assessing effects of the various reform
initiatives of NSF. The project targets lower division programs
for the entire population of students, not just those
majoring in mathematics. Concern is for the broad spectrum
of public and private institutions including community
colleges, liberal arts colleges, comprehensive universities,
and research universities. A national panel of experts
in undergraduate mathematics education and assessment
is expected to release a report in early 1996.
Undergraduate indicators are proposed for
u curriculum and instruction— the content and pedagogy

of educational programs;
u student outcomes and assessment— what students

know about mathematics and how that knowledge is
assessed;
u student participation— the characteristics of students

served by mathematics programs; and
u educational institutions and systems— the context

within which the teaching and learning of mathematics
takes place.
The recommendations of the AERA committee will
form a useful basis for restructuring the current data collection

educational systems of the United States. Yet, as efforts


DeAngelis, K., & Talbert, J.E. (1995, April). Social inequalities in high school mathematics
achievement: Cognitive dimensions and learning opportunities.
Paper presented to the
annual meeting of the
American Educational Research Association, San Francisco, CA.

Miller, J.D. (1992, April). Persistence and success in mathematics: What we are learning
in the longitudinal study of American youth. Paper presented to the annual meeting of the
American Educational Research Association, San Francisco, CA.

Schiller, K.S. (1995, May 30). So, you want to go to college? The SAT as an
incentive system for mathematics achievement in high school. In Improving mathematics
and science learning: A school and classroom approach.
Second year progress report, NSF
Grant RED-9255880.

Schneider, B., Plank, S., & Wang H. (1994, August). Output-driven systems:
Reconsidering roles and incentives in schools. Paper presented to the annual meeting of the
American Sociological Association, Los Angeles, CA.

Sui-Chu, E.H., & Willms, J.D. (1995, April). The effects of parental involvement on
eighth grade achievement.
Paper presented to the annual meeting of the American
Educational Research Association, San Francisco, CA.

Weiss, I.R. (1987). Report of the 1985-86 national survey of science and mathematics
education.
Research Triangle Park, NC: Research Triangle Institute.

Weiss, I.R., Matti, M.C., & Smith, P.S. (1994). Report of the 1993 national survey
of science and mathematics education.
Chapel Hill, NC: Horizon Research, Inc.

Chapter 5 References


Page Navigation Panel

  1   2   3   4   5   6