Title  : NSF 93-75, Knowledge Dissemination and use in Science and mathematics
         education: A Literature Review
Type   : Report
NSF Org: EHR / RED
Date   : May 1, 1993
File   : nsf9375




KNOWLEDGE DISSEMINATION AND USE

IN SCIENCE AND MATHEMATICS EDUCATION:

A LITERATURE REVIEW






Janet Hutchinson

Michael Huberman












Prepared for the Directorate of Education and Human Resources, Divison of 
Research, Evaluation and Dissemination, National Science Foundation by
The Network under Order Number CB2649X-00-0

May, 1993

Knowledge Dissemination and Use
in Science and Mathematics Education:
A Literature Review

Janet Hutchinson, University of Pittsburgh,
and M. Huberman, The NETWORK, Inc.


Introduction

     Dissemination of educational research and effective practice is hardly a 
new endeavor in the United States.  Its level of activity, however, along with 
its sophistication, have burgeoned in the past two decades.  This report 
reviews the research on knowledge use in science and mathematics education and 
highlights approaches and strategies for dissemination which might be 
particularly useful to the National Science Foundation.  In particular, the 
influence of constructivist perspectives in knowledge use and education on the 
dissemination of knowledge are highlighted.

     It is, in effect, almost anachronistic to recall the earlier efforts to 
move scientific knowledge and its by-products into the schools.  At that 
juncture, the regnant model comprised a progression from the "laboratory" to 
the educational "market place," in line with the models used to commercialize 
physical technology.

     From here, educational developers followed the scenario of R. Havelock 
(1969) and other members of the Institute for Social Research at Michigan (see 
Figure 1).  Havelock postulated a RDDE cycle, consisting of Research, 
Development of prototypes, Diffusion of the amended prototypes and Evaluation 
of the product.  The model was extremely influential -- it even figured in the 
creation of the U.S. Department of Education's Laboratories and Centers -- and 
it constituted a new starting point for a closer, more expeditious movement of 
research-based products to the universe of school practice (cf. Guba, 1968).

     Critiques of this approach, based on studies done during the 1970s and 
published at the end of that decade, found that the RDDE model had fundamental 
problems (Berman and McLaughlin, 1979).  For example, it cast the flow of 
knowledge as a one-way process, and did not take into account the motivations, 
contexts, and realities of the intended recipients. At the same time, 
educational researchers began to postulate factors that were important for 
successful dissemination.  These factors reflect a shift in the knowledge use 
literature from a rational or "imperativist" perspective to a more 
conflict-theoretic and constructivist perspective (Dunn and Holzner, 1988; 
Huberman, 1990).  The shift focuses on the ways that knowledge is mediated in 
particular settings and on the "schemata" and representations that "users" 
bring to bear on information and expertise presented to them.  According to 
this approach, the user acts upon information by relating it to existing 
knowledge, imposing meaning and organization on experience and, in many cases, 
monitoring understanding throughout the process.  This casts the user as an 
active problem-solver and a constructor of his or her own knowledge, rather 
than as a more passive receptacle of information and expertise.

     Similarly, and almost coincidentally, researchers, scientists and teachers 
began debating the merits of using this perspective rather than other 
approaches to mathematics and science education (e.g., Schuell, 1986).

Clarification of Terms

     The field labeled "knowledge use" includes a wide range of practices and 
perspectives, including those of "dissemination" and "diffusion."  In 
particular, the term "knowledge dissemination" has different meanings to 
different people.  Its most common definition is the transfer of knowledge 
within and across settings, with the expectation that the knowledge will be 
"used" conceptually (as learning, enlightenment, or the acquisition of new 
perspectives or attitudes) or instrumentally, (in the form of modified or new 
practices.)  There are, however, those who see dissemination as having other 
legitimate outcomes.  Some of these outcomes include:  (1) increased awareness; 
(2) ability to make informed choices among alternatives; and (3) the exchange 
of information; materials or perspectives.  In 1977, a  conference of 
dissemination professionals, the Dissemination Analysis Group, defined 
dissemination as including these outcomes, as well as conceptual and 
instrumental use of new knowledge.  The implication was also drawn that 
different strategies are needed to achieve each purpose.  In this review we 
focus most of our attention on use as the ultimate goal of dissemination, with 
the recognition that validated knowledge is mediated.

     Another clarification in this review has to do with what is being 
disseminated.  In the studies reviewed, the object of study ranges from 
research results (e.g., a teacher's use of wait time or learning opportunities 
as applied to a classroom environment)  to craft-validated knowledge, (such as 
constructs developed by teachers, then used by their peers), to products (e.g., 
curriculum materials, guidebooks, or videos).  We will also take into account 
new practices (such as approaches to teaching scientific concepts or peer 
coaching), along with policies (such as state mandates), and new forms of 
cognition involved in science and mathematics teaching and learning.  In the 
synthesis,  we will also be discussing the extent to which factors found to 
influence successful dissemination do so for several of these units of 
analysis.

     Still another variation in the studies reviewed involves the "target" of 
dissemination.  Some analysts in research utilization see as the target those 
people traditionally called "users" or "subjects" of the study or evaluation.  
These are generally practitioners, often teachers.  The dilemma here is that 
researchers typically study topics which may not be of obvious import to 
practitioners.  Alternatively, the products being disseminated, such as reports 
or articles, do not necessarily read in ways that practitioners can understand 
and apply.  How then does one maximize the usefulness of information derived 
from those from whom it was gathered in the first place?

     Finally, other dissemination studies examine the spread or transfer of 
practices developed in one location to another.  In this case the "targets" of 
dissemination are not at the site of development.  Therefore, the core question 
becomes, how is knowledge mediated or "packaged" for use elsewhere from where 
it originated?

Context

     The growth of the field of knowledge dissemination and utilization began 
with large-scale efforts to improve the nation's schools in the 1950s and 
1960s.  Some of the earliest efforts were a result of Sputnik, which mobilized 
the U.S. to invest in the development or invention of new classroom materials 
and methodologies for improving  the education system, particularly the 
curriculum of mathematics and science.  The dissemination strategies used were 
based on the simple conviction that if materials were good they would be used, 
and that their use would result (automatically) in further diffusion.  To some 
extent, the task of improving schools was conceived to be a problem of getting 
such innovations adopted, and the prime locus of improvement was posited to be 
the classroom.

     During this period, the National Science Foundation began to use Teacher 
Institutes to promote the use of the newly developed materials and 
methodologies.  At the same time the U.S. Office of Education used another 
strategy:  demonstration projects.  It appeared that neither agency's strategy 
resulted in robust and durable changes at the classroom level.

     As educators and researchers discovered that utilization of these 
materials and approaches was not automatic, they grew more interested in the 
successive "stages" of the dissemination and utilization processes.  These 
include adoption -- the decision to use innovations --(Rogers & Shoemaker, 
1971); implementation -- actual use -- (Fullan & Pomfret, 1977); and 
institutionalization -- "routinized" use -- (Yin, 1978; Crandall, 1984).

     In the next phase, roughly in the 1970s and early 1980s, two trends were 
evident in the knowledge use literature.  One was the shift from centralized to 
decentralized knowledge transfer systems (Rogers, 1986), the apparent result of 
the adoption of a more systemic perspective on planned change.  Another was the 
growing influence of constructivism.  As noted earlier, constructivism, when 
applied to knowledge use, views the knowledge generated and used by policy 
makers and practitioners as largely self-constructed.  In the sub-field of 
social constructivism, this construction of meaning is typically a product of 
social interaction (Dunn and Holzner, 1988; Rogers and Kincaid, 1981).  In the 
classroom context, this perspective was construed as "mutual adaptation," when 
innovations designed for replication were implemented but were also modified by 
users (Fullan and Pomfret, op. cit.).

     Similarly, efforts to restructure mathematics and science learning systems 
were being viewed differently at this juncture.  For example, restructuring, as 
it is currently conceived, has to do with total systems change, all the way 
from individual practices through organizational levels, then up to senior 
policy makers.  Schlechty (1990:xvi) defines restructuring as "altering systems 
of rules, roles, and relationships so, that schools can serve existing purposes 
more effectively or serve new purposes altogether."   In discussing the 
influential Concerns Based Adoption Model (CBAM) Hall and Hord (1987) and 
Loucks-Horsley and Stiegelbauer (1991) speak of total systems change:  change 
involving people, processes, practices, policies, power [and empowerment], and 
philosophy.  It is now argued that multiple perspectives from multiple 
disciplines are required to assess the needs of complex and multi-faceted 
systems.  It is now accepted by many contributors to the mathematics and 
science literature that it is inadequate to focus on changing individuals 
[teachers] if the goal is long-term impact on a system. Nonetheless, individual 
change in the teaching and learning environments of science and mathematics 
still presents some persistent difficulties (Darling-Hammond, 1990; Cohen and 
Ball, 1990; Tobin and Espinet, 1989).

     Staying for a moment with this theme, Davis, Maher and Noddings (1990) 
identify two schools of thought regarding current concerns about the quality of 
mathematics teaching in the United States.  The first is characterized by 
increasingly explicit directives:  more days per year and more math per week; 
more homework, more explicit identification of knowledge that students are 
meant to acquire.  The second, almost antithetical to the first, proposes less 
testing, a less constrained learning environment, and a more natural fit with 
students' daily lives.  The latter school of thought corresponds more to a 
constructivist perspective on the teaching and learning of mathematics.  This, 
according to the authors, has become the prevailing view among mathematics 
education researchers.  For example, Confrey (1990:108-111) describes 
constructivism in mathematics education as follows:

Constructivism [is] essentially a theory about the limits of human knowledge, a 
belief that all knowledge is necessarily a product of our own cognitive acts . 
. . When one applies constructivism to the issue of teaching, one must reject 
the assumption that one can simply pass on information to a set of learners and 
expect that understanding will result. . .  Thus as a constructivist, when I 
teach mathematics, I am not teaching students about the mathematical structures 
which underlie objects in the world; I am teaching them how to develop their 
cognition, how to see the world through a set of qualitative lenses which I 
believe provide a powerful way of making sense of the world, how to reflect on 
those lenses to create more and more powerful lenses and how to appreciate the 
role these lenses play in the development of their culture.  I am trying to 
teach them to use one tool of the intellect, mathematics.

     The systemic and constructivist perspectives now so prominent in the 
literature of knowledge use, as well as in the mathematics and science 
education literature imply the need for reviewing and rethinking methods for 
disseminating knowledge related to the outcomes of teaching and learning in 
mathematics and science.  The main point of this literature is that not only 
should new initiatives reflect this thinking, but also materials which were 
unsuccessfully disseminated during earlier initiatives might be introduced anew 
using a constructivist approach.

     For example, it cannot be said with certainty that prior innovations were 
without benefit, since vestiges of the early policies in science and 
mathematics are very much in evidence in many of today's classrooms.  
Furthermore, in declaiming the benefits of earlier reform efforts, researchers 
may well have overlooked significant intervening variables which, as Davis 
(1990:94) posits in his review of the "new math" experience, might have cast 
earlier programs in a different, perhaps more positive, light.



     Historical Development

     The late 1950s and early 1960s began a period of unprecedented activism in 
education. The National Science Foundation supported major, large-scale 
curriculum development to improve science education.  Using primarily Teacher 
Institutes and commercial publishers as the dissemination vehicles, the results 
were mixed.  Some good materials were developed but rarely used--used, in fact, 
far less than the investment warranted.  This was particularly true for the 
elementary level where ambitious programs were developed, but often sat unused 
in closets nationwide.

     The Elementary and Secondary Education Act of 1965 also supported the 
major development of educational innovations.  Arising out of uncertainty about 
just which "interventions" would improve education, the Act created 
demonstration programs in local schools.  Great faith was placed in the 
inventive capacity of the people in demonstration schools to address the 
problems of schools in general.  In fact, most sponsors at demonstration sites 
assumed that their programs would diffuse almost by "osmosis" to other schools. 
 However, the evaluations carried out in the late 1960s and early 1970s found 
that while most demonstration sites had a dissemination plan, they did not put 
sufficient effort into creating awareness on the part of other schools.

     As a result, U.S. Department of Education efforts in dissemination were 
expanded in the late 1970s, experiments were conducted; and new "delivery" 
systems were created.  Most notable are:

The Project Innovation Packages (PIPs) project, in which six complete 
innovation "packages" were created, representing new approaches to compensatory 
education in reading and mathematics.  Having been proven effective in school 
district sites, the packages were designed to be implemented by teachers and 
administrators with no other information or assistance beyond the printed 
materials.  (Ironically, they were delivered in custom-crafted plexiglas boxes, 
which still endure in many places long after the accompanying instructional 
materials which they contained have disappeared.) The evidence suggests that 
the PIPs resulted in little change.  At best, teachers timidly implemented the 
approaches, adapting them so much that there appeared to be little difference 
from prior practice.  (Horst et al, 1975).

The Pilot State Dissemination Project (PSDP) set up "dissemination agents" in 
three States, each agent serving a school district or a county.  The function 
of these agents, modeled on the county agents of the Agricultural Extension 
Service, was to provide a direct link between schools and researchers and their 
innovations.  The project was highly successful, producing effective use of 
research-based practice in the targeted districts (Sieber, Louis, and Metzger, 
1972).  The cost, however, of extending the network to the whole country -- by 
one estimate a billion dollars per year -- prevented the program's expansion.

The Research and Development Utilization (RDU) Program operated from 1976 to 
1979.  Its approach was neither as remote as PIPs nor as intensive as PSDP. 
Sponsored by the National Institute of Education to disseminate its educational 
research results to 300 schools, the program included several elements known to 
promote successful dissemination:  on-site assistance (Havelock, 1969); 
matching of R&D to the schools' contexts; quality assurance and control of the 
information provided; and funding for school research utilization efforts.  An 
evaluation of the program concluded that it was a model of a well-designed 
dissemination strategy which "can be effective in promoting improvements in 
schools, in educational practice, and in benefits to students" (Louis and 
Rosenblum, 1981).

The National Diffusion Network (NDN) was set up in 1974 to create a delivery 
system for effective programs, and it still operates some 18 years later.  The 
NDN was created with the hope that a broader educational community would 
benefit from the initial investment in program development made by the U.S. 
Department of Education, in order to create a national mechanism for 
dissemination of curricula and other instructional products.  The system 
included the most promising features from the previous experiments.  For 
example, programs were formally validated and judged by a panel of experts, and 
person-to-person assistance was built into the system (Crandall et al, 1982; 
Huberman and Miles, 1984).  "State facilitators" (at least one for each of the 
50 states and the territories) help schools in their State to learn about, 
assess and, when appropriate, to use innovations that are locally developed or 
generated. Developer/demonstrators" (at least one representing each validated 
program) provide materials and training programs to facilitate adoption and 
implementation.  Reviewers have concluded that the NDN represents "one of the 
few highly successful Federal efforts to make wide-scale use of important 
developmental improvements in educational state-of-the-art" (Emrick & Peterson, 
1978).  It is interesting to note that the NDN is currently spending more time 
on the later phases of implementation, e.g., consolidation, stabilization and 
routinization and is redesigning itself to focus more on these areas.

The Educational Resources Information Center (ERIC) Clearinghouse system is 
another dissemination strategy created during the 1970s and still functional 
today. Clearinghouses serve an archival function as well as a means of getting 
information out.  It has been assumed that, generally, their impact is limited 
in that they produce mainly print materials (syntheses, analysis, etc.).  
However, when other support services are connected with these functions in the 
form of tailored searches with the help of people who are knowledgeable about 
the data base and understand the request, consequential use is increased, 
especially when the client has a specific reason for the information.  ERIC 
also generates knowledge through information synthesis publications, such as 
monographs and digests on topical subjects.

Commercial distribution is another widely used mechanism for disseminating 
educational information and programs.  However, this strategy has been found to 
be ineffective in actually promoting use of educational materials (Crandall et 
al, 1982). In addition, the experiences of large-scale curriculum developers 
supported by the National Science Foundation in the 80s highlight the problems 
associated with commercial distribution when it is used as a unique vehicle for 
dissemination.


     Each of these programs can be characterized by its approach to knowledge 
dissemination and diffusion:  top-down, bottom-up and middle-out.  Here, too, 
there is a relatively robust body of evidence.  More globally, the pertinent 
questions to pose in reviewing these and similar knowledge dissemination and 
utilization efforts are fairly straightforward:  what can be learned from these 
efforts?  How can future dissemination designs capitalize on the successes and 
failures of prior work?  What are the salient barriers to utilization?  Which 
factors are the most systematically associated with successful use?

Barriers to Utilization

     The experiments of the mid-1970s, along with program evaluations through 
the present time, indicate that top-down, linear models of dissemination, 
consonant with the RD&D or RDD&E models that were the starting points for the 
field of dissemination and use, were of limited value in making potential users 
aware of new information, and were generally ineffective in promoting 
institutionalization of ideas and incorporation of materials into the teaching 
curricula of local schools.  In effect, a number of barriers to utilization 
associated with top-down approaches to dissemination are cited in the 
literature including a natural tendency on the part of school personnel to 
preserve ongoing norms and practices, and to cope with a set of confusing, 
sometimes contradictory policy and program directives.  Clearly, when we look 
at the micro-politics of school life, including the tradeoffs between subgroups 
and the contrasting pressures on school staff, we are not operating in the 
realm of "rational" policy and practice.  Or rather, this configuration may be 
functional for local staff, but exotically singular for external observers.

     Other constraints to an orderly process of adoption and implementation 
include shortcomings in the mastery of subject-matter content on the part of 
individual teachers, as well as the inattention to more systemic issues in the 
school's social environment.   Other barriers have to do with the mismatch 
between the demands of the product or program under review, and the core values 
and beliefs -- the teaching and learning culture -- in mathematics and science 
education on the part of local staff.

     Let us return to the constraint of preserving the status quo.  Prior 
research in knowledge use suggests that failures to institutionalize knowledge 
are attributable to what is known as "knowledge disavowal," particularly  in 
the case of knowledge in the form of findings from research.  Zaltman 
(1983:173) describes knowledge disavowal as avoiding knowledge "in order to 
preserve or maintain the status quo or to avoid a difficult choice or a 
threatening situation."  (We should remind ourselves that such "disavowal" can 
occasionally be justified, i.e., that some people and organizations may have 
seen accurately that a given product or project might be detrimental to the 
organization or to themselves).

     Another observation which persists in the literature on organizational 
research is that researchers and users belong to separate discourse 
communities, with very different values and ideologies.  Oftentimes, the social 
and cultural distance between the two communities impedes utilization (Beyer 
and Trice, 1982).

     Still, findings in studies of mental health and among social work 
professionals suggest a different pattern in the use of social science 
research.  Weiss (1977, 1980) describes an "enlightenment" model in which 
research findings gradually become a useful part of the professional's frame of 
reference, and are called upon as needed in the conduct of policy- and 
decision-making.  These findings were corroborated by Sunesson and Nilsson 
(1989), who found that circumstance and context were important indicators of 
research utilization and that scientific rigor was not necessarily a detriment 
to use.

     Studies of the early, linear, one-way models of dissemination show while 
they resulted in educators' adapting innovations, this occurred often in 
unproductive ways.  Educators were found to be more likely to modify an 
innovation than to use it as its developers had intended. While adaptation is 
often a way of enhancing the usefulness of new knowledge in a new setting, it 
can also result in "mutating" the form so greatly that it no longer has the 
shape, nor the outcomes, that originally attracted the user (Hall & Loucks, 
1978; Huberman & Miles, 1984).  This appeared to be the case in the projects 
reviewed in these empirical studies.

     Dissemination strategies are challenged by the need to make order out of 
confusion in schools.  In referring to California's mathematics reform 
initiative, Cohen and Ball (1990) and Darling-Hammond (1990) point to the 
evidence of previous dissemination efforts which can be found in most American 
classrooms today.  They suggest that the features of today's schools are an 
amalgam of prior instructional policies and programs, and that much of what is 
not right in today's classrooms reflects teachers' efforts to make sense of the 
layers of old and new policies and methods.  This observation has concrete 
implications for the implementation process in the dissemination cycle.  As 
Cohen and Ball (1990) suggest, changing teaching methods is not like changing 
one's socks.  That is, it can not be done quickly or cavalierly, and when real 
implementation occurs, it will not be easy to undo it for the next wave of 
instructional reform.

     Implementation can be elusive.  For example, the Rand study of federal 
dissemination efforts (Berman & McLaughlin, 1978) found that specific project 
content did not insure program success.  Nor did the amount of money or effort 
expended in dissemination. Similarly, external consultants and one-shot 
training approaches tended to fail.  Finally, those innovations that did make 
it into the schools were adapted in some way, sometimes dramatically.

     Findings from studies of use of innovations help to show why this is so.  
One of the classic studies of innovations at the classroom level (Hall and 
Loucks, 1978) found that when a new practice is introduced to the classroom, 
teachers deal with it on many different levels. Some would not use it at all; 
others would use the new product or practice in a more mechanical, yet 
disjointed, way.  Still others would get farther in the process of technical 
mastery by integrating the innovation into their routine teaching practice, 
without, however, incorporating the more complex and demanding components of 
the innovation into their instructional repertoire.

     These different Levels of Use (LOU) are now widely accepted as a 
description of the process through which individuals move as they become more 
comfortable and proficient in their use of an innovation.  By inference, 
different dissemination mechanisms would be needed to address the differing 
needs teachers have as they move through the stages.  The significant finding 
from the earlier studies is that, even when the linear models for dissemination 
were "successful" in getting a product through the classroom door, they were 
not decisive in firmly rooting the innovation in place (Hall and Loucks, 1978; 
Fullan and Stiegelbauer, 1991).

     In effect, those innovations that do make it into practice may not be very 
long lived. The Rand study of large-scale, federally-sponsored innovations 
found that improvements dissipated quickly or even disappeared when the 
disseminated program came to an end.  The end of financial support or the 
departure of key personnel may also be fatal.  In addition, innovations that do 
last longer may revert to the original practices they were meant to supplant.  
In that way, the "innovative" practice gradually becomes indistinguishable from 
its predecessor.

     A 1990 study of classroom experiences with "constructivist" mathematics 
reform suggests some reasons for this phenomenon.  In this study of mathematics 
instruction following new policy and curriculum directives in California (Cohen 
and Ball, 1990) resume their research by pointing out that, "any teacher, in 
any system of schooling interprets and enacts new instructional policies in 
light of his or her own experience, beliefs, and knowledge . . . Teachers view 
themselves as independent, autonomous professionals . . .  Even the most 
obedient and traditional teachers observed, enacted policies in their own ways 
and were proud of their contributions." (p. 253) These researchers suggest that 
local transformations occurred all along, and that developers who returned a 
few years later were surprised by transformations that did, apparently, harken 
back to the original practices.  How far they got along the Hall & Loucks 
spectrum remains unclear.

     Paradoxically, many of these innovations were meant to be implemented 
"faithfully." Darling-Hammond (1990) suggests that the adaptation of 
innovations in the classroom is a natural extension of the process by which 
dissemination occurs.  "Teachers receive the message through a filter with much 
of the information and most of the contextual clues screened out.  They are not 
expected to interpret the policy by constructing meaning for themselves, but 
only to implement the simplified version of it that reaches them (p. 236)." As 
we see with Cohen and Ball, this is not what transpires; there is a way to 
avoid the local reconstruction of the practice, as local staff make sense of it 
in their own context.  Given the conditions of local "delivery," there was no 
way that a "faithful" version of the new practice could have been implemented.

     Knowledge deficiencies, too, present a barrier.  In studying the effects 
of coaching in high school science teaching, Tobin and Espinet (1989) cite 
teacher preparation as a significant impediment to change, notably when the 
teacher is poorly grounded in science and science pedagogy.  A root problem may 
be that the teacher's qualifications in science and as a science teacher may be 
insufficient, making it doubly difficult to improve science teaching.

     These findings are significant to science education because of enduring 
concerns for improving high-level cognitive learning in science (Tobin 1987) 
and the importance to effective science teaching of teachers' knowledge base 
(Arzi, White & Fensham, 1987). Science teachers frequently are encouraged to 
implement new curricula which utilize different approaches to teaching and 
learning.  Many also endeavor to improve aspects of teaching through 
self-observation, informal feedback from peers, or through formal evaluations.  
These are imperfect mechanisms.  There may well be no way to progress 
technically and to stay abreast of the successive waves of reform in curriculum 
and instruction without support from experienced science educators and peers 
(p. 119).

     Organization or system-wide issues can also impede implementation of 
innovation.  Use of innovative practices that have been in place several years, 
and that are clearly recognized as superior to previous practice may still be 
discontinued, essentially because systemic issues have not been adequately 
addressed or because the conditions in the setting do not support continued 
use.  For example, successfully- introduced innovations may disappear because 
the requisite resources and supplies were not provided by the organization.  In 
science education in particular, resupply of expendable materials used in 
hands-on science programs, particularly in elementary schools, is crucial.  Its 
absence is likely to result in abandonment of the innovation (Anderson et al., 
1992.)

     These innovations may also disappear because the requisite funding never 
became a line-item in the organization's budget, or because job categories were 
not revised in order to make the coordinator's post a regular classification, 
or because other organizational arrangements were not "routinized (Yin et al., 
1978).  Additionally, innovations could fail because training practices within 
an organization were not changed to incorporate the necessary training routines 
leading to mastery of the innovation in question.  Clearly, there are many 
reasons why an innovation that seems to be successful after an introductory 
period may nevertheless be abandoned.  Thus successful dissemination efforts 
would lead to the creation of mechanisms for ensuring longer-term 
organizational support (Miles, 1983).

     What was wrong, then, with the idea that superior knowledge, properly 
researched and developed, would diffuse into the appropriate settings, be 
recognized for the benefits it offered, and become a lasting part of a new 
user's routine?  In summary, the top-down character of this approach "paid 
little attention to users' frames of reference and assumed --through a sort of 
magical hyper-rationalism -- that strong findings would override countervailing 
policies and practices to which locals had been committed up to then" 
(Huberman, 1989).  It did not work because the process of acquiring new 
knowledge is not a linear process.  New understandings are grafted onto prior 
understandings.  Individuals must have ample opportunity to experience the new 
information and develop their understanding of its meaning to their existing 
knowledge (Huberman, 1973).  More generally, the notion that teachers are 
conduits for information and knowledge, not active participants in creation, 
dissemination and utilization, is flawed.  Teaching cannot be controlled by 
prescriptions for practice embodied in texts, tests, and monitoring schemes.  
(Darling-Hammond and Berry, 1988).

     So teachers are the agents who cause the instructional problems that State 
and federal policies seek to correct, yet, at the same time, they are the 
agents for their own correction. More often than not, the conditions for 
successful monitoring and correction are not met. Here is a pertinent excerpt 
from a study of the California mathematics reform initiative (Cohen and Ball, 
1990).

"Few teachers had opportunities to see examples of the sort of teaching that 
the State thinks it wants.  Few have been offered opportunities to learn a new 
mathematics.  Few have been given opportunities to cultivate a new sort of 
teaching practice, and even fewer have been offered assistance in the endeavor. 
In a word, teachers have yet to be engaged in the sort of conversation -- with 
themselves, with other teachers, with university mathematicians, and many 
others--that would support their efforts to learn a new mathematics, and a new 
mathematical pedagogy (p.254).

     Top-down, linear prescriptions for knowledge transfer were self-evident in 
the earlier (1955-1975) knowledge use literature, just as they were evident in 
the science and mathematics education literature of the 1950s and 1960s.  Davis 
(1990:104) suggests that during this period, when curriculum improvement 
projects' were being poorly implemented and incorrectly analyzed, the dominant 
psychology was behavioristic.  Behavioral theory held that the concern for 
cognitive events  -- what was going on in a person's mind -- was not scientific 
since it speculated about matters that were essentially unknowable.  So 
teaching strategy focused on demonstrating and supervising and on tests and 
immediate reviews, much as in "drilling."  Knowledge was viewed as the ability 
of the student to render factual and operational components successfully.  
Constructivism was generally unknown.  Although scientists, mathematicians and 
teachers involved in the several curriculum improvement projects knew 
intuitively that a 'stimulus-response' approach misrepresented the complexity 
of real-life instruction and learning, a formalized conceptualization of the 
process of learning mathematics was not readily available.  Davis argues that 
the reason for the "sudden eruption of 'constructivism' as a central concern of 
so many contemporary researchers" is the recognition of "very great need for a 
better way to think about how human beings deal with the subject called 
'mathematics'." (p. 104)  Science education researchers voice similar arguments 
for restructuring science learning and pedagogy (Wheatley, 1991, Lythcott and 
Duschl, 1990; Duschl, 1990; Driver, 1983; Pines and West, 1986, among others.)

     Rejection of the 'stimulus-response' methods of teaching science and 
mathematics in favor of the constructivist approach also suggests another 
direction for the improvement of dissemination efforts.  If people can interact 
with those who produce new knowledge, articulate how they might modify it, 
suggest how the originator of the program or project producer can revise it to 
make it more applicable, potential adopters might feel a greater sense of 
propriety over the knowledge base.  In short, this theory of knowing leads to 
an explanation for the utilization of knowledge known as the problem-solving 
model, one including social interactions as well as a "dialogue" between the 
producer of information and the user, at several points in the problem-solving 
cycle (Huberman, 1973).

     According to this model, knowledge producers and users should belong to 
overlapping networks and have ongoing communication for utilization to occur 
(see Figure 2). Communication between knowledge producers and users as well as 
practitioners and program and product developers, then becomes a two-way 
street.  The feedback by which users communicate their needs to knowledge 
producers becomes as important as the transfer of knowledge "downstream" from 
expert to user.  The features missing from the R, D and D models -- features 
that must be introduced into more effective dissemination strategies -- are 
two-way communications, to provide feedback from the users of their needs, and 
iteration, so that users can learn new products (materials, practices, ways of 
thinking) by repeatedly readjusting their understandings, by checking as needed 
with the expert and designated intermediaries as new practices are introduced.
     Louis and Dentler (1988:51-52) use the term "social processing" to 
describe how educators can use information more effectively.  By testing the 
value of the information in terms of its 'fit' to the local setting, and by 
their involvement in adaptation and development activities, local educators 
begin to build or affirm their commitment to information use.  Impetus for 
social processing is likely to be gained through explicit requirements for 
meeting and planning activities and by the promise of concrete gains in terms 
of professional growth and rewards.

What Works?

     Louis and Dentler (1988) cite criticisms of the bottom-up approaches to 
implementing innovations in the classroom, including impediments such as 
conservative school culture (Sarason  1971), competing pressures on teachers 
(Huberman 1983, 1985; Hargreaves 1984), the propensity to choose innovations of 
low quality (Nelson and Sieber 1976), and the recycling of reforms which lead 
to perpetually reinventing the wheel (Mann 1978).  However, the authors 
disagree with these criticisms and argue that a school-focused knowledge use 
approach as a strategy for inducing change is well founded in the "classic" 
knowledge use studies in education (Mort 1964; Carlson, 1965; Sieber, Louis and 
Metzger 1972; Louis, Rosenblum and Molitor, 1981; Crandall et al.,1983; Sieber 
1981; and Menlo 1985).

     Louis, Dentler and Kell (1984) conducted a three year study of activities 
undertaken in Regional Education Laboratories and State Education Agencies.  In 
revisiting their work, Louis and Dentler (1988) describe implementation 
problems that surfaced in the study findings and which led to developing a 
school-focused knowledge use model for implementing education reform policies 
promulgated by the State.  The model "is school-focused in the sense that local 
conditions within specific schools are expected to influence the course of 
knowledge use and improvement in all phases of change (p. 37)."

     The authors suggest four elements based on their research which can 
constitute a knowledge use model for school improvement:

capitalizing on known or identifiable incentives for change;

providing information or knowledge that has characteristics that make it 
'useable' by the relevant practitioners;

creating or supporting opportunities for shared understandings of how new ideas 
could help to improve local practice; and

stimulating increased diffusion of new ideas within and between educational 
agencies.

     The model is neither a top-down nor a bottom-up approach, but combines 
aspects of each. It also reflects some of the concerns articulated by Cohen and 
Ball (1990) in their interviews with teacher change-agents, as mentioned 
above.

     Louis and Dentler concur with other educational researchers that the 
process of implementation is a core issue in educational reform (Cuban, 1990; 
Sykes, 1990; Lieberman, 1990; Schlechty, 1990; Hall and Hord, 1987, among 
others).  This is also the case for those writing specifically on mathematics 
and science pedagogy (Davis, 1990; Davis, Maher, and Noddings, 1990; Anderson 
et al, 1992; Anderson 1984).  For example, several studies found that both 
personal incentives and organizational incentives were strongly associated with 
use, but that personal incentives were a more potent force.  External stimuli 
alone have limited impact in producing the openness required for the adoption 
of new ideas.  However, mandates, when combined with personal incentives, 
improve the prospects for implementation.  Mandates may stimulate strong 
personal incentives when professional rewards are visible, concrete and 
personally meaningful.

     This line of inquiry affirms the value of carefully mixed implementation 
strategies.  Louis and Dentler (op. cit.) warn that mandates without resources 
to 'fit' new requirements to context in meaningful ways can have a damaging 
effect.  "If time, money and knowledge resources of the right sort are 
available, school staff can often make good improvements without restrictive 
and potentially damaging new state statutes (p. 59)."

     Observations on the value of personal-professional incentives appears to 
have been borne out in the personal experiences of K-8 mathematics teachers 
participating in an on-going project in New Jersey schools (Maher and Alston, 
1990.)  The project's constructivist perspective on learning was based on the 
concept that learning is contingent upon the activity and involvement of the 
learner.  "Teachers-as-learners" and "teachers-as-researchers" were encouraged 
to communicate among themselves and with project staff their own mathematical 
thinking and that of their students.  Profound changes were reported in both 
administrators and teachers over the course of the project, both in 
participants' willingness to assume leadership roles in diffusion of 
innovations, and in their professionalism as teachers of mathematics.  
Diffusion was widespread; teachers presented at professional conferences, 
consulted with colleagues, participated as workshop leaders in other projects 
and summer institutes, and regularly received visitors who came to observe 
children doing this kind of mathematics (p. 163).

     With regard to diffusion strategies, Louis and Dentler note that indirect 
diffusion, when the strategy is well-planned and guided, works more effectively 
than direct diffusion strategies. Indirect diffusion involves information with 
educators who are trained to convey it to others. The authors also note that 
"diffusion behaviors tend to damp out as they move further from the initial 
source of communication" (p. 50-52)."

     In studying the diffusion behaviors of educators in the so-called status 
hierarchy, the authors found that high diffusion behaviors were associated with 
R&D laboratories and universities, and that low diffusion behaviors were 
associated with teachers and administrators (p. 53).  Although teachers are the 
intended audience for most educational reforms, the study findings indicate 
that information is rarely sent directly to them, but rather through 
intermediaries within the school, that is, school-based specialists and 
administrators. Principals, despite their role in mediating communications 
within schools, are the least likely role group to engage in frequent 
communication behavior.  They tend to receive a great deal of information, but 
rarely pass it on, even to their teachers.  This study also indicates that when 
teachers do receive information, they tend to share it with their peers.  When 
information did not diffuse, it was perceived to be inappropriate or not 
useful.  Summarizing their data, the authors conclude that "if teachers do not 
regularly receive information with positive pressures to use and discuss it, 
the opportunities for school-focused strategies for improvement are clearly 
limited (p. 54)."

Middle-Out Dissemination Strategies

     Indirect diffusion strategies are not a new concept.  Described in the 
knowledge use literature as "linking agents" (Havelock, 1973; Davis and 
Salasin, 1978; Louis 1980), people who act as intermediaries between knowledge 
producers (e.g. researchers) and users, these conveyers perform many useful 
functions for the user.  They provide information, they cast it in forms 
relevant to the user, they answer questions, they provide a way for the user to 
test his or her understanding, and they can take questions back to the 
knowledge producers.  An effective dissemination process may require linkage to 
more and more remote resource people, creating what Havelock called the "chain 
of knowledge utilization," or "chain of knowledge."

     For the dissemination field that started with an "action-at-a-distance" 
model of information transmission, the notion of a series of person-to-person 
links between producers and users was a revolutionary one.  The Research, 
Development and Utilization (RDU) program, with its emphasis on bottom-up 
decision making, was aimed at maximizing teacher involvement in selection and 
ownership of the program along the way, also incorporated these principles.  If 
what has been learned about linkage were summed up in two points, they would be 
that 1) the nature of the material that is being disseminated is less important 
than the links all the way down the line, and 2) the shorter the chain of 
knowledge, that is, the fewer the number of links, the more effective the 
process is likely to be.

     In the field of organizational research, many analysts have called for the 
development of new roles and occupations to provide linkages between 
researchers and users.  There is, however, a trade-off.  Knowledge producers 
must weigh the advantages of completeness, accuracy, and strong advocacy in 
direct contact with a few potential users against a less complete, less 
accurate, less hortatory but wider dissemination via indirect linkages.  When 
the goal of dissemination is replication of model programs, the concern for 
accuracy of the implementation of key elements of the model often outweighs the 
concern to reach large numbers of potential users.  This has presented a 
dilemma for professionals associated with dissemination systems, like the 
National Diffusion Network, that often counted the numbers of adoptions -- not 
the quality of adoptions -- in making funding decisions.  Ironically, it is the 
labor intensive use of linkers that has contributed so much to the NDN's 
success as a dissemination system.

     Because linkers also aid the flow of information across social and 
organizational boundaries, they are also called "boundary spanners."  A review 
of the experience of the Agricultural Extension Service found that a "spannable 
social distance" between links in the chain of knowledge, where "distance 
reflects levels of professionalism, formal education, technical expertise, and 
specialization" was one of the elements that characterized the success of that 
dissemination model (Rogers, 1988).
     A veritable industry of intermediaries who serve this function in 
education has grown up over the past twenty years.  They are employed by State 
educational agencies, consulting firms, organizational entities created to 
carry out a specific dissemination program, schools or school districts, or 
research organizations.  They serve varied roles that are meant to promote use 
of knowledge from practice and research.  These roles include:  convener, 
problem-solver, trainer, evaluator, and planner.  More generically, we might 
call them "change agents."  The experience of the field agents in the RDU 
program and that of the State Facilitators and Developer-Demonstrators of the 
National  Diffusion Network illustrate the positive impact of dissemination 
programs that encourage sustained interactions with the users, and that stress 
the role of linkage agents in providing two-way communication.

     In the RDU program, there has been a double linkage between the research 
organization and the users.  Closest to the user is typically a "field agent," 
who engages in face-to-face contact with teachers in local schools.  Another 
link is provided by a person in a given State, region, or consortium project 
office.  This is a "knowledge-base staff," with extensive training in or 
experience with R&D products.  A third link, used in some cases, is a still 
more highly-trained technical expert, perhaps one of the specialists who has 
developed one of the products under consideration.

     Typically, a group is formed in each school which meets with the field 
agent to identify a problem to address.  Consulting with the next link up the 
chain, the field agent would then select a set of up to 10 products for review. 
 This array is then screened by the group and reduced to two or three.  The 
knowledge-base staff would then typically be consulted directly by the teachers 
for more detailed information about the products selected.  In addition, at 
some point in the implementation process, the field agent would arrange for the 
local group to get focused training (the third link).  Depending on the 
project, this process might be repeated several times, with direct contacts by 
the users to higher links in the chain.

     A process of choice among many alternatives was thus built into the RDU 
program, along with a process for exchange of information about the suitability 
of each of these alternatives. Furthermore, the exchange of information was 
seldom a one-time activity, such as a single training session, but a process to 
be repeated as needed.  Finally, the users were active in the process, seldom 
being put in the classic mode of more or less passive recipients.

     The National Diffusion Network relies on similar linkages to move 
validated programs from place to place.  A "close-to-the-customer" logic 
prevails, with the State Facilitator serving as the linker for the schools 
throughout the State to literally hundreds of substantive experts representing 
the Developer-Demonstrator projects (models) within the National Diffusion 
Network.  The system is organized so that the first link (the State 
Facilitator) interacts with school personnel to identify areas of innovation 
which might respond to local needs and problems, typically by creating 
awareness of the set of validated programs and products available.  The State 
Facilitator then provides linkage to the substantive trainers who can provide 
training, follow-up assistance, and coaching, along with the curriculum and 
other materials designed to support full implementation of the program.

     The combination of elements such as choice, ample in-person assistance 
from a variety of people, and high-quality services, information, and use of 
craft (and oftentimes more scientific) validation of its programs appears to 
contribute to this relatively successful dissemination system.  These 
ingredients were already in the formula for "what works, at least in middle-out 
dissemination strategies.  In particular, the combination of "bottom-up" 
features (attention to local needs, connection to personal and professional 
incentives, focus on "useable" characteristics of the knowledge base) and the 
"middle-out" characteristics just reviewed, appears to predict fairly well the 
extent of implementation in a set of given school districts.

Dissemination Revisited

     Generally, the pooled experience of those carrying out trial dissemination 
programs in the 1970s resulted in an understanding of the complexity of using 
dissemination as a means to improve education.  In effect, disseminated 
products or programs tend to be multiple-purposed and to serve four basic 
functions -- to spread information, to facilitate choice, to promote exchanges 
among practitioners and professionals, and to implement new programs, products, 
and ideas in new settings.  The Dissemination Analysis Group (cf. Klein and 
Gwaltney, 1991) defined these four vehicles of dissemination as:

 -- the one-way broadcasting of information, in order to increase awareness;

Choice -- the provision of information on options intended to help users 
compare alternative resources;

Exchange -- interchange of information, materials or perspectives; and

Implementation -- technical assistance, training, or other forms of support 
designed to change attitudes or behavior and to institutionalize changes over 
time.

     Different dissemination activities appear to work best for each of these 
functions.  For example, when "spread" is the focus of the dissemination 
effort, the disseminator would make information available in a variety of forms 
(e.g. print, audio, video) and through a wide range of vehicles (e.g. 
publications, presentations, telecommunications).  General purpose 
dissemination activities conducted by Regional Laboratories and R&D Centers 
often serve this purpose.  In reporting findings on the characteristics of 
"often-used" research, Weiss (1986) cites brevity, timeliness, attractive 
formats, and good writing as important features.

     "Choice" often is facilitated when an information center or library 
responds to client requests or queries.  Still, this function can also be 
activated through publications, awareness sessions or conferences that select 
and describe programmatic options, along with their respective advantages and 
disadvantages.

     "Exchange" occurs when there are sufficient opportunities for information, 
materials, or perspectives to be shared.  These exchanges can include 
interactive feedback that is solicited and received through visits, meetings, 
field testing, or technology-assisted communication.  In his survey of 
information consumers, Hood (1989) found that informal contacts with colleagues 
are the most frequently used and preferred information source.  In effect, the 
educational information "market" is segmented by work roles that limit exchange 
of information, and the most preferred information gathering format is 
face-to-face, including workshops, seminars and one-to-one contact.

     "Implementation" is fostered through direct assistance, training, and 
sustained support for changes in behavior in classrooms and schools.  
Context-specific applications are recommended here, where the questions asked 
are:  "How will users apply this information?" and "How best can this 
information be tailored to these specific applications?"

     These functions -- from spread to implementation -- form a continuum in 
terms of the level of effort required on the part of the disseminator.  In 
other words, dissemination requires far fewer resources for materials available 
on a database than does the provision of person-to-person assistance.  At the 
same time, the potential impact ranges from low to high. That is, there is a 
greater chance that actual change in practice will result from focused 
assistance than from access to print materials.  And yet it is often the full 
range of activities, implemented in a staged fashion, that results in the most 
successful dissemination effort.  To streamline this discussion, the continuum 
may be represented as follows:

spread . . . choice . . . exchange . . . implementation.

     Activities aimed at "spread" alone may, in certain contexts, contribute to 
use of knowledge. In cases where there is a strong perceived need or a mandate 
for change, simply getting information to a teacher or administrator or school 
district may produce results (Louis and Dentler, 1988; Louis, Dentler, Kell, 
1984).  On the other hand, lasting implementation typically requires a more 
sustained effort, usually associated with dissemination activities on the right 
side of the dissemination continuum.  It is here that the role of the user is 
substantially more interactive than in the R,D & D model.  In the first phase, 
information flows primarily from the producer to the user, but in successive 
stages the flow of knowledge is a two-way or multiple-way flow, as the users 
ask questions and make choices among several options.  Whereas the R,D & D 
approach can wander into casting the user as a "target" or "empty vessel," the 
current concept of dissemination recognizes that the user will play a role, 
whether intentional or not.  Thus approaches which do not build in multiple 
ways for the user to interact with the information, program, or product being 
disseminated are more likely to fail.

     Disseminators are often challenged to clarify -- legitimately -- the 
purpose of their dissemination effort before selecting the most appropriate 
dissemination activities.  The objectives of dissemination would usually 
dictate which mechanisms, delivery systems, or activities would best support a 
given dissemination effort.  In effect, one may not need to employ the full 
range of the continuum shown above if one is disseminating a discrete 
publication or research finding.  For example, the U.S. Department of Education 
used "spread" strategies quite effectively in the early 1980s to release the 
findings of "A Nation At Risk" and recently repeated this strategy with a 
publication for parents:  "Helping Your Child Learn Science."  Yet as the 
object for dissemination becomes complex, products and processes such as 
integrated curriculum, teacher preparation, and mechanisms for systemic 
restructuring call for a more comprehensive organization of dissemination 
modes.  Quite apart from the "delivery" aspect, one must take into account the 
high demands for communication, involvement, and participation of people in the 
schools, together with those of intermediaries in the flow of knowledge.  
Further, it appears that the dissemination strategy must be sensitive to the 
degree of change which are required in these more complex efforts on the part 
of students, teachers, and administrators.

     A brief word on a complementary perspective to the set we have just 
examined.  If one looks closely,  several features are akin to what has come to 
be called the "social marketing" aspects of the change process.  In this 
regard, Schlechty (1990:84-95) differentiates marketing from the 'sell and 
tell' approach:  "Selling begins with a product and then endeavors to persuade 
customers that they want or need what the product offers. . . .  Marketing, 
however, begins with the customer -- what the customer values and needs."   In 
the educational environment, however, "customer" values differ by constituency, 
and the critical tasks are those of determining the core values of the 
constituencies that will be affected.  In particular, one wants to know which 
of these values are likely to be served by a particular change, and, which are 
likely to be threatened.  Although it is impossible to satisfy everyone's 
needs, it is possible to convey an understanding of constituents' concerns, 
both to achieve participation and to show a healthy respect for professional 
collegiality in the process.


Factors Promoting Utilization

     Clearly, the activities undertaken by disseminators contribute to the 
success of the dissemination effort.  Successful dissemination is best 
characterized by seven factors drawn from our review of the research.  Similar 
lists have been compiled by Havelock (1971), Rogers and Shoemaker (1971) and 
Louis and Dentler (1988).

Accessibility, Availability, and Adaptability -- this factor addresses the 
extent of availability of the material.  For example, the ERIC users study 
found that "one- stop shopping" was critical in ensuring use.  If a source was 
listed in a bibliography and was also available at the time of inquiry, it 
would typically be used.  Also under investigation here was the accessibility 
of a given knowledge product to those who do not already share assumptions or 
frame of reference, and the degree to which a knowledge product encourages 
local adaptation or is specifically designed for local adaptation (Louis and 
Dentler, 1988).

Relevance, Compatibility -- this factor addresses the degree to which the ideas 
and information being disseminated actually fit the real-world contexts and 
concerns of practitioners.  One might want to determine, for example, how 
user-friendly the materials are, how well they are matched to the user, how 
well-targeted they prove to be.  Also expressed here is the degree to which the 
content of the material being disseminated is compatible with the concerns of 
users, is adaptable to local conditions, is accessible intellectually for 
inexperienced or uninitiated users, and has been field-tested or field 
developed.

Quality -- plays a key role in getting materials into certain dissemination 
systems, e.g., the National Diffusion Network, the Laboratory Network Program, 
and State dissemination systems which require programs and materials to undergo 
evaluation or validation.  Empirical studies show, however, that people will 
not automatically use knowledge because of its quality or because of its lack 
thereof.

Redundancy of Messages -- This factor underscores the need of a variety of 
dissemination modes.  Several modes of knowledge working together over time, 
through different channels and formats, and suited to different styles of 
learning, mixed with purposeful redundancy, facilitate adoption (Muthard and 
Felice, 1982).

Linkage among Users -- includes important strategies such as interpersonal 
exchanges and connections, along with interpersonal support.  Research on 
knowledge utilization found ". . . a supporting social process . . . In other 
words, interaction with colleagues and peers about the value of information, 
its potential utility and how it might actually affect practice or planning was 
extremely important" in moving from initial levels of use -- learning something 
new -- to more active levels -- deciding to change a practice or a policy 
(Louis, Dentler & Kell, 1984).

Engagement -- This factor addresses the extent to which users are given 
opportunities to actually engage with the new program, practice or knowledge 
product, e.g., hands-on activities to try out a new practice and to get 
comfortable in its use.  This includes analyzing and reflecting about results, 
planning and making it one's own or even "reinventing" it, as well as linking 
with specialists who can facilitate engagement.

"Sustained Interactivity" -- Overall, the best single predictor of knowledge 
use and gain is intensity of contact(s) between disseminators and receivers.  
"The process that succeeds best . . . involves frequent contact, some 
face-to-face interaction, and an exchange between dissemination specialists and 
participants that lasts more than a few months over time (Louis, Dentler, Kell, 
1984, p. 17)."  Sustained interactivity was also illustrated by the RDU 
experience.  The reviewers of that experience found that the amount of time 
that field agents spent with staff at the local schools, both before and after 
initial implementation, was one of the most important predictors of success in 
the effort (Louis, et al., 1981).

     Huberman (1989, 1990) provides a conceptual basis for the idea of 
sustained interactivity for the use of research results.  He defines sustained 
interactivity as "multiple exchanges between researchers and potential users of 
that research at different phases of a given study." He also identifies the 
decisive interactions, interventions and exchanges between researchers and 
users at each of the following stages of a research project's development:

prior to the actual conduct of the study;

during the conduct of the study;

during analysis and write-up; and

during the dissemination phase, when study findings are brought to the user's 
environment.

     Huberman connects this conceptualization and "meta-analysis' with work by 
cognitive psychologists in developing alternative theories of knowing, which 
explains both the nonlinear nature of this process and the need for a highly 
interactive component to make sure that disseminated knowledge is correctly 
represented in the mind of the user.

     Robert Yin and Gwendolyn Moore (1985) conducted case studies of the 
utilization of knowledge resulting from research in the natural hazards field 
(earthquake, flood and other natural disasters) and found that the social 
interactions of researcher with users was the best predictor of consequential 
use of the knowledge generated.  Interestingly, informal contacts, even before 
the research projects were underway contributed to a research use, a finding 
also mentioned in Huberman's literature review (1987).  They summarized their 
description of successful projects as follows:

"What was discovered in these case studies was the persistent role of social 
interactions -- between investigators doing the research and the potential 
users of research results.  The interactions led to a continued exchange of 
ideas, creating what might be called a "market-place of ideas," in which 
investigators learn more about users' conditions, and users learn more about 
the ongoing array of research.  In some cases, the exchange of ideas was 
facilitated by the activities sponsored by professional associations.  In other 
cases, the exchange was the result of an active and communicative principal 
investigator.  Overall, communications started earlier than and continued far 
beyond the ending of a specific research project.  Further-more, the project 
design and conduct could be influenced by information from users, making the 
research more relevant to users' needs.  Where, in contrast, research projects 
were conducted in a more traditional manner --  i.e., far removed from 
potential users -- utilization was impaired." (Yin and Moore, 1985, p. 18).

     A similar finding comes from the field of electronics research.  Drawing 
upon the characteristics of the projects they observed, Yin and Moore 
formulated a set of specific guidelines for principal investigators to follow 
in order to promote interaction with users.  In normative or prescriptive 
terms, at least:

Investigators should join and become active in associations and other 
organizations to which both knowledge producers and knowledge users belong.

During the design of a new research project, investigators should identify the 
specific groups that may use the results of the project.

In the course of a research project, investigators should be sensitive to the 
ways in which the research design might be modified, without compromising the 
integrity or the quality of the work, to meet the needs of the potential user 
groups.

Investigators should plan on producing a major product, aimed directly at user 
groups, rather than only at research groups.

     Any one of these actions on the part of research-funding organizations, 
the authors observed is likely to increase the chances of utilization.

     There are analogous recommendations from research in the field of 
organizational sociology.  At the end of their synthesis piece, Beyer and Trice 
(1982) postulate that, if researchers genuinely want their work to be used 
outside the scientific community itself, they should be operating as their own 
best advocates, for example, by placing research findings in magazines read by 
non-researchers, as well as in professional journals.  The authors also 
recommend that researchers engage in long-term efforts of consulting and 
training for the levels of management appropriate to the subjects under study.

The Respective Worlds of Researchers and Users

     Sustained interactivity has its limits.  For one thing, it assumes that 
the researchers will communicate directly with the users, which means that the 
level of sophistication in the dissemination process is assured.  This can not 
be taken for granted.  One of the reasons, in fact, that intermediaries are 
used so often is precisely because the will, the competence or the requisite 
time for interactivity is wanting on the researchers' end, the users' end, or 
both. Typically, researchers are given the responsibility for dissemination, 
but allowed -- sometimes encouraged -- to hire out intermediaries to actually 
carry the dissemination effort beyond the scientific community.  This, too, has 
its problems.  As noted above, the more links added to the formula, the more 
uncertain the result.  Also, intermediaries typically have lower substantive 
mastery of the research they are disseminating, but often enjoy high 
credibility among user groups.  This can result in simplified or even in 
distorted dissemination.

     Let us explore for a moment the universes of researchers and users, using 
a study in mathematics.  Cobb et al (1990) have described a research program 
supported by the National Science Foundation and designed to match a 
constructivist view of learning mathematics with an analysis of children's 
mathematics instruction in a public school setting.  In one of the key studies, 
researchers (Steffe and Cobbe, 1983; Steffe, 1983) designed the extension of a 
"constructivist teaching experiment" to the classroom.  The transfer of 
research-based models to school professionals was abandoned when it became 
clear that researchers interpreted the responses of teachers as resistance, 
while for the teachers, there was the perception that this so-called 
"unwillingness to adopt" reflected an attempt to make sense of the approach in 
light of the ensuing changes in classroom interactions in class, the sequencing 
of units and the constraints of time and pupil management.

     In concluding, the authors direct attention to the variant modes of 
reasoning and the different evidential criteria used respectively by 
researchers and teachers.  In particular, they contrast teachers' context of 
"pragmatic pedagogical problem-solving,"  characterized by the necessity to 
make on-the-spot-decisions in specific, but often uncertain situations, with 
the more formalized, abstract universe of cognition in which researchers 
operate.  This is a question also treated at some length by Huberman (1983) in 
his text on the school environment, "Recipes for Busy Kitchens."  Here, too, a 
situational analysis of everyday life in the classroom yields an understanding 
of the "press" of classroom life (simultaneity, immediacy, personal 
involvement, etc.) and a map of the forms in which information and expertise 
need to be shaped in particular forms in order for research-based and 
craft-validated knowledge to be used routinely.

     The "coda" in the Cobb et al study merits some attention.  The authors 
spend some time on the power-authority relationships between researchers and 
teachers.  Their study brings to light the implicit asymmetry between role 
incumbents of different status, in this case, researchers and teachers (but in 
other cases intermediary specialists and teachers). One set has expert 
"authority" and the other set is meant is fall in line with "the right way" 
indicated by the extrapolations from research findings.  The process is a 
subtle but insidious one; many teachers actively seek out the prescribed 
pedagogical activity, and discount reflections on their prior experience and 
understanding.

     In this research program, the authors report that, gradually, researchers 
and teachers "mutually constructed" a social context, and researchers gradually 
revised their view that teachers could be readily transformed into social 
constructivists, neatly aligned with the researchers' epistemology.  At the 
end, the goal was to work with teachers in the development of several forms of 
practice that would improve the quality of pupils' mathematical education, 
without insisting on an epistemological alignment.


Concluding Remarks:  The Sweep of D&U

     We have reviewed the literature on D&U in several fields, including 
physical sciences, architecture, waste management, and organizational 
management.  The best fruit of those synthesis, perhaps, for those working in 
this arena, has been the list of factors in the section on "promoting 
utilization:" availability, relevance, quality, redundancy, linkage, 
engagement, interactivity.

     We have also insisted on the importance of a social constructivist 
approach to dissemination and use of knowledge.  In effect, much earlier work 
simply assumed that intermediaries and users were relatively passive "targets" 
to whom information and expertise could be transferred faithfully.  We know 
better.  For one thing, work on organizational life has shown clearly that, 
within any given social setting, there are a sufficient number of tensions, 
differences in perception, differences in influence or authority, etc. to 
preclude any "straightforward" communication of information or innovation.  
Also, a constructivist view of knowledge use shows us that users must transform 
inputs simply to apprehend them, even if they are as unaware of the process as 
was the teacher in the Cobb & Steffe study.  When we look at "outcomes," then, 
we must assume that users have reconfigured their understanding and use of a 
given practice simply to integrate it into their repertoire.

     There may be one final point to make about the infrastructure of 
dissemination and utilization of knowledge.  Up to now, we have focused our 
attention on the adoption and use of discrete programs or practices.  There is 
a school of thought that holds that these mechanisms need to be buffered by 
tighter links between researchers and users.  The stronger the linkage between 
these two universes, it is claimed, the easier it will be to run new research 
along this conduit, which would gradually comprise more people at different 
levels both in the practitioner and researcher communities.  In a European 
study, Huberman (1990) showed that fairly intensive dissemination among 
researchers and users could lead to closer ties between the two organizations, 
whereas weaker dissemination left few traces of any relationship at all.  
Figure 3 shows the initial relationships between a research institute and two 
"target publics" (guidance and counseling centers).  One sees that the links 
are weak ones, and that they tend to be restricted for the most part to senior 
administrators.  Figure 4 shows the relationship 18 months after the study.  
One sees the increase for one of the target publics --in which interactivity 
was the greater -- not only in the strength and number of ties, but also in the 
hierarchical levels and the nature of the new interactions beyond a conduit 
exclusively for research.

   For others, (e.g., Sarason et al., 1977) institutional ties can be more 
burdensome than networks between individuals and units which are physically 
remote.  Some of the same ties can be forged, it is claimed, at far less cost 
and effort.  True, in-person contacts would need to be provided for, if the aim 
were to go beyond information, exchange and simple coordinating experiments, 
e.g., conduct of an experiment in real time by a subset of people and units in 
the network.  If one considers the possibilities at present with 
telecommunications, the idea of a "network" linking intermediaries, users and 
researchers is a plausible option.  It might, however, need multiple forms of 
contact (telephone, mail, meeting to 'launch' a project or to include one) 
along with the kinds most often associated with telecommunications (video 
conferencing, electronic mail, access to and discussions around remote 
libraries or data bases).

     This leaves us with a plurality of options:  the dissemination of 
knowledge, of products, of research findings, of full-scale science/math 
projects.  This knowledge, as we saw, can be packaged and disseminated in 
multiple ways.  Then, too, there are instances in which we are trying at the 
same time to disseminate knowledge and strengthen inter-institutional links or 
informal networks.  In other words, things can get complex when we cross all 
the questions: what are we disseminating?  to whom it is going?  what effects 
are we after?  how might we do it?  These questions can be answered -- they 
will probably need to be answered before a dissemination plan can be drawn up.

     The research reviewed above has provided a filter for answering all of 
these questions and, in particular, the last one:  how to do it?  Here, the 
most useful contribution in this text might be the six factors which appear to 
be the most predictive for successful implementation in general:  
accessibility, relevance, quality, redundancy of messages, linkage, and 
engagement.

     We would need, however, to determine whether these variables play out 
differently in more extensive ("spread") or intensive "(implementation") 
scenarios.  Just as important, we need to be clearer about the shifts in 
perspective and strategy from a more "engineered" design for dissemination to a 
more constructivist one, and the programmatic implications.  It may well be 
that many of explanatory factors in the "D & U" literature were derived from a 
perspective that has now given way to a very different one.  We don't know 
whether the two perspectives are conciliable or, if they are not, what a set of 
key components in a constructivist dissemination plan might look like.






















REFERENCES



Arzi, H.J., R.T. White, and Fensham, P.J.  "Teachers' Knowledge of Science:  An 
Account of a Longitudinal Study in Progress".  Paper presented at the annual 
meeting of the American Educational Research Association, Washington, D.C, 
(quoted in Tobin and Espinet, 1989), 1987.

Anderson, R.S., et al.  "Review of the Literature Pertaining to Curriculum 
Reform in Science, Mathematics and Higher-Order Thinking Across the 
Disciplines", The Curriculum Reform Project, The University of Colorado, 1992.

Anderson, R.D., et al.  "Selecting interventions for improving science 
education".  In Bybee, R.W., Carlson, J. and McCormack, A.J. (eds.), NSTA 
Yearbook:  Redesigning Science and Technology Education.  Washington, DC:  
National Science Teachers Association, 1984.

Berman, P., and McLaughlin, M.  Federal Programs Supporting Educational Change. 
 Vol.3. Santa Monica: Rand Corporation, 1978.

Beyer, M., and Trice, H.  "The utilization process:  A conceptual framework and 
synthesis of empirical findings." Administrative Science Quarterly, 27:597-622, 
1982.

Carlson, R.  Adoption of Educational Innovations.  Eugene:  University of 
Oregon Press, 1965.

Cohen, D.K., and Ball, D.L.  "Relations between policy and practice:  A 
commentary."  Educational Evaluation and Policy Analysis, 12:3, 249-256, 1990.

Confrey, J.  "What constructivism implies for teaching."  In Davis, R., Maher 
C. and Noddings, N. (eds.), Constructivist Views on the Teaching and Learning 
of Mathematics.  Journal for Research in Mathematics Education, Monograph 
No.4:107-122.  Reston, VA:  National Council of Teachers of Mathematics, 1990.

Cox. P.L., Kahn, K.A., and French, L.C.  "A study of clients of informational 
service providers and their use of ERIC-based resources and services."  Making 
the Match for Use of Educational Information:  A Multimethodological Study of 
Providers and Users of ERIC-based Resources and Services, (vol. 3. June, 1985). 
 Andover, MA:  The NETWORK, Inc., 1985.

Crandall, D.P.  "School improvement:  What the research says."  Keynote address 
presented at the April 1984 conference on School Excellence, sponsored by the 
Appalachia Educational Laboratory, 1984.

Crandall, D.P. et al.  People, Policies and Practices:  Examining the Chain of 
School Improvement, (vols. I-X).  Andover, MA:  The NETWORK, Inc., (quoted in 
Louis and Dentler, 1988), 1983.

Crandall, D.P. et al.  A Study of Dissemination Efforts Supporting School 
Improvement, (vols. 3-4).  Andover, MA:  The NETWORK, Inc, 1982.

Cuban, L.  "A fundamental puzzle of school reform.", 71-77.  In Lieberman, A. 
(ed.), Schools as Collaborative Cultures:  Creating the Future Now.  New York:  
The Falmer Press, 1990.

Darling-Hammond, L.  "Instructional policy into practice: `The power of the 
bottom over the top'". Educational Evaluation and Policy Analysis, 12:3, 
233-241, 1990. Darling-Hammond, L. and Berry, B.  The Evolution of Teacher 
Policy.  Santa Monica, CA:  RAND Corporation.  (quoted in Darling- Hammond, 
1990.), 1988.

Davis, H.R. and Salasin, S.E.  "Strengthening the contribution of social R&D to 
policy making."  In Lynn, Jr., L.E. (ed.), Knowledge and Policy:  The Uncertain 
Connection.  Washington, D.C.:   National Academy of Sciences, 1978.

Davis, R.B.  "Discovery learning and constructivism."  In Davis, R., Maher, C. 
and Noddings, N. (eds), Constructivist Views on the Teaching and Learning of 
Mathematics:  Journal for Research in Mathematics Education, Monograph No. 
4:93-106.  Reston, VA:  National Council of Teachers of Mathematics, 1990.

Davis, R., Maher, C. and Noddings, N. (eds.),  Constructivist Views on the 
Teaching and Learning of Mathematics:  Journal for Research in Mathematics 
Education, Reston, VA:  National Council of Teachers of Mathematics, 1990.

Driver, R.  The Pupil as Scientist?,  Milton Keynes, England:  The Open 
University Press, 1983.

Dunn, W.N. and Holzner, B.  "Knowledge in society:  Anatomy of an emergent 
field."  Knowledge in Society, (Spring 1988), 3-26, 1988.

Duschl, R.A.  Restructuring Science Education:  The Important of Theories and 
Their Development,  New York:  Teachers College Press, Columbia University, 
1990.

Emrick, J. and Peterson, S.  A Synthesis of Findings Across Five Recent Studies 
in Educational Dissemination and Change.  San Francisco:  Far West Laboratory 
for Educational Research and Development, 1978.

Fullan, M. and Pomfret, A.  "Research on curriculum and instruction 
implementation."  Review of Educational Research, 47, 1:335-397, 1977.

Fullan, J.G. and Steigelbauer, S.  The New Meaning of Educational Change,  New 
York:   Teachers College Press, Columbia University, 1991.

Guba, E.  "The process of educational innovation."  In Goulet, R. (ed.), 
Educational Change. New York: Citation Press. 136-153, 1968.

Hall, G.E. and Hord, S.M.  Change in Schools:  Facilitating the Process.  
Albany:  State University of New York Press, 1987.

Hall, G. and Loucks, S.  "Teacher concerns as a basis for facilitating and 
personalizing staff development." Teachers College Record, 80(1):36-53, 1978.

Hargreaves, A.  "Experience counts, theory doesn't:  How teachers talk about 
their work."  Sociology of Education, 57:244-254.  (quoted in Louis and 
Dentler, 1988.), 1984.

Havelock, R.  Planning for Innovation Through Dissemination and Utilization of 
Knowledge.  Ann Arbor, MI: Institute for Social Research, 1969.

Havelock, R.G.  Planning for Innovation through Dissemination and utilization 
of Knowledge,  Ann Arbor: University of Michigan Center for Research on 
Utilization of Scientific Knowledge, Institute for Social Research, 1973.

Havelock, R.G.  "The Utilization of Educational Research and Development."  
British Journal of Educational Technology, 2(2, 1971):  84-98, 1971.

Hood, Paul D.  "How can studies of information consumers be used to improve the 
education communication systems?"  Presentation at the 1989 Annual Meeting of 
the American Education Research Association in San Francisco, 1989.

Hord, S.M.  "Facilitating change in secondary schools -- myths and management." 
 NASSP Bulletin, l73 (516): 68-73, 1989.

Horst, D.P. et al.  Evaluation of the Field Test of Project Information 
Packages, Volume II:  Recommendations for Revisions,  Los Altos, CA:  RMC 
Research Corp., and Menlo Park, CA:  Stanford Research Institute, 1975.

Huberman, M.  "Experiments and innovation in education."  Understanding Change 
in Education:  An Introduction, Paris:  The Unesco Press, 1973.

Huberman, M.  "Linkage between Researchers and Practitioners, a Qualitative 
Study."  American Education Research Journal, 27(2):  363-391, 1990.

Huberman, M.  "Experiments and innovation in education".  Understanding Change 
in Education:  An Introduction.  Paris:  The Unesco Press, 1973.

Huberman, M.  "Recipes for busy kitchens:  A situational analysis of routine 
knowledge use in schools." Knowledge, 4(4):478-510, 1983.

Huberman, M.  "What knowledge is of most worth to teachers:  A knowledge use 
perspective."  Paper presented at the American Educational Research Association 
Meetings, April, (quoted in Louis and Dentler, 1988), 1985.

Huberman, M.  "Steps toward an integrated model of knowledge utilization."  
Knowledge, 18, 4:586-611, 1987.

Huberman, M.  "Predictors of conceptual effects in research utilization."  
Knowledge in Society, 2:3:6-24, 1989.

Huberman, M.  "Linkage between researchers and practitioners:  A qualitative 
study."  American Educational Research Journal, 27, 2:363-391, 1990.

Huberman, M., and Crandall, D.P.  People, Policies, and Practices:  Examining 
the Chain of School Improvement, Vol. IX:  Implications for Action.  Andover, 
MA:  The NETWORK, Inc, 1982.

Huberman, M., and Miles, M.  Innovation Up Close:  How School Improvement 
Works. New York:  Plenum.

Klein, S.S. and Gwaltney, M.K.  "Charting the Education Dissemination System:  
Where we are and where we go from here."  Knowledge:  Creation, Diffusion, 
Utilization, 12(3): 241-265, 1991.

Lieberman, A. and Miller, L. (eds.),  Staff Development for Education in the 
90s:  New Demands, New Realities, New Perspectives,  New York:  Teachers 
College Press, Columbia University, 1991. Lieberman, A. (ed.),  Schools as 
Collaborative Cultures:  Creating the Future Now.  New York:  The Falmer Press, 
1990. Loucks, S.  Planning for Dissemination.  Chapel Hill:  TADS, 1983.

Loucks-Horsley, S. and Stiegelbauer, S.  "Using knowledge of change to guide 
staff development.  In Lieberman, A. and Miller, L. (eds.),  Staff Development 
for Education in the '90s:  New Demands, New Realities, New Perspectives.  New 
York:  Teachers College, Columbia University.

Louis, K.S. and Dentler, R.A.  "Knowledge use and school improvement.  
Curriculum Inquiry, 18:1, 34-62. The Ontario Institute for Studies in 
Education.  New York:  John Wiley & Sons, Inc., 1988.

Louis, K.S., Dentler, R.A. and Kell, D.G.  "Putting knowledge to work, issues 
in education dissemination." Final report, NIE, AAI. No. 85-6. Cambridge, MA:  
Abt Associates, 1984.

Louis, K.S., Rosenblum, S., Molitor, J.A. and others.  Strategies for Knowledge 
Use and School Improvement. Washington, DC:  National Institute of Education, 
1981.

Louis, K.S., and Rosenblum, S.  Designing and Managing Interorganizational 
Networks.  Cambridge, MA: Abt Associates, Inc., 1981.

Louis, K.S.  The Role of External Agents in Knowledge Utilization, Problem 
Solving and Implementation of New Programs in Local School Contexts,  San 
Francisco:  Far West Lab for Educational Research and Development, (ERIC 
ED203462), 1980.

Lynn, L.E., Jr. (ed.),  Knowledge and Policy:  The Uncertain Connection.  
Washington, D.C.:  National Academy of Sciences, 1978.

Lythcott. J. and Duschl, R.  "Qualitative research:  From methods to 
conclusions."  Science Education, 74(4): 445-460, 1990.

Maher, C. and A. Alston.  "Teacher development in mathematics in a 
constructivist framework".  In Davis, R.B., Maher, C. and Noddings, N. (eds.),  
Constructivist Views on the Teaching and Learning of Mathematics: Journal for 
Research in Mathematics Education.  Monograph No.4:147-165.  Reston, VA:  
National Council of Teachers of Mathematics, 1990.

Mann, D.  "The user driven systems and a modest proposal."  In Making Change 
Happen, Mann, D. (ed.), 285-307.  New York:  Teachers College Press, (quoted in 
Louis and Dentler, 1988), 1978.

McClure, R.M.  "Individual growth and institutional renewal."  In Lieberman, A. 
and Miller, L. (eds.),  Staff Development for Education in the '90s:  New 
Demands, New Realities, New Perspectives, 221-241.  New York:  Teachers 
College, Columbia University, 1991.

Menlo, A.   "A reconceptualization of resistance to change and its application 
to the institutionalization process."  Paper presented to the International 
School Improvement Project, Lucerne, Switzerland, June. (quoted in Louis and 
Dentler, 1988), 1985.

Miles, M.  "Unraveling the mystery of institutionalization."  Education 
Leadership, (vol. 41, no. 3, November):14-19, 1983.

Miles M., and Huberman, M.  "Qualitative data analysis."  Knowledge.  Beverly 
Hills, CA:  Sage, 1984.

Mort, P.  "Studies in educational innovation from the Institute of 
Administrative Research."  In Miles, M. (ed.), Innovation in Education.  New 
York:  Teachers College Press, 1964. Muthard, J. and Felice, K.  "Assessing 
activities for utilization of rehabilitation research."  Knowledge, 
4(2):309-328.  Sage Publications, Inc., 1982.

Murphy, S.  "Problems in research utilization:  A review."  Rehabilitation 
Counseling Bull, 18(4):365-376, 1975.

Nelson, M. and Sieber, S.  "Innovation in urban secondary schools."  School 
Review, 27:101-119, (quoted in Louis and Dentler 1988), 1976.

Pines, A.L. and West, L.H.  "Conceptual understanding and science learning:  An 
interpretation of research within a sources-of-knowledge framework."  Science 
Education, 70(5):583-604, 1986. Rogers, E, and Shoemaker, F.F.  Communications 
of Innovations.  New York:  Free Press, 1971.

Rogers, E.  Diffusion of Innovations.  New York:  Macmillan, 1962.

Rogers, E.  "The intellectual foundation and history of the agricultural 
extension model."  Knowledge, 9(4):492-510, 1988.

Rogers, E.M. and Kincaid, D.L.  Communication Networks:  Toward a New Paradigm 
for Research,  New York:  Free Press, 1981.

Sarason, S.  The Culture of the School and the Problem of Change, Boston:  
Allyn & Bacon, 1971. Sarason, S., Carroll, D., Maton, K., Cohen, S. and 
Lorentz, E.  Human Services and Resource Networks, San Francisco:  Jossey-Bass, 
1977.

Schlechty, P.C.  Schools for the Twenty-First Century, San Francisco:  
Jossey-Bass Publishers, 1990.

Schuell, T.  "Cognitive conceptions of learning".  Review of Educational 
Research, 56:  411-436, 1986.

Sieber, S.D.  "Knowledge utilization in public education:  Incentives and 
disincentives."  In Lehming, R. and Kane, M. (eds.), Improving Schools:  Using 
What we Know, 115-167, Beverly Hills, CA:  Sage, (quoted in Louis and Dentler, 
1988), 1981.

Sieber, S., Louis, K. and Mertzger, L.  The Use of Educational Knowledge:  
Evaluation of the Pilot State Dissemination Program, (vols. 1 & 2), New York:  
Columbia University, Bureau of Applied Social Research, 1972.

Steffe, L.P.  "The teaching experiment methodology in a constructivist research 
program.",  Zweng, M., Green, T., Kilpatrick, J., Pollak, H. and Suydam, M. 
(eds.),  Proceedings of the Fourth International Congress on Mathematical 
Education,  Boston:  Birkhauser, 469-471, 1983.

Sunesson, S. and Nilsson, K.  "Intervening factors in the utilization of social 
research."  Knowledge in Society: The International Journal of Knowledge 
Transfer, 2:1, 42-56, 1989.

Sykes, G.  "Teaching incentives:  Constraint and variety."  In A. Lieberman 
(ed.),  Schools as Collaborative Cultures:  Creating the Future Now, 103-125.  
New York:  The Falmer Press.

Tobin, K. and Espinet, M.  "Impediments to change:  Applications of coaching in 
high school science teaching."  Journal of Research in Science Teaching, 26:2, 
105-120.  National Association for Research in Science Teaching, John Wiley & 
Sons, Inc., 1989.

Tobin, K.  "Teaching for high-level cognitive learning in high school science?" 
 Paper presented at the annual meeting of the National Association for Research 
in Science Teaching, Washington, D.C., 1987.

Weiss, C.H.  "Research for policy's sake:  The enlightenment function of social 
research."  Policy Analysis, 3, 531-545, 1977a.

Weiss, C.H. with Bucuvalas, M.J.  Social Science Research and Decision-Making, 
New York:  Columbia University Press, 1980.

Weiss, C.H.  "The circuity of enlightenment:  Diffusion of social science 
research to policy makers." Knowledge:  Creation, Diffusion, Utilization, 
8(2):274-281, 1986.

Wheatley, G.J.  "Contructivist perspectives on science and mathematics 
learning,"  Science Education, 75(1): 9-21, 1991.

Yin, R. and Moore, G.  The Utilization of Research:  Lessons for the Natural 
Hazards Field, Washington, DC: Cosmos Corp, 1985.

Yin, R.K. with Quick, S.K., Bateman, P.M., and Marks, E.L.  Changing Urban 
Bureaucracies:  How New Practices Become Routinized, Santa Monica, CA:  Rand 
Corporation; and 1979. Lexington, MA:  D.C. Heath, 1978.

Zaltman, G.  Theory-in-use Among Change Agents in Handbook of Community and 
Social Intervention, Seidman, E. (ed.),  Beverly Hills:  Sage Publications, 
1983.

Zaltman, G., Duncan, R., and Holbek, J.  Innovations and Organizations, New 
York:  Wiley, 1973.







































































The National Science Foundation has TDD (Telephonic Device for the Deaf) 
capability which enables individuals with hearing impairments to communicate 
with the Divison of Personnel and Management for information relating to NSF 
programs, employment, or for general information.  This number is (202) 
357-7492.
{page|1}