Beverly Woolf
Department of Computer Science
University of Massachusetts, Amherst

Pamela Kurstedt
Assistant Dean of Engineering
Virginia Polytechnic and State University

Matt Ohland
Assistant Director
SUCCEED Coalition
University of Florida

Sam Awoniyi
Assistant Dean of Engineering
Florida State University/Florida A&M University

Workshop chair Beverly Woolf explained the objective of the session: to identify potential best practices in developing and implementing new multimedia courseware. It was agreed that, given the participation in the large workshop of many novices as well as experts, the groups identification of best practices as a whole should be considered provisional, if in some instances well-grounded.

Seven topic areas were amplified and then discussed; for each, the suggestions for best practices in that area are summarized below. No consensus was sought or reached by the workshop participants on which of the identified best practices appear most valuable.

1.     Best Practices in Multimedia Development. How should the curriculum be assessed and knowledge gathered? How many kinds of expertise are needed on a development team (e.g., subject matter experts, instructional technologists, multimedia specialists)?

The material selected for multimedia development should lend itself well to that format. Multimedias special characteristics should be exploited, rather than simply presenting multimedia material in a way that models textbooks. Students should interact with the program, which should be student-driven and aimed at accommodating different students' capabilities as appropriate and feasible. Research has shown that interactivity -- in the form of "intelligent tutors" -- can reduce time of instruction and errors in learning.

Tasks particularly suited to multimedia presentation include role-playing, concept acquisition, and visualization-based skills. A large range of products could be called "multimedia"; but, regardless of its form, multimedia material should be interactive and student-centered.

To develop multimedia products successfully, a variety of expertise is required -- although, of course, such a range may be found in one person. These skills include instructional design, programming, writing, graphic art, subject matter, funding development, and perhaps project management as well. Without sufficient grounding in such expertise, multimedia products are very likely to be marginal. Expert advice can sometimes be had through the university's center for educational technology or outsourcing.

Multimedia tools should be tested over the course of their development. This will permit units to be restructured and other needed changes to be made in a timely way.

Several major obstacles stand in the way of multimedia courseware development. Beyond the reluctance of universities to transform their educational settings to accommodate such materials, educators have few incentives to develop multimedia products. University support, in the form of technical and copyright expertise and motivation of faculty through tenure and other rewards, would be very useful in furthering the development of such courseware. Mechanisms are also needed to provide reimbursement for creative efforts accessed through the World Wide Web and to recover development costs. In particular, faculty want to limit the distribution of their personal instruction materials.

2.     Best Practices in Selecting Multimedia Tools. What are the pros and cons of available software? How can we move beyond current forms of presentation?

Developers need to identify the kind of media elements they want to use, to select the optimal programming software. Those interested in developing multimedia courseware should begin by seeking expert advice and sampling existing programs -- including those on the university campus -- and by networking with interest groups. They should attend conferences and arrange for demonstrations to acquire further competence.

Table 1 identifies various multimedia elements and some of the software tools that are most useful for producing and presenting them.

Table 1: Multimedia Elements and the Software Packages Best at Producing Them
Element Tool
Simulation (mathematical) Many, including Extend, Power Symbol, C++
3-D graphics 3-D Studio, Director, Visual Basic,Tri-Spe
Animation Director, Visual Basic
Audio Sound Edit
Video Premier
Hypertext Authorware, Word 97
Real-time interaction (collaboration) Visual Basic, C++
Object-oriented interactivity mTropolis
Internet Java

Each programming language also has its drawbacks (e.g., Authorware and mTropolis require significant programming skills). Web browsers can also be used as a shell. Their advantages include platform-independence, lack of cost, and pervasiveness; their drawbacks are their primitiveness and slowness. Potential developers should further explore such software specifics, both pros and cons, by consulting the experts.

Regardless of the software chosen for development, it should produce multimedia tools that are easy to use, provide immediate feedback, and track users (e.g., via ties to central computers). The programs should capture students' work. Program tasks should also be designed to require or at least encourage communication and collaboration -- for example, via teaming, the sharing of terminals, and e-mail.

Multimedia tools can serve a variety of clientele, providing both regular courses and distance learning. They can be used as entire courses or supporting material.

In any application, multimedia courseware changes the structure of education and the role of the educator. From the educator's perspective, a variety of tasks can be offloaded: rote learning, frequently asked questions, the provision of much background material, grading, recordkeeping, and lecture/slide and lecture/audio instruction. The use of multimedia software opens up the classroom to creative instruction and applications, integration of knowledge, and expanded student-teacher and even wider communication (as via the Internet). Outside the classroom, by means of e-mail, the student can get a fast response from the instructor on any question of interest. The instructor becomes coach and motivator, rather than authority, in a setting that can address real-world problems. Thus, while multimedia tools can fundamentally restructure the educational process, they cannot replace the functions of good teachers.

Clearly, the role of the learner in this educational process has changed as well. Today, students must acquire a new category of skills. From the flood of available information, they must know how to extract data, recognize problems, seek solutions through good search strategies, and synthesize information meaningfully. They must know how to work in integrated teams. Above all, they must learn how to learn, in order to continue acquiring the new skills demanded in a fast-evolving culture.

3.     Best Practices in Multimedia Implementation. How should interfaces, controls, and student interaction be implemented? How should Web-based systems be implemented? How should individual components be implemented?

In implementing multimedia products, educators should test, emulate, and adapt from other quality products. As in developing multimedia courseware, they should exploit all available resources efficiently.

Specific issues for multimedia implementation include ease of use and flexibility of individual pacing, and the use of tiered material structure, to reach students of all levels and backgrounds. Again, active student participation in addressing real-world problems should be the overarching goal. Actual problems may be presented, for example, by means of video loops. The use of a game approach (e.g, with sound effects) can increase the entertainment value of programs and enhance learning.

4.     Best Practices in Assessment and Evaluation. How do we know if systems are effective? Are students really learning better and more with multimedia? How is learning enhanced by multimedia? What are documented success stories?

The current paradigm of the academy is not a multimedia approach. Well-established best practices or benchmarks to assess multimedia courseware are therefore lacking. How is the success of such courseware to be judged -- by controlled experiment, the market, or still other means? Until a paradigm shift occurs, the notion of assessment might be simplified in this context. Thus, if a multimedia program has a mechanism for the user to self-test, and if the program supplements course material and complements the text, it might at present be judged an "effective package."

A few well-grounded guidelines can be followed in the meantime: establish a learning baseline (pretest), present small (self-contained) learning modules, require activity on the part of the student, provide regular feedback, and get students' perspectives on the program.

New programs need to be designed to collect data on learner errors and other information of use for program evaluation. Developers can also begin to collaborate with professional evaluators toward establishing good assessment mechanisms for the new multimedia courseware.

Finally, in evaluating multimedia courseware, an important point should be kept in mind: the multimedia approach itself inherently enhances specific learning skills (e.g., in visualization and communication), thereby affecting the very goals of the learning process.

5.     Best Practices in Standardization, Organization, and Dissemination. What standardized formats are best for collaborative development and dissemination? How can these systems be catalogued and peer reviewed? What efforts are needed to keep systems working, current, and revised?

Because multimedia approaches are so new, there is still no clear choice of standards for their form. Potential benefits of standardization include ease of dissemination, collaboration, customization, and adoption, while potential drawbacks include making the wrong choices and limiting development horizons unnecessarily.

Several practices might encourage standardization of a desirable sort: a peer-reviewed archival database, on the model of a research publication; NSF cross-coalition work; and attempts at revising engineering curricula such as those seen recently in the coalitions and elsewhere. Dissemination in general facilitates standardization, as in the form of the National Engineering Education Delivery System (NEEDS) being developed by the Synthesis Coalition or by providing free, limited versions of material by means of the Web. Such dissemination can also be aided by peer-reviewed system testing at beta sites.

6.     Best Practices in Enabling Adoption of Multimedia. What can be done to enable adoption of more effective new technologies? What are the barriers?

The adoption of multimedia programs faces several obstacles: lack of related funding; the general reluctance to move away from textbooks, including students' desire for text (though, again, multimedia programs can be used as a supplement to, rather than as a replacement for, a traditional course). Also needed to encourage multimedia courseware adoption are the more widespread availability and use of laptops, PCs and printers, and Internet and e-mail links. Program developers themselves need to be sold on the idea of multimedia courseware; outreach must be made to content experts, engineering educators, and universities. A faculty champion and/or other university area specialist can strongly encourage use of multimedia. Universities can be given a financial stake in developing such programs -- although faculty roles might need to evolve in this arrangement, since textbooks traditionally belong to the author. Universities might also help potential authors clarify copyright issues. Again, multimedia developments could be established as one basis for faculty advancement. Finally, students and educators both could be provided opportunities to become familiar with the new technology through workshops and presentations, encouraged by new seed money.

Other practices to facilitate multimedia adoption include improving the program's platform flexibility, such as through a basis in Java or standard low-level language, increasing the capability to be adapted for individualized use (e.g., via modules), and making basic software upgrades.

Adoption of multimedia courseware offers educators several strong advantages. It can make life easier by: reducing administrative work, providing normal course materials, and offering such special features as Web-based interaction with students to provide them with expert feedback quickly.

7.     Best Practices in Licensing Multimedia Material. How can you work with the university to license your multimedia? How can you work with publishers and venture capitalists? How can you negotiate fair terms to commercialize the software?

Intellectual property rights in the area of multimedia developments are currently both complex and evolving. Good practices appear to include licensing as narrowly as possible (e.g., over markets or time periods); establishing small companies (C or S corporations) for liability protection; encouraging publishers to compete for material; and involving the universitys research or patent office. Copyright law itself is short, easy to read, and helpful to consult. Because multi-institutional multimedia programs have already been developed, a publication (or CD-ROM) covering this area, with input from all types of parties involved, would also be useful. Copyright now belongs to the employer only in the case of "work for hire"; thus, in this area, potential developers should do their homework before embarking on new multimedia projects. Similarly, great care must be taken with publishers regarding any rights the author wishes to retain.


In her summary of the workshop to the conference attendees in plenary session, Dr. Woolf took an integrative, cross-cutting approach to formulating "some radical suggestions" for changes needed in the university and college structure in order to facilitate the introduction of multimedia courseware.

  1. To improve the implementation of multimedia, the role of faculty will need to be redefined to allow more creativity in teaching; one-on-one tutoring, coaching, and mentoring; and to permit faculty to off-load rote learning aspects of education. The university itself will need to be restructured to allow faculty to be rewarded for development and implementation of multimedia, to provide more labs, and possibly to permit the partial elimination of books and lectures.

  2. To facilitate the dissemination of multimedia, a paradigm shift in education will be necessary. Also needed will be peer review and archiving of courseware.

  3. Evaluation of multimedia courseware must take into account the multiple teaching methods available and the fact that new student skills in browsing/searching, communication, and collaboration are available under the multimedia approach.

  4. With regard to standardization of multimedia courseware, it is important to standardize objects, not packages. Digital resources must be made more widely available, and development should not be constrained.

  5. In terms of copyright and licensing, it is necessary to realize that the university might not release a multimedia product for public use. Also, liability is an open question.