This project investigated how augmented reality (AR) can best help workers in manufacturing develop agility and adaptability on the shop floor, while avoiding the risks associated with dependence on technology and stifled innovation. The three frontiers of the Future of Work at the Human-Technology Frontier were explored. (1) Future work: Labor market analyses were conducted to illuminate the changes in employer skill requirements for specific job roles in manufacturing to understand the degree to which AR technologies have been introduced in the U.S. and the types of skills workers will need in the future manufacturing workforce. (2) Future technology: A prototype intelligent AR technology was ideated and designed to interpret and guide the behavior of AR-supported manufacturing workers through adaptive scaffolding of instructions to the expertise level of individual workers, which is also anticipated to inform the design of future AR technologies. (3) Future worker: Hypothesis-driven human subject studies were performed using the AR training apps to understand the impacts of adaptive AR scaffolds on the ability of workers to learn new tasks in a way that enhances their flexibility in transferring their knowledge to new situations.

The project researched the potential for the future of advanced manufacturing work to be positively influenced by AR technology through lifelong experiential training that enables rapid and continuous upskilling of workers. This project discovered that even in large manufacturing firms, the recent trend is to enhance/augment/improve worker performance and safety rather than reduce headcount. At the same time, firms are changing the composition of skills they seek in current and new hires. Preliminary results based on a proprietary database of 160 million online job vacancies aggregated by Burning Glass Technologies indicated that firms have increasingly sought technicians and engineers with some postsecondary education beyond a high school degree to facilitate the automation of some tasks. Moreover, as manufacturing in the U.S. has continued to shift towards more high-precision production, there is a greater need for quality and inspection that has changed the types of skills demanded of production workers needed to fill critical advanced manufacturing positions such as CNC programmers, avionics technicians, civil designers, and production supervisors. 

An AR training app was developed in Unity using Microsoft Mixed-Reality Toolkit and deployed to HoloLens 2 headsets for conducting the planned laboratory experiments. The experiments involved the assembly of an engine component using standard mechanical tools. The AR app included expert-capture GoPro videos with vocal cues, textual descriptions and images of parts, and interactive 3D CAD animations. 20 Mechanical and Industrial Engineering students were recruited as participants (6 females, 11 undergrads/9 grads, 2 URMs) and divided into two groups: AR and paper. A questionnaire was used to collect their demographics and related prior experiences to counterbalance the groups. All participants received initial online training on the AR app and hardware (HoloLens). Each participant performed three assembly cycles in separate dates using their designated mode of instruction and returned after a few days to perform a final assembly using the opposite mode of instruction. At the end of each session, the experimenters recorded time to completion, number of errors, frequency of help-seeking behavior, and the types of errors and questions, and the participants reported their cognitive load, self-efficacy, experience with AR headset and app, and general feedback through structured and open-ended questions. Findings revealed that AR reduces the number of errors by 31-84%. Most participants reported absolute independence from AR after two/three cycles, which points to the effectiveness of AR in improving task competency, and yet its low utility as an ?assistive tool? for routine tasks. Further, several participants suggested devising interactive help and voice command systems.

The project team assembled a diverse advisory committee from MassMEP, MassBay Community College, Springfield Tech Community College, Northeastern College of Professional Studies, Aalborg University (Denmark), NIST, Festo, GE Aviation, Burning Glass Technologies, and PTC Inc. to illuminate the potential potentials, adoption barriers, risks of AR for workplace-based learning in manufacturing. Important highlights of the discussions are as follows: (1) AR can potentially be a disruptive assistive technology for manufacturing tasks that are not rote and require complex reasoning (e.g., inspection). (2) The acceptability of AR as a ?tool? is likely to differ among incumbent and future (tech-savvy) workers and different demographics. (3) AR can increase accessibility of manufacturing jobs to workers with different demographic characteristics. (4) AR can create new opportunities for remote learning and assistance from larger, and possibly more diverse, pools of physically/temporally distant coworkers. (5) Successful industry adoption of AR will require rigorous justification through both proof-of-concepts and economic cost-benefit analyses. (6) Scalability must be regarded as a key criterion for the ideation and design of AR technologies.

Publication: Moghaddam, M., Wilson, N.C., Modestino, A.S., Jona, K. and Marsella, S.C., 2021. Exploring augmented reality for worker assistance versus training. Advanced Engineering Informatics, 50, p.101410.

Last Modified: 02/03/2022
Modified by: Mohsen Moghaddam