Award Abstract # 1928506
FW-HTF-RL: Collaborative Research: Shared Autonomy for the Dull, Dirty, and Dangerous: Exploring Division of Labor for Humans and Robots to Transform the Recycling Sorting Industry

NSF Org: CMMI
Division of Civil, Mechanical, and Manufacturing Innovation
Recipient: WORCESTER POLYTECHNIC INSTITUTE
Initial Amendment Date: July 29, 2019
Latest Amendment Date: May 11, 2021
Award Number: 1928506
Award Instrument: Standard Grant
Program Manager: Alexandra Medina-Borja
amedinab@nsf.gov
 (703)292-7557
CMMI
 Division of Civil, Mechanical, and Manufacturing Innovation
ENG
 Directorate for Engineering
Start Date: September 1, 2019
End Date: August 31, 2024 (Estimated)
Total Intended Award Amount: $604,314.00
Total Awarded Amount to Date: $612,314.00
Funds Obligated to Date: FY 2019 = $604,314.00
FY 2021 = $8,000.00
History of Investigator:
  • Berk Calli (Principal Investigator)
    bcalli@wpi.edu
  • Jacob Whitehill (Co-Principal Investigator)
Recipient Sponsored Research Office: Worcester Polytechnic Institute
100 INSTITUTE RD
WORCESTER
MA  US  01609-2280
(508)831-5000
Sponsor Congressional District: 02
Primary Place of Performance: Worcester Polytechnic Institute
Worcester
MA  US  01609-2274
Primary Place of Performance
Congressional District:
02
Unique Entity Identifier (UEI): HJNQME41NBU4
Parent UEI:
NSF Program(s): FW-HTF-Adv Cogn & Phys Capblty,
FW-HTF Futr Wrk Hum-Tech Frntr
Primary Program Source: 01001920DB NSF RESEARCH & RELATED ACTIVIT
01002122DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 063Z, 116E, 9178, 9231, 9251
Program Element Code(s): 082Y00, 103Y00
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.041

ABSTRACT

This Future of Work at the Human-Technology Frontier (FW-HTF) project investigates a novel human-robot collaboration architecture to improve efficiency and profitability in the recycling industry, while re-creating recycling jobs to be safer, cleaner, and more meaningful. The specific goal is to improve the waste sorting process, that is, the separation of mixed waste into plastics, paper, metal, glass, and non-recyclables. The US scrap recycling industry -- which represents $117 billion in annual economic activity and more than 530,000 US jobs -- is struggling to meet increasingly challenging standards in domestic and international markets. A major problem for the industry is poor sorting of waste, resulting in materials impurity and a significant decrease in the quality and value of the recycled product. Human perception and judgement are essential to handle the object variety, clutter level and changing characteristics of the waste stream. Yet waste-sorting workers currently face health risks and discomfort arising from sharp and heavy objects, toxic materials, noise, vibration, dust, noisome odors, and poor heating, ventilation, and air conditioning. The innovative robotics component of this project, especially in object detection, manipulation, and human-robot interaction, will allow new sorting facility architectures, creating new, safer roles for human workers. The project complements these technological advances with economic analyses to determine the facility configurations that best remove processing bottlenecks, target materials of high value, and boost the end-to-end efficiency of the recycling process. Division of labor between humans and robots will be investigated to improve job desirability and worker motivation, incorporating consideration of the workers' well-being. In particular, the project will explore ways to utilize robots to amplify worker expertise and value. A holistic and interconnected research approach will be taken for all these aspects, i.e. developing robotics technology, designing the human-machine interfaces, investigating workers' workers' role in the new sorting plant architectures, and understanding and incorporating workers' needs and well-being into the design process.

This project will develop the appropriate robotics technology for recycling industry deployment, which will require advancing the state of the art in waste classification and manipulation to handle the conditions associated with recycling facilities. Deep Neural Networks-based object detection and semantic segmentation frameworks will be designed for rich, multi-modal sensor data in order to solve challenges regarding a high-level of clutter, occlusion and object variety. Novel robotic manipulation algorithms based on dynamic and soft manipulation strategies will be utilized to separate and pick classified items from the cluttered waste stream. Robust and dexterous robot hardware will be developed, including the robotic arms and end effectors. Human-machine interfaces will be designed and implemented to achieve these tasks in an intuitive, efficient and practical workflow that optimizes the contributions of both human workers and automated technologies. The robotics technology will also allow expanding the facilities from simply sorting the incoming materials into a whole recycling ecosystem; additional process lines for onsite materials processing units will enable conveying partially-finished products to next stage manufacturers. This expansion will require a novel systems approach, and will help achieve more efficient recycling plants and a much more comprehensive employment ladder for current and new workers. These technological and structural changes in the interactional system of work will shift both the task and relational landscape of the work. The effect of these shifts on worker satisfaction and motivation will be investigated via worker interviews with simulated systems. The new technological landscape will be formed accordingly for improved work experience.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 11)
Alladkani, Fadi and Akl, James and Calli, Berk "ECNNs: Ensemble Learning Methods for Improving Planar Grasp Quality Estimation" 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021 https://doi.org/10.1109/ICRA48506.2021.9561038 Citation Details
Bashkirova, Dina and "ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022 https://doi.org/10.1109/CVPR52688.2022.02047 Citation Details
Chatterjee, Sreejani and Doan, Duc and Calli, Berk "Utilizing Inpainting for Training Keypoint Detection Algorithms Towards Markerless Visual Servoing" , 2024 https://doi.org/10.1109/ICRA57147.2024.10610006 Citation Details
Chertow, Marian and Reck, Barbara K and Wrzesniewski, Amy and Calli, Berk "Outlook on the future role of robots and AI in material recovery facilities: Implications for U.S. recycling and the workforce" Journal of Cleaner Production , v.470 , 2024 https://doi.org/10.1016/j.jclepro.2024.143234 Citation Details
Dina Bashkirova, Samarth Mishra "VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting" Proceedings of Machine Learning Research , v.220 , 2022 Citation Details
Enyedy, Albert and Aswale, Ashay and Calli, Berk and Gennert, Michael "Stereo Image-based Visual Servoing Towards Feature-based Grasping" , 2024 https://doi.org/10.1109/ICRA57147.2024.10611604 Citation Details
Gandhi, Abhinav and Chatterjee, Sreejani and Calli, Berk "Skeleton-based Adaptive Visual Servoing for Control of Robotic Manipulators in Configuration Space" Proceedings of 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2022 https://doi.org/10.1109/IROS47612.2022.9981159 Citation Details
Kyriacou, Harrison and Ramakrishnan, Anand and Whitehill, Jacob "Learning to Work in a Materials Recovery Facility: Can Humans and Machines Learn from Each Other?" Learning Analytics and Knowledge , 2021 https://doi.org/10.1145/3448139.3448183 Citation Details
Li, Zeqian and He, Xinlu and Whitehill, Jacob "Compositional clustering: Applications to multi-label object recognition and speaker identification" Pattern Recognition , 2023 https://doi.org/10.1016/j.patcog.2023.109829 Citation Details
Natarajan, Sabhari and Brown, Galen and Calli, Berk "Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods" Frontiers in Robotics and AI , v.8 , 2021 https://doi.org/10.3389/frobt.2021.696587 Citation Details
Whitehill, Jacob and Erfanian, Amitai "How to Give Imperfect Automated Guidance to Learners: A Case-Study in Workplace Learning" International Conference on Artificial Intelligence in Education , 2022 https://doi.org/10.1007/978-3-031-11644-5_1 Citation Details
(Showing: 1 - 10 of 11)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project focused on developing a framework for human-robot collaboration to boost the efficiency of the municipal solid waste recycling process. The project team comprised roboticists, AI experts, industrial ecologists, and occupational psychologists to develop new robotics and AI technologies that can work in coordination and collaboration with the human workforce. In particular, the team investigated strategies to simultaneously improve the recycling economy, recycling efficiency, training process of the recycling personnel, and the safety and meaning of work for the recycling workers. To identify the design parameters at the intersection of these goals, numerous workshops, interviews, and surveys were conducted, facilitating discussions and information exchange with recycling companies, robotics companies, recycling workers, and academic researchers. Based on this experience, the interdisciplinary team worked closely to identify the type of robots, their optimal combinations, and their functions to achieve desired economic, environmental, and occupational outcomes. 

Collaborating with the recycling facilities in the Northeast United States, the project team produced the first public and free recycling dataset to facilitate object classification research and develop algorithms to identify the objects' materials from their colored images. Machine learning algorithms were developed to train these material classifiers. Also, several algorithms were developed to help robots automatically train themselves by observing the workers’ sorting actions. Such training strategies focusing on human-robot partnership enable continuous learning for robots, an essential capability to cope with the ever-changing recycling stream. Various robot end-effectors are also developed to handle different items effectively. Several robotic waste rearrangement techniques were developed to declutter the waste stream and facilitate the sorting operation. In addition, the team also investigated the efficacy of various methods for providing training cues to the recycling workers on waste classification and sorting. Considering the current technological capabilities and projecting them into the future, techno-economic analyses were conducted to determine the most suitable future directions for integrating robotics technology at waste sorting facilities.

The project team organized several workshops to discuss the role of robots in waste management with the broader robotics community. Additionally, the project provided excellent opportunities for numerous educational activities, with sustainability being a major topic of interest, especially for younger generations. As such, the team incorporated their research findings into existing courses, providing engaging case studies and discussing realistic challenges related to the integration of robots for environmentally focused tasks. They also developed a unique course titled “Robots for Recycling,” which trains future engineers to thoroughly understand and address the industry’s problems. Furthermore, various activities were designed for high school students to encourage them to think about the role of technology in achieving sustainability goals.


 

 


Last Modified: 01/28/2025
Modified by: Berk Calli

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page