NSF Org:	CNS Division Of Computer and Network Systems
Recipient:	UNIVERSITY OF DELAWARE
Initial Amendment Date:	August 7, 2024
Latest Amendment Date:	September 16, 2024
Award Number:	2343601
Award Instrument:	Standard Grant
Program Manager:	Marilyn McClure mmcclure@nsf.gov  (703)292-5197 CNS  Division Of Computer and Network Systems CSE  Directorate for Computer and Information Science and Engineering
Start Date:	October 1, 2024
End Date:	September 30, 2027 (Estimated)
Total Intended Award Amount:	$600,000.00
Total Awarded Amount to Date:	$600,000.00
Funds Obligated to Date:	FY 2024 = $600,000.00
History of Investigator:	Weisong Shi (Principal Investigator) weisong@udel.edu Liangkai Liu (Former Co-Principal Investigator)
Recipient Sponsored Research Office:	University of Delaware 550 S COLLEGE AVE NEWARK DE  US  19713-1324 (302)831-2136
Sponsor Congressional District:	00
Primary Place of Performance:	University of Delaware 220 HULLIHEN HALL NEWARK DE  US  19716-0099
Primary Place of Performance Congressional District:	00
Unique Entity Identifier (UEI):	T72NHKM259N3
Parent UEI:
NSF Program(s):	CSR-Computer Systems Research
Primary Program Source:	01002425DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s):	7923, 9150
Program Element Code(s):	735400
Award Agency Code:	4900
Fund Agency Code:	4900
Assistance Listing Number(s):	47.070

ABSTRACT

Deep Neural Network (DNN) technology has achieved significant success in autonomous driving systems, particularly in environment sensing and perception tasks. However, ensuring the timing predictability of DNN decisions during operation of autonomous vehicles (AV) is crucial for safety. Substantial time variations persist in most DNN models within AV systems, and this project?s novelties are systematically controlling such variations for autonomous vehicles secure operation. The project's broader significance is enhancing the reliability and safety of autonomous vehicle perception systems, ultimately reducing accidents and improving road safety.

This project consists of three research thrusts: (1) understanding the challenges of timing predictability in DNN inference for autonomous vehicles; (2) designing a framework for predictable DNN inference in multi-sensor and multi-task AV perception; and (3) integrating this framework into Autoware, a real AV pipeline. The project develops a configurable profiling framework to comprehensively understand the root causes of variability in the DNN inference pipeline. This framework allows fine-grained profiling of time variation issues, including data variability (sensor, weather, and traffic scenarios), model variability, and runtime system variability (communication middleware, operating system, and hardware architecture). To mitigate DNN inference time variations and ensure predictability, this project addresses single DNN inference variations through feature maps caching and fusion techniques. Additionally, multi-tenant DNN inference is optimized through co-scheduling across the application, middleware, operating system, and architectural layers. The team integrates the multi-tenant co-scheduling framework into Autoware, creating a lightweight message-wise timeline checkpoint with a feedback-based co-scheduler. Comprehensive evaluations are conducted using open AV datasets, an indoor connected and autonomous testbed (ICAT), and a 2018 Lincoln MKZ-based level-4 AV equipped with Autoware at the University of Delaware.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Please report errors in award information by writing to: awardsearch@nsf.gov.