Award Abstract # 1956071
Collaborative Research:SHF:Medium:Machine Learning on the Edge for Real-Time Microsecond State Estimation of High-Rate Dynamic Events

NSF Org: CCF
Division of Computing and Communication Foundations
Recipient: UNIVERSITY OF SOUTH CAROLINA
Initial Amendment Date: July 23, 2020
Latest Amendment Date: December 20, 2023
Award Number: 1956071
Award Instrument: Continuing Grant
Program Manager: Almadena Chtchelkanova
achtchel@nsf.gov
 (703)292-7498
CCF
 Division of Computing and Communication Foundations
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: August 1, 2020
End Date: July 31, 2026 (Estimated)
Total Intended Award Amount: $690,248.00
Total Awarded Amount to Date: $738,248.00
Funds Obligated to Date: FY 2020 = $377,808.00
FY 2022 = $328,440.00

FY 2023 = $16,000.00

FY 2024 = $16,000.00
History of Investigator:
  • Jason Bakos (Principal Investigator)
    jbakos@cse.sc.edu
  • Austin Downey (Co-Principal Investigator)
Recipient Sponsored Research Office: University of South Carolina at Columbia
1600 HAMPTON ST
COLUMBIA
SC  US  29208-3403
(803)777-7093
Sponsor Congressional District: 06
Primary Place of Performance: University of South Carolina at Columbia
Room 2213, Storey Innovation Cen
Columbia
SC  US  29208-0001
Primary Place of Performance
Congressional District:
06
Unique Entity Identifier (UEI): J22LNTMEDP73
Parent UEI: Q93ZDA59ZAR5
NSF Program(s): Software & Hardware Foundation
Primary Program Source: 01002021DB NSF RESEARCH & RELATED ACTIVIT
01002223DB NSF RESEARCH & RELATED ACTIVIT

01002324DB NSF RESEARCH & RELATED ACTIVIT

01002425DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7924, 7942, 9150, 9251
Program Element Code(s): 779800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Computer control of dynamic systems from the manufacturing, robotics, and aviation fields traditionally operate on timescales of 10s or 100s of milliseconds. For example, an avionics system traveling at 1000 kilometers per hour and operating at 10 milliseconds per control decision will move three meters in the time allocated to each control decision. However, emerging hypersonic, space, and military systems require active control while operating at extreme velocities or while being subjected to accelerations or decelerations caused by explosions or high-speed collisions. These applications require control at timescales on the order of microseconds. Making control decisions for such systems often requires that the controller estimate the state of the system from indirect measurements such as vibration. Traditional methods for state prediction are based on first principles using finite element analysis (FEA), whose execution time scales as a square of the number of elements. This makes it impractical to evaluate FEA models at microsecond timescales. Models derived from machine learning can estimate the state of the system based on pre-curated datasets and require less workload as compared to an equivalent FEA model. Such models, when combined with domain-specific processors, could provide equivalent accuracy with higher throughput than FEA models, making microsecond-scale state modeling possible. However, there are currently no suitable development methodologies for systematic generation of machine-learning models at such extreme performance constraints. The objective of this research is to develop a structural model compiler that meets a given accuracy constraint, as well as a corresponding overlay generator on which the generated model meets a given microsecond-scale latency constraint. This research will advance the fundamental knowledge and skills required for the real-time decision-making and control of active structures that experience high-rate dynamic events.

This project addresses two distinct but synergistic problems: (1) technologies to enable real-time decision-making and control of active structures that experience dynamic events at the microsecond timescale and (2) development of tools for optimization and synthesis of domain-specific processors for trained models. Recent academic and industrial work focusing on development of specialized architectures for evaluating Long Short Term Memory (LSTM) models generally yield ?one-off? designs tuned to a specific Field Programmable Gate Array (FPGA)--often a server class FPGA--and have rigid, ?baked in? design decisions. This makes it difficult to compare alternative or competing optimization techniques for a desired target FPGA platform. To solve this, this project is developing a generalized programmable processor architecture that incorporates a repertoire of optional features designed to accelerate specific aspects of LSTMs and support associated model optimizations. The architecture is both programmable and customizable, allowing it to serve as a common platform for evaluating different approaches for accelerating LSTM models. Concurrently, the investigators are developing a set of benchmark datasets for structural state estimation with accuracy and performance requirements. The project is also developing useful artifacts for subsequent research in edge-based machine learning, including a method for comparing different LSTM model-pruning and compression approaches and comparing different microarchitecture designs. Code and hardware designs developed from this project are open-source.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 14)
Alexander B. Vereen, Emmanuel A. "Optimal Sampling Methodologies for High-rate Structural Twinning" Proc. 26th International Conference on Information Fusion, Jun. 27-30, 2023 (FUSION 2023) , 2023 Citation Details
Atiyehsadat Panahi, Ehsan Kabir "High-rate machine learning for forecasting time-series signals" In 2022 IEEE 30th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM 2022) , 2023 Citation Details
Chowdhury, Puja and Barzegar, Vahid and Satme, Joud and Downey, Austin and Laflamme, Simon and Bakos, Jason D. and Hu, Chao "Deterministic and low-latency time-series forecasting of nonstationary signals" Active and Passive Smart Structures and Integrated Systems XVI. SPIE, Apr. 2022 , 2022 https://doi.org/10.1117/12.2629025 Citation Details
Chowdhury, Puja and Conrad, Philip and Bakos, Jason D. and Downey, Austin "Time Series Forecasting for Structures Subjected to Nonstationary Inputs" ASME 2021 Conference on Smart Materials, Adaptive Structures and Intelligent Systems , v.85499 , 2021 https://doi.org/10.1115/SMASIS2021-68338 Citation Details
Ehsan Kabir, Daniel Coble "Accelerating LSTM-based High-Rate Dynamic System Models" Proc. 33rd International Conference on Field Programmable Logic and Applications (FPL 2023) , 2023 Citation Details
Joud Satme, Daniel Coble "Progress towards data-driven high-rate structural state estimation on edge computing devices" In Volume 10: 34th Conference on Mechanical Vibration and Sound (VIB). American Society of Mechanical Engineers, aug 2022. , 2023 Citation Details
MD Arafat Kabir, Ehsan Kabir "FPGA Processor In Memory Architectures (PIMs): Overlay or Overhaul?" Proc. 33rd International Conference on Field Programmable Logic and Applications (FPL 2023) , 2023 Citation Details
M. Kabir and J. Hollis and A. Panahi and J. Bakos and M. Huang and D. Andrews "Making BRAMs Compute: Creating Scalable Computational Memory Fabric Overlays" Proc. of the 31st IEEE International Symposium On Field-Programmable Custom Computing (FCCM 2023) , 2023 https://doi.org/10.1109/FCCM57271.2023.00052 Citation Details
Ogunniyi, Emmanuel A. and Drnek, Claire and Hong, Seong Hyeon and Downey, Austin R.J. and Wang, Yi and Bakos, Jason D. and Avitabile, Peter and Dodson, Jacob "Real-time structural model updating using local eigenvalue modification procedure for applications in high-rate dynamic events" Mechanical Systems and Signal Processing , v.195 , 2023 https://doi.org/10.1016/j.ymssp.2023.110318 Citation Details
Ogunniyi, Emmanuel and Downey, Austin R. and Bakos, Jason "Development of a real-time solver for the local eigenvalue modification procedure" SPIE Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2022 , 2022 https://doi.org/10.1117/12.2613208 Citation Details
Panahi, Atiyehsadat and Kabir, Ehsan and Downey, Austin and Andrews, David and Huang, Miaoqing and Bakos, Jason D. "High-Rate Machine Learning for Forecasting Time-Series Signals" 2022 IEEE 30th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) , 2022 https://doi.org/10.1109/FCCM53951.2022.9786127 Citation Details
(Showing: 1 - 10 of 14)

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page