Award Abstract # 1329878
CPS: Synergy: Collaborative Research: Formal Models of Human Control and Interaction with Cyber-Physical Systems

NSF Org: CNS
Division Of Computer and Network Systems
Recipient: UNIVERSITY OF NEW MEXICO
Initial Amendment Date: September 9, 2013
Latest Amendment Date: May 14, 2015
Award Number: 1329878
Award Instrument: Standard Grant
Program Manager: Sylvia Spengler
sspengle@nsf.gov
 (703)292-7347
CNS
 Division Of Computer and Network Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 15, 2013
End Date: August 31, 2017 (Estimated)
Total Intended Award Amount: $124,164.00
Total Awarded Amount to Date: $137,164.00
Funds Obligated to Date: FY 2013 = $124,164.00
FY 2015 = $13,000.00
History of Investigator:
  • Meeko Oishi (Principal Investigator)
    oishi@unm.edu
Recipient Sponsored Research Office: University of New Mexico
1 UNIVERSITY OF NEW MEXICO
ALBUQUERQUE
NM  US  87131-0001
(505)277-4186
Sponsor Congressional District: 01
Primary Place of Performance: University of New Mexico
NM  US  87131-0001
Primary Place of Performance
Congressional District:
01
Unique Entity Identifier (UEI): F6XLTRUQJEN4
Parent UEI:
NSF Program(s): Special Projects - CNS,
Secure &Trustworthy Cyberspace
Primary Program Source: 01001314DB NSF RESEARCH & RELATED ACTIVIT
01001516DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7918, 7434, 9150, 9102, 9251, 9178
Program Element Code(s): 171400, 806000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Cyber-Physical Systems (CPS) encompass a large variety of systems including for example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. One of the most important challenges in the design and deployment of Cyber-Physical Systems is how to formally guarantee that they are amenable to effective human control. This is a challenging problem not only because of the operational changes and increasing complexity of future CPS but also because of the nonlinear nature of the human-CPS system under realistic assumptions. Current state of the art has in general produced simplified models and has not fully considered realistic assumptions about system and environmental constraints or human cognitive abilities and limitations. To overcome current state of the art limitations, our overall research goal is to develop a theoretical framework for complex human-CPS that enables formal analysis and verification to ensure stability of the overall system operation as well as avoidance of unsafe operating states. To analyze a human-CPS involving a human operator(s) with bounded rationality three key questions are identified: (a) Are the inputs available to the operator sufficient to generate desirable behaviors for the CPS? (b) If so, how easy is it for the operator with her cognitive limitations to drive the system towards a desired behavior? (c) How can areas of poor system performance and determine appropriate mitigations be formally identified? The overall technical approach will be to (a) develop and appropriately leverage general cognitive models that incorporate human limitations and capabilities, (b) develop methods to abstract cognitive models to yield tractable analytical human models (c) develop innovative techniques to design the abstract interface between the human and underlying system to reflect mutual constraints, and (d) extend current state-of-the-art reachability and verification algorithms for analysis of abstract interfaces, iin which one of the systems in the feedback loop (i.e., the user) is mostly unknown, uncertain, highly variable or poorly modeled.

The research will provide contributions with broad significance in the following areas: (1) fundamental principles and algorithms that would serve as a foundation for provably safe robust hybrid control systems for mixed human-CPS (2) methods for the development of analytical human models that incorporate cognitive abilities and limitations and their consequences in human control of CPS, (3) validated techniques for interface design that enables effective human situation awareness through an interface that ensures minimum information necessary for the human to safely control the CPS, (4) new reachability analysis techniques that are scalable and allow rapid determination of different levels of system safety. The research will help to identify problems (such as automation surprises, inadequate or excessive information contained in the user interface) in safety critical, high-risk, or expensive CPS before they are built, tested and deployed. The research will provide the formal foundations for understanding and developing human-CPS and will have a broad range of applications in the domains of healthcare, energy, air traffic control, transportation systems, homeland security and large-scale emergency response. The research will contribute to the advancement of under-represented students in STEM fields through educational innovation and outreach. The code, benchmarks and data will be released via the project website.

Formal descriptions of models of human cognition are in general incompatible with formal models of the Cyber Physical System (CPS) the human operator(s) control. Therefore, it is difficult to determine in a rigorous way whether a CPS controlled by a human operator will be safe or stable and under which circumstances. The objective of this research is to develop an analytic framework of human-CPS systems that encompasses engineering compatible formal models of the human operator that preserve the basic architectural features of human cognition. In this project the team will develop methodologies for building such models as well as techniques for formal verification of the human-CPS system so that performance guarantees can be provided. They will validate models in a variety of domains ranging from air traffic control to large scale emergency response to the administration of anesthesia.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Abraham P. Vinod, Tyler Summers, and Meeko Oishi "User-interface design for MIMO LTI human-automation systems through sensor placement" User-interface design for MIMO LTI American Control Conference, Boston, MA , 2016 , p.5276 10.1109/ACC.2016.7526496
Abraham P. Vinod, Yuqing Tang, Meeko M. K. Oishi, Katia Sycara, Christian Lebiere and Michael Lewis "Validation of Cognitive Models for Collaborative Hybrid Systems with Discrete Human Input" IEEE/RSJ International Conference on Intelligent Robots and Systems , 2016 , p.3339 10.1109/IROS.2016.7759514
Abraham Vinod and Meeko Oishi "Scalable Underapproximation for the Stochastic Reach-Avoid Problem for High-Dimensional LTI Systems Using Fourier Transforms" IEEE Control Systems Letters , v.1 , 2017 , p.316--321 10.1109/LCSYS.2017.2716364
Hao-Tien Chiang, Nicholas Malone, Kendra Lesser, Meeko Oishi, Lydia Tapia "Hybrid Dynamic Moving Obstacle Avoidance Using a Stochastic ReachableSet Based Potential Field" IEEE Transactions on Robotics , v.33 , 2017 , p.1124 - 11 10.1109/TRO.2017.2705034
Hao-Tien (Lewis) Chiang, Baisravan HomChaudhuri, Abraham P. Vinod, Meeko Oishi, Lydia Tapia "Dynamic risk tolerance: Motion planning by balancing short-term and long-term stochastic dynamic predictions" Int'l Conference on Robotics and Automation , 2017 , p.3762 10.1109/ICRA.2017.7989434
Joseph Gleason, Abraham P. Vinod, Meeko M. K. Oishi, and R. Scott Erwin "Viable Set Approximation for Linear-Gaussian Systems with Unknown, Bounded Variance" IEEE Int?l Conference on Decision and Control , 2016 , p.7049 10.1109/CDC.2016.7799355
Kendra Lesser and Meeko Oishi "Approximate Safety Verification and Control of Partially Observable Stochastic Hybrid Systems" IEEE Transactions on Automatic Control , v.62 , 2017 , p.81 10.1109/TAC.2016.2535128
Kendra Lesser and Meeko Oishi "Reachability for partially observable discrete time stochastic hybrid systems" Automatica , v.50 , 2014 , p.1989 10.1016/j.automatica.2014.05.012

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

 Cyber Physical Systems (CPS) consist of networked entities  with integrated sensing, actuation, control, and communication capabilities. Examples of CPS include critical systems, such as future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation.  Even with recent significant advancement in autonomous systems technology, these CPS will have humans in the loop. With the increasing number of sophisticated systems that often cooperate closely with humans or that need to be directly operated by a human operator, major questions arise of how to understand this interaction and ensure the safe performance of the overall system. One of the key challenges is how to formally guarantee that human-CPS are effectively controllable under realistic assumptions about the operating environment and about human capabilities and limitations.

Past work in human factors has not for the most part considered complex dynamics of the underlying system. On the other hand, work on systems control has not considered realistic models of the human. As a result, only very simplified models have been produced that so not begin to address real world challenges. Our research goal was to close this gap by designing cognitive models, based on human experiments and on the validated ACT-R cognitive architecture in various domains, derive an analytic model based on the cognitive model and develop  verification techniques that consider the cognitively-based analytic model.

Following this overall methodology, our research produced models and formal verification techniques in two domains, namely (a) human control of a simulated robotic swarm and  (b) fluid management in critical medical care to address situations such as hemorrhagic shock (rapid loss of blood) that can result in in multiple organ failure. In particular the key scientific outcomes of the research are:

  • Results of human experiments in the control of robotic swarms in an environment with obstacles and in fluid management of critical care.
  • Cognitive models for the robot swarm control task and the fluid management task 
  • An analytic Markov model, based on the corresponding cognitive model,  for the swarm control task
  • An analytic model based on a Long Short Term Memory (LSTM) RNN architecture  for the critical care task
  • Theory and computational techniques for stochastic reachable sets (and stochastic optimal controllers) under incomplete information. 
  • Scalable algorithms to compute stochastic reachable sets and underapproximations of stochastic reach-avoid sets and probability measures, based on a Fourier transform. 

Besides multiple publications and one best paper award, our research produced one new course on verification algorithms considering human and CPS, as well as research training for underrepresented minorities, such as female and Hispanic graduate and undergraduate students.

Additionally, the research has broader impacts in that the developed algorithms could identify problems in safety critical, high-risk, or expensive CPS before they are built, tested and deployed in a  range of applications in the domains of healthcare, energy, air traffic control, transportation systems, homeland security and large-scale emergency response.


Last Modified: 12/01/2017
Modified by: Meeko Oishi

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page