
NSF Org: |
CNS Division Of Computer and Network Systems |
Recipient: |
|
Initial Amendment Date: | August 4, 2020 |
Latest Amendment Date: | August 4, 2020 |
Award Number: | 1952102 |
Award Instrument: | Standard Grant |
Program Manager: |
Ralph Wachter
rwachter@nsf.gov (703)292-8950 CNS Division Of Computer and Network Systems CSE Directorate for Computer and Information Science and Engineering |
Start Date: | October 1, 2020 |
End Date: | September 30, 2022 (Estimated) |
Total Intended Award Amount: | $150,000.00 |
Total Awarded Amount to Date: | $150,000.00 |
Funds Obligated to Date: |
|
History of Investigator: |
|
Recipient Sponsored Research Office: |
313 N 13TH ST MILWAUKEE WI US 53233-2244 (414)288-7200 |
Sponsor Congressional District: |
|
Primary Place of Performance: |
WI US 53201-1881 |
Primary Place of
Performance Congressional District: |
|
Unique Entity Identifier (UEI): |
|
Parent UEI: |
|
NSF Program(s): | S&CC: Smart & Connected Commun |
Primary Program Source: |
|
Program Reference Code(s): |
|
Program Element Code(s): |
|
Award Agency Code: | 4900 |
Fund Agency Code: | 4900 |
Assistance Listing Number(s): | 47.070 |
ABSTRACT
Public safety/security (PSS) is the most basic human need and on a college campus it is a must for college students to focus on their academics as they prepare to be productive citizens and faculty/staff to safely practice their educational mission. Improving PSS will also drive economic development in the surrounding communities. This project aims to identify social and cultural challenges of PSS on Marquette University (MU) campus with its surrounding communities and conduct a preliminary feasibility study of partly automating the current security monitoring infrastructure to improve detection of incidents while addressing the community concerns. The results of this project will be used to write an SCC-IRG grant proposal with the long-term objective and the national impact of creating a blueprint that is adaptable to thousands of college campuses in the US to establish an automated network of smart sensors for maintaining safe and secure zones on these campuses and their surrounding neighborhoods. This will improve the existing monitoring by law enforcement officers or security staff, and it will provide more reliable and actionable information for incident management.
The main objectives of this proposal are to: a) Understand in-depth the PSS challenges and the community concerns regarding the use of certain technologies on MU campus and in adjacent communities, b) Determine if existing technologies in multi-modal sensing and AI are sufficient to enable the level of automation that will lead to substantially enhanced diagnosis and prognosis decisions made by human operators, c) Combine multi-modal sensory data for cross-correlation in order to improve detection and predictive capabilities, d) Decide which features and system components need to be improved to enable real-time operation, e) Characterize the relation between the observables such as behavioral patterns, and PSS incidents. The scope of the proposed work and the methods and approaches used for each research item will be to: (i) Explore the PSS issues and challenges on MU campus and its periphery in collaboration with the MU Police Department and social scientist colleagues with expertise in crime mapping, analysis, and prevention, (ii) Solicit feedback from relevant stakeholders through community meetings and multidisciplinary workshops, and (iii) Conduct initial feasibility studies involving multi-modal (e.g. camera and audio) data together with contextual information simulating various scenarios (e.g. robbery in its early stages and tracking a suspect) in our labs to see whether some behavioral patterns can automatically predict certain PSS incidents with the help of MU Police Department. The intellectual merit of this proposal is in its paving the way to novel community-supported and data-driven solutions involving advances in real-time image processing, machine learning, predictive analysis, statistical detection and estimation toward developing a smart sensor network to enhance PSS.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
Objectives
This planning project was conducted to lay the foundations of an SCC-IRG proposal with the overarching goal of creating a smart sensor network to enhance safety and security on college campuses and their surrounding communities in the US.
Motivation
Safety and security are some of the most basic human needs. It is a must for college students to be able to focus on their academics and faculty and staff to practice their educational mission safely.
Project Outcomes and Findings
To establish the groundwork for the SCC-IRG proposal, the planning efforts involved a range of activities:
(i) exploring the safety and security issues and challenges in our campus and other college campuses in the US. We decided on three scenarios of security breach for the feasibility study and decided to explore others for the IRG proposal.
(ii) building collaborations with and soliciting feedback from relevant stakeholders as to the issues of concern. Feedback received from the community centered primarily on bias and profiling issues in law enforcement.
(iii) building a core group of faculty with technical, social and educational expertise. In addition to our existing technical expertise, we added a criminologist and an engineering professor with expertise in engineering education.
(iv) sharpening the research questions and hypotheses. We revised our security alert scheme to provide a greater control and the final decision-making ability to the surveillance personnel.
(v) recruiting and training students. We recruited 2 PhD students and an MS student. They have been trained in machine learning, image/video processing, detection and estimation theories. The MS student has completed his thesis work specifically on this project. The research work of 2 PhD students will form parts of their dissertations.
(vi) conducting feasibility studies on automatic violence detection, tracking a suspect without using facial features and displaying a weapon (as in a robbery),
(vii) analyzing and minimizing training and prediction bias in convolutional neural networks.
Automatic violence detection results by our machine learning algorithms met or exceeded the accuracies of other published results, as detailed in [1]. Other feasibility studies on tracking a suspect and detecting the display of a weapon are being prepared for publication.
We conducted a first-of-its-kind skin-tone bias analysis of the RWF-2000 dataset, which is frequently used in training machine learning algorithms for detecting violence in videos. This analysis revealed that people with some skin tones are more often depicted as involved in violence than others. The results are presented in an MS thesis [2].
After the quantitative analysis of model accuracy, a 3D extension of Grad-CAM is used to provide a qualitative understanding of model decisions or if there is algorithmic bias and therefore addressing the interpretability issue for building trust.
Broader Impacts
This planning grant efforts allowed the identification and possible ways of tackling technical challenges as well as social sensitivities associated with safety and security issues impacting communities on and around college campuses in the US.
The type of rapid alert system that we are conceiving for the NSF-IRG proposal would empower first responders by providing them faster, more accurate information before they arrive on-scene, potentially saving resources and possibly lives. This would represent a marked improvement over current systems, which depend the inefficient method of few people monitoring many cameras.
The training and algorithmic bias studies we have conducted are first of their kind in violence detection. As such, such bias analyses in machine learning will be extremely valuable to establishing community trust in law enforcement.
If IRG grant is funded, the research success will create a rapid alert system as a major deterrent to criminal behavior. This will enhance the feeling of security and therefore improve teaching and learning. It will drive economic development around the surrounding communities. It will also help in attracting to the university new faculty and student talents who enjoy engaging and serving the community through their research. The much broader impact will be in the form of a move starting on college campuses, where the technical expertise exists to adapt the conceived security system to their own unique needs, enhancing their safety and security which will spread to adjacent communities and finally to the towns where these campuses reside.
[1] J. Su, E. Clemens, P. Her, E. Yaz, S. Schneider and H. Medeiros, “Violence Detection using 3D Convolutional Neural Networks” Proc. of the 18th IEEE AVSS, DOI: 10.1109 / AVSS56176.2022.9959393, 2022.
[2] Erik Clemens, Marquette University EECE Department MS Thesis, Transfer Learning, Model Interpretation, and Dataset Bias Analysis for Automated Violence Detection from Video, May 2023.
Last Modified: 01/19/2023
Modified by: Edwin E Yaz
Please report errors in award information by writing to: awardsearch@nsf.gov.