Award Abstract # 1952102
SCC-PG: Safety and Security of College Campuses and their Adjacent Communities

NSF Org: CNS
Division Of Computer and Network Systems
Recipient: MARQUETTE UNIVERSITY
Initial Amendment Date: August 4, 2020
Latest Amendment Date: August 4, 2020
Award Number: 1952102
Award Instrument: Standard Grant
Program Manager: Ralph Wachter
rwachter@nsf.gov
 (703)292-8950
CNS
 Division Of Computer and Network Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2020
End Date: September 30, 2022 (Estimated)
Total Intended Award Amount: $150,000.00
Total Awarded Amount to Date: $150,000.00
Funds Obligated to Date: FY 2020 = $150,000.00
History of Investigator:
  • Edwin Yaz (Principal Investigator)
    edwin.yaz@marquette.edu
  • Fabien Josse (Co-Principal Investigator)
  • Majeed Hayat (Co-Principal Investigator)
  • Cristinel Ababei (Co-Principal Investigator)
  • Henry Medeiros (Co-Principal Investigator)
Recipient Sponsored Research Office: Marquette University
313 N 13TH ST
MILWAUKEE
WI  US  53233-2244
(414)288-7200
Sponsor Congressional District: 04
Primary Place of Performance: Marquette University
WI  US  53201-1881
Primary Place of Performance
Congressional District:
04
Unique Entity Identifier (UEI): HKJCKTFJNBM7
Parent UEI:
NSF Program(s): S&CC: Smart & Connected Commun
Primary Program Source: 01002021DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 042Z
Program Element Code(s): 033Y00
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Public safety/security (PSS) is the most basic human need and on a college campus it is a must for college students to focus on their academics as they prepare to be productive citizens and faculty/staff to safely practice their educational mission. Improving PSS will also drive economic development in the surrounding communities. This project aims to identify social and cultural challenges of PSS on Marquette University (MU) campus with its surrounding communities and conduct a preliminary feasibility study of partly automating the current security monitoring infrastructure to improve detection of incidents while addressing the community concerns. The results of this project will be used to write an SCC-IRG grant proposal with the long-term objective and the national impact of creating a blueprint that is adaptable to thousands of college campuses in the US to establish an automated network of smart sensors for maintaining safe and secure zones on these campuses and their surrounding neighborhoods. This will improve the existing monitoring by law enforcement officers or security staff, and it will provide more reliable and actionable information for incident management.

The main objectives of this proposal are to: a) Understand in-depth the PSS challenges and the community concerns regarding the use of certain technologies on MU campus and in adjacent communities, b) Determine if existing technologies in multi-modal sensing and AI are sufficient to enable the level of automation that will lead to substantially enhanced diagnosis and prognosis decisions made by human operators, c) Combine multi-modal sensory data for cross-correlation in order to improve detection and predictive capabilities, d) Decide which features and system components need to be improved to enable real-time operation, e) Characterize the relation between the observables such as behavioral patterns, and PSS incidents. The scope of the proposed work and the methods and approaches used for each research item will be to: (i) Explore the PSS issues and challenges on MU campus and its periphery in collaboration with the MU Police Department and social scientist colleagues with expertise in crime mapping, analysis, and prevention, (ii) Solicit feedback from relevant stakeholders through community meetings and multidisciplinary workshops, and (iii) Conduct initial feasibility studies involving multi-modal (e.g. camera and audio) data together with contextual information simulating various scenarios (e.g. robbery in its early stages and tracking a suspect) in our labs to see whether some behavioral patterns can automatically predict certain PSS incidents with the help of MU Police Department. The intellectual merit of this proposal is in its paving the way to novel community-supported and data-driven solutions involving advances in real-time image processing, machine learning, predictive analysis, statistical detection and estimation toward developing a smart sensor network to enhance PSS.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Su, J. and Her, P. and Clemens, E. and Yaz, E. and Schneider, S. and Medeiros, H. "Violence Detection using 3D Convolutional Neural Networks" IEEE International Conference on Advanced Video and Signal Based Surveillance , 2022 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

Objectives

This planning project was conducted to lay the foundations of an SCC-IRG proposal with the overarching goal of creating a smart sensor network to enhance safety and security on college campuses and their surrounding communities in the US.   

 Motivation

Safety and security are some of the most basic human needs. It is a must for college students to be able to focus on their academics and faculty and staff to practice their educational mission safely.  

 Project Outcomes and Findings

To establish the groundwork for the SCC-IRG proposal, the planning efforts involved a range of activities:

(i)             exploring the safety and security issues and challenges in our campus and other college campuses in the US. We decided on three scenarios of security breach for the feasibility study and decided to explore others for the IRG proposal.

(ii)           building collaborations with and soliciting feedback from relevant stakeholders as to the issues of concern. Feedback received from the community centered primarily on bias and profiling issues in law enforcement.

(iii)         building a core group of faculty with technical, social and educational expertise. In addition to our existing technical expertise, we added a criminologist and an engineering professor with expertise in engineering education.

(iv)          sharpening the research questions and hypotheses. We revised our security alert scheme to provide a greater control and the final decision-making ability to the surveillance personnel.

(v)           recruiting and training students. We recruited 2 PhD students and an MS student. They have been trained in machine learning, image/video processing, detection and estimation theories. The MS student has completed his thesis work specifically on this project. The research work of 2 PhD students will form parts of their dissertations.

(vi)          conducting feasibility studies on automatic violence detection, tracking a suspect without using facial features and displaying a weapon (as in a robbery),

(vii)        analyzing and minimizing training and prediction bias in convolutional neural networks.

Automatic violence detection results by our machine learning algorithms met or exceeded the accuracies of other published results, as detailed in [1]. Other feasibility studies on tracking a suspect and detecting the display of a weapon are being prepared for publication.

 We conducted a first-of-its-kind skin-tone bias analysis of the RWF-2000 dataset, which is frequently used in training machine learning algorithms for detecting violence in videos. This analysis revealed that people with some skin tones are more often depicted as involved in violence than others. The results are presented in an MS thesis [2].

After the quantitative analysis of model accuracy, a 3D extension of Grad-CAM is used to provide a qualitative understanding of model decisions or if there is algorithmic bias and therefore addressing the interpretability issue for building trust.   

Broader Impacts  

This planning grant efforts allowed the identification and possible ways of tackling technical challenges as well as social sensitivities associated with safety and security issues impacting communities on and around college campuses in the US.

The type of rapid alert system that we are conceiving for the NSF-IRG proposal would empower first responders by providing them faster, more accurate information before they arrive on-scene, potentially saving resources and possibly lives. This would represent a marked improvement over current systems, which depend the inefficient method of few people monitoring many cameras.   

The training and algorithmic bias studies we have conducted are first of their kind in violence detection. As such, such bias analyses in machine learning will be extremely valuable to establishing community trust in law enforcement.

If IRG grant is funded, the research success will create a rapid alert system as a major deterrent to criminal behavior. This will enhance the feeling of security and therefore improve teaching and learning. It will drive economic development around the surrounding communities. It will also help in attracting to the university new faculty and student talents who enjoy engaging and serving the community through their research. The much broader impact will be in the form of a move starting on college campuses, where the technical expertise exists to adapt the conceived security system to their own unique needs, enhancing their safety and security which will spread to adjacent communities and finally to the towns where these campuses reside.

[1] J. Su, E. Clemens, P. Her, E. Yaz, S. Schneider and H. Medeiros, “Violence Detection using 3D Convolutional Neural Networks” Proc. of the 18th IEEE AVSS, DOI: 10.1109 / AVSS56176.2022.9959393, 2022. 

[2] Erik Clemens, Marquette University EECE Department MS Thesis, Transfer Learning, Model Interpretation, and Dataset Bias Analysis for Automated Violence Detection from Video, May 2023. 

 


Last Modified: 01/19/2023
Modified by: Edwin E Yaz

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page