Goal: In the aftermath of each natural disaster, researchers travel to the site to collect perishable photographic evidence needed to document the performance of our infrastructure and learn from that event. In a past project we established the fundamental knowledge needed to successfully implement image classification methods on unstructured reconnaissance data to organize these data and accelerate their use for scientific purposes. A researcher-oriented tool, the Automated Reconnaissance Image Organizer (ARIO), was built by leveraging automated image classification in a domain-specific manner to assist users to automatically classify, sort and organize their image data. 

This project, Integrating Human and Machine for Post-Disaster Visual Data Analytics: A Modern Media-Oriented Approach, extended ARIO from a tool to a comprehensive cloud-based data analytics service, Visual Structural Expertise Replicator (ViSER). In our work machine learning (ML) is used not to replace a human expert, but to augment and support the human expert by improving productivity and consistency while reducing tedious and time-consuming tasks. The recent explosion in funding in ML, AI, and computer science has encouraged rapid development and supported a transition from their use solely for idealized "toy" problems to practical implementations.  Our establishment of a curated dataset composed of 140,000 real-world field images from disaster scenes enabled the VISER platform to be realized here. VISER includes specialized modules for concrete buildings and bridges. 

ViSER consists of several components: a web-based post-disaster visual data organization and report generation application, an Infrastructure-as-Code based approach that seeks to create secure extend layer 2 services for provisioning bare systems (bare metal or virtual machines) over a wide area network, and a prototype image classification service that can be deployed on Purdue's community clusters. The fundamental applications (for buildings and bridges, and earthquakes and hurricanes) developed enable: (a) image localization within buildings to provide important context to the data; (b) similarity search to merge disparate data from a single structure; and (c) expanded the data schema and added relevant capabilities for organizing, searching, and filtering through image data. 

VISER provides researchers with the power to exploit vast amount of data and to rapidly search across construction type, damage features, geographical region, etc. Engineers will be empowered to design and construct a more resilient physical infrastructure using more comprehensive knowledge about structural performance, directly incorporated into building codes and influencing practice and policy. 

Partnerships: Our team's April 2023 workshop "Envisioning AI in a Decade" engaging 30+ industry/academic participants to discuss future opportunities and challenges of using AI in this domain. These discussions revealed important practical lessons. Practicing structural engineers are quite eager to use ML to make their work more efficient, while directing their time and energy toward resolving problems. Engineers feel confident about such results when they are given information to help them understand the rationale behind a classification decision, rather than simply accepting the outcome. Insufficient background and training in AI methods are major barriers to achieving this goal. Thus, now is the time to establish strong partnerships between in industry and academia to address these barriers and realize the potential of ML in engineering practice.

Partnering with a team of researchers from the Institute of Engineering-Mexico (UNAM) we integrated their post-earthquake survey experiences with our expertise on deep learning technologies. They shared the database they developed after the 2017 Puebla Earthquake with the Institute for Building Safety of Mexico City. 

Furthermore, after the devastating Mw. 7.8 Kahramanmaraş Earthquake in Turkey, a member of our team travelled to Turkey, joining the ACI 133 Reconnaissance team. During this trip, she visited dozens of buildings and collected over 100GB of building imagery to support this research.  

Impact on the Team: Research conducted by the graduate students at Purdue University includes work related to data collection, algorithm development and algorithm implementation. To participate in this process, they learned about the latest state of art on computer vision, building and infrastructure reconnaissance, visual inspection, data organization, technical writing, and the research process. 

Impact on the Built Environment and Society:

The methods developed in this project converted unstructured data in the form of images, that would have been otherwise difficult to analyze into reliable and organized data with well-defined and structural engineering - oriented categories. These categories, meant to support engineers in their routine decision-making, will assist engineers in the creation of design guidelines and pre-standards grounded on the new knowledge generated by the organized data. Ultimately, these methods will directly impact the design codes to build a safer and more resilient infrastructure to support our communities and societies. 

Last Modified: 09/28/2023
Modified by: Shirley J Dyke