In this project, we investigated how AI, robotics, and generative design could be leveraged to transform the construction industry by tackling issues such as the reliance on skilled labor, inefficient data recording, and worker idle times. By integrating Artificial Intelligence (AI), robotics, and generative design in the context of modular construction, the project sought to reduce human errors, improve production line efficiency, and build a cyberinfrastructure for industry-academia collaboration.

We developed digital twin technologies by scanning a modular factory to create a 3D replica for semantic grounding and dynamic simulation. This enabled the creation of realistic synthetic image datasets in virtual reality (VR) to train AI models for workpiece recognition, proving a cost-effective alternative to real-world data collection. VR-trained models successfully detected wall subassemblies in a real factory with 59.4% accuracy, despite visual clutter. A robot-assisted AR system was prototyped, using a projector on a robotic dog to display virtual project information directly onto the construction site for workers, removing the need for individual AR goggles. This prototype helped the team win a Department of Energy (DOE) ERobot competition and led to a provisional patent application for robotic inspection. We enhanced self-supervised visual place recognition (TF-VPR) for mobile robots to navigate cluttered construction sites

and developed DeepMapping2 for robust LiDAR map optimization on large datasets, improving 3D mapping for autonomous robots. To enable AI-based generative design for modular construction standardization, approximately 8,000 residential unit designs were converted into BIM/IFC format for AI training. Our research revealed that state-of-the-art AI models (House-GAN++) often miss architects' "unwritten rules" for design. To address this, we developed a knowledge-informed Deep Reinforcement Learning (DRL) model (RL-PLAN), which integrates architectural knowledge, showing improved privacy levels and space allocation in generated floor plans. A prototype cyberinfrastructure platform was built for sharing modular designs and hosting challenges, uniquely allowing 3D object searches based on feature similarity.

The project directly addresses modular construction inefficiencies, leading to faster project completion, reduced material waste, and enhanced worker safety by moving hazardous activities to controlled factory environments. These technologies can also benefit other heavy manufacturing industries, such as shipbuilding and agriculture. Extensive human resources development occurred, with numerous PhD, MS, undergraduate, and high school students from various backgrounds and experience, gaining research experience in AI, robotics, and modular construction. Project findings were integrated as learning modules into graduate courses at New York University and the University of Michigan.

Last Modified: 08/05/2025
Modified by: Semiha Ergan