Award Abstract # 1031329
Reciprocal Reconstruction and Recognition for Modeling of Constructed Facilities

NSF Org: CMMI
Division of Civil, Mechanical, and Manufacturing Innovation
Recipient: GEORGIA TECH RESEARCH CORP
Initial Amendment Date: August 25, 2010
Latest Amendment Date: August 17, 2012
Award Number: 1031329
Award Instrument: Standard Grant
Program Manager: elise miller-hooks
CMMI
 Division of Civil, Mechanical, and Manufacturing Innovation
ENG
 Directorate for Engineering
Start Date: September 1, 2010
End Date: August 31, 2015 (Estimated)
Total Intended Award Amount: $306,043.00
Total Awarded Amount to Date: $306,043.00
Funds Obligated to Date: FY 2010 = $306,043.00
History of Investigator:
  • Patricio Vela (Principal Investigator)
    pvela@ece.gatech.edu
  • Ioannis Brilakis (Former Principal Investigator)
  • Patricio Vela (Former Co-Principal Investigator)
Recipient Sponsored Research Office: Georgia Tech Research Corporation
926 DALNEY ST NW
ATLANTA
GA  US  30318-6395
(404)894-4819
Sponsor Congressional District: 05
Primary Place of Performance: Georgia Tech Research Corporation
926 DALNEY ST NW
ATLANTA
GA  US  30318-6395
Primary Place of Performance
Congressional District:
05
Unique Entity Identifier (UEI): EMW9FC8J3HN4
Parent UEI: EMW9FC8J3HN4
NSF Program(s): CIS-Civil Infrastructure Syst
Primary Program Source: 01001011DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 023E, 036E, 039E, 1057, CVIS
Program Element Code(s): 163100
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.041

ABSTRACT

The research objective of this project is to evaluate whether a novel framework proposed by the PIs can progressively reconstruct a reinforced concrete frame structure into an object-oriented geometric model, for the purpose of automating the Building Information Model (BIM) making process of constructed facilities in a cost-effective manner. According to the project's framework, the modeler videotapes the structure from all accessible angles to minimize occlusions. During this stage, the structural members (concrete columns and beams in this study) in the resulting stream of images are detected and their occupying region is marked in all images. These regions are used to establish correspondence at the object level across images, and solve the rough registration problem efficiently. Line-based structure from motion is then applied to the result to produce a rendered 3D view of the structure with the recognized regions marked. This loops back to the detection of structural members, which can now be also performed on the spatial data covered by the visually marked regions. The result is more robust element detection (by combining visual and spatial detection results), and consequently improved element matching and reconstruction. The resulting object-oriented model is expected to be an accurate 3D representation of the structure with the load bearing linear members detected. This model is provided to the modeler, who can then use it to complete the model making process. As a result, the key intellectual merit of this framework lies in its reciprocal use of the results; the video recognition of building elements is used to assist the 3D reconstruction of their spatial data, while the 3D reconstruction provides the spatial data needed for spatial recognition to assist in more robust element detection.

The immediate advantage that will result from this work is the ability to automate the modeling of frequent elements during the as-built model generation process, which translates to tremendous time savings for the modeler. The National Academy of Engineering recently listed Restoring and Improving Urban Infrastructure as one of the Grand Challenges of Engineering in the 21st century. Two of the greatest issues that cause this grand challenge are the need for more automation in construction, through advances in computer science and robotics, and the lack of viable methods to map and label existing infrastructure. Over two thirds of the effort needed to model even simple infrastructure is spent on manually converting surface data to a 3D model. The result is that as-built models are not produced for the vast majority of new construction and retrofit projects, which leads to rework and design changes that cost up to 10 percent of the installed costs. Any efforts towards automating the modeling process will increase the percentage of infrastructure projects being modeled and, considering that construction is a $900 billion industry, each 1 percent of increase can lead up to $900 million in savings.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 15)
A. Rashidi and F. Dai and I. Brilakis and P.A. Vela "Optimized Selection of Key Frames for Monocular Videogrammetric Surveying of Civil Infrastructure" Journal of Advanced Engineering Informatics , v.27 , 2013 , p.270 ? 282 10.1016/j.aei.2013.01.002
A. Rashidi and H. Fathi and I. Brilakis "Innovative Stereo Vision-Based Approach to Generate Dense Depth Map of Transportation Infrastructure" Journal of the Transportation Research Board , v.2215 , 2011 , p.93 ? 99 10.3141/2215-10
A. Rashidi and I. Brilakis and P. Vela "Generating Absolute-Scale Point Cloud Data of Built Infrastructure Scenes Using a Monocular Camera Setting" ASCE Journal of Computing in Civil Engineering , v.29 , 2015 , p.n04014089 10.1061/(ASCE)CP.1943-5487.0000414
Brilakis, I., Dai, F. and Radopoulou, S.C. "Achievements and Challenges in Recognizing and Reconstructing Civil Infrastructure" Lecture Notes in Computer Science Journal , 2011
Brilakis, I; Fathi, H; Rashidi, A "Progressive 3D reconstruction of infrastructure with videogrammetry" AUTOMATION IN CONSTRUCTION , v.20 , 2011 , p.884 View record at Web of Science 10.1016/j.autcon.2011.03.00
Dai, F., Rashidi, A., Brilakis, I. and Vela, P "Sparse Point Cloud generation of Infrastructure Scenes Using Monocular Videogrammetry" Journal of Computing in Civil Engineering , 2012 , p.929
Dai, F., Rashidi, A., Brilakis, I. and Vela, P. "Comparison of Image- and Time-of-Flight-Based Technologies for Three Dimensional Reconstruction of Infrastructure" ASCE Journal of Construction Engineering and Management , v.139 , 2013 , p.69?79 10.1061/(ASCE)CO.1943-7862.0000565
F. Dai and A. Rashidi and I. Brilakis and P.A. Vela "Comparison of Image-Based and Time-of-Flight-Based Technologies for Three-Dimensional Reconstruction of Infrastructure" ASCE Journal of Construction Engineering and Management , v.139 , 2013 , p.69?79 10.1061/(ASCE)CO.1943-7862.0000565
G. Zhang, P. Vela, I. Brilakis, and P. Karasev "Sparsity-Inducing Optimization Based Algorithm for Planar Patch Extraction from Noisy Point-Cloud Data" Computer-Aided Civil Infrastructure Engineering , v.30 , 2015 , p.85 10.1111/mice.12063
I. Brilakis and S. German and Z. Zhu "Visual Pattern Recognition Models for Remote Sensing of Civil Infrastructure" ASCE Journal of Computing in Civil Engineering , v.25 , 2011 , p.388 ? 393 10.1061/(ASCE)CP.1943-5487.0000104
I. Brilakis H. Fathi and A. Rashidi "Progressive 3D reconstruction of Infrastructure with Videogrammetry" Journal of Automation in Construction , v.20 , 2011 , p.884?895 10.1016/j.autcon.2011.03.005
(Showing: 1 - 10 of 15)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The research objective of this project was to evaluate whether a novel
framework proposed by the PIs could  progressively reconstruct a reinforced
concrete frame structure into an object-oriented geometric model, for
the purpose of automating the Building Information Model (BIM). The process takes as input an image sequence recorded from video and outputs a solid model of the sensed infrastructure where each component is semantically labelled, classified, and its geometry defined. Automatic generation of as-built BIM for constructed facilities would be highly cost-effective, as the majority of modeling efforts go into the creation of object-oriented, semantically-labelled geometric models.

Intellectual Merit. The research associated to this award consisted of two main components (i) the creation of a 3D reconstruction of the scanned infrastructure from video (e.g., videgrammetry), and (ii) the automatic interpretation of the 3D reconstruction for conversion to a BIM. Accomplished research on the first item led to the creation of a system that can select keyframes within the video sequence for high quality reconstruction.  Furthermore, the accuracy of videogrammetry was evaluted and shown to be accurate enough for take-off, but still short of that required for defect detection. Improvements in the underlying bundle adjustment algorithm may bring this down further, as indicated by the most recent research results. Once the structure has been reconstructed (with sufficient density), the next tsep is to apply detection, modeling, and classification algorithms to convert the point cloud into a semantically-enriched solid mode.  The proposed solution is a combination of surface primitve clustering and modeling, followed by a machine learning-based surface merging and classification algorithm. The learning approach could be of any type, however boosted random forests were found to work quite well, especially with regards to identifying what neighboring information was of utility for merging surfaces into a solid model.  The net result is a framework for going from input video to a BIM.  The BIM model is only partially populated with meta-data, as no material recognition is performed.  The BIM only provides the 3D geometry of the object as well as a semantic label indicating its (structural) function (e.g., beam, column, deck, sidewall, etc). The structure of application was defined to be concrete beam-deck bridges and other structurally similar types. The resulting object-oriented model is an accurate 3D representation of the structure with the load bearing linear members detected.

Broader Impacts. The research achieved is a milestone in the process of automating BIM creation from scans (visual and/or depth) of as-built infrastructure. The output model is provided to the modeler, who can then use it to complete the model making process. A large portion of the time and  cost of generating an as-built is in the model creation phase. Automating this process will lower both, leading to more efficient workflows. Furthermore, a fast and accurate model creation algorithm would enable real-time support for project management decisions during construction, and provide more accurate information during infrastructure modification projects. Continuous industrial engagement ensured that the work was relevant to potential stakeholders, as well as to keep them appraised of the state-of-research. Emphasis on a technology transfer pathway has led to collborations with governmental civil infrastructure authorities as well as with several civil engineering companies (large and small).

In fulfilling the proposed work, the team also actively engaged in outreach and education.  High school students were actively engaged in learning how to read and interpret architectural plans, as ...

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page