Journal of Computing in Civil Engineering

The exponential growth in reality capture and Building Information Modeling (BIM)-enabled construction workflows has created a surge in computer vision–driven solutions that automatically model and compare as-built conditions against BIM, offering project teams actionable insight into construction progress and quality. Despite their significant impact, the performance of these methods heavily relies on the accuracy and completeness of the reality capture. In addition, and especially in the case of reality captures conducted with camera-equipped unmanned aerial vehicles (UAVs), operational requirements—including battery capacity and operator’s line of sight (LOS)—should be carefully considered for safe flight execution. Accounting for these technical and operational requirements during reality capture planning requires expertise. In addition, it involves a significant amount of manual tweaking that does not scale well to ongoing changes due to progress on construction projects. To address these limitations, this paper presents a novel multiobjective optimization method to improve reality capture plans aiming to maximize (1) visual coverage of the monitored structure, (2) redundant observation of the structure’s components in the collected frames, (3) resolution of the structure’s elements in the captured data, (4) canonical camera viewpoints to the structure’s topology, and (5) stability of three-dimensional (3D) reconstruction algorithms used to process the data altogether, while (6) reducing the data collection duration. The objectives are also set to meet other technical and operational requirements, particularly for camera-equipped UAVs. Furthermore, a client-server system architecture is presented to visualize, simulate, and optimize reality capture missions in a web-based 3D environment using four-dimensional (4D) BIM to indicate the as-planned expected changes. Five conducted experiments using real-world data demonstrated the method’s capability to enhance the quality of user-created reality capture plans. The optimization process resulted in a 7.65% improvement in visual coverage, 30.89% enhancement in the structure’s resolution, and 8.95% more stable 3D reconstruction while ensuring the flight paths meet operational requirements.

Regression modeling has been based on analyzing error terms between the predicted output and the target output without addressing the variance of the predicted output and the impact of individual input parameters on the variance. This research critically reviews established methods for variance analysis on commonly applied multiple linear regressions (MLR). An MLR model with high accuracy (the mean of the prediction close to the target value) but low precision (too high of a variance of the prediction) would be deemed insufficient in the context of cost estimating applications. An analytical method to account for the impact of the uncertainty associated with each input parameter on the uncertainty of the final output has yet to be formalized. This research integrates the error propagation theory with MLR modeling in an attempt to quantify the variance of the MLR predicted output in estimating labor cost for prefabricated products. A metric based on the resulting variance analysis (i.e., the ratio of the standard deviation over the mean) is found effective to gauge the precision of the MLR model. The research has advanced regression modeling methods with respect to MLR variance analysis and contributed to the estimating practice for prefabricated products such as structural steel fabrication, precast concrete, industrial modules, and building modules.

This paper presents a systematic method to create universally applicable synthetic training image datasets for computer vision-based construction object detection. The synthetic images created by inserting a virtual object of interest into a real site image allows us to minimize the time and effort for training image data collection and annotation. In addition, the use of synthetic images has an additional benefit that training images can be easily customized for a target construction site by considering the context of the site (e.g., different background scenes, camera positions, and angles) and the possible variability of target objects to be detected (e.g., different sizes, locations, rotation angles, and postures) on images. An automated approach proposed in this study attempts to systematically create the synthetic images using the Unity game engine in which context- and variability-related parameters can be controlled. The proposed method was validated by training a deep learning-based object detection algorithm [i.e., a faster regions with convolutional neural network (R-CNN) model] with synthetic images and testing it on real images from earthwork construction sites to detect an excavator. The CNN models trained with synthetic images showed an average precision value of more than 90%; in particular, the classifier using synthetic images outperformed the one using real site images. The detection results also demonstrated an improved capability to capture the high irregularity of a construction object on images when using techniques of context customization and variability randomization. The findings from this study demonstrate the feasibility and practicality of the use of synthetic images for vision-based approaches in a construction domain. Ultimately, the proposed approach serves as an alternative way to build comprehensive image datasets for construction entities, contributing to facilitating vision-based studies on construction.

First responders often lack information and visual clues regarding interior spaces in disaster rubble, preventing efficient, effective, and safe search and rescue for victims trapped in collapsed structures. Rapidly detecting and acquiring information about the voids in collapsed structures that could contain surviving victims is critical for urban search and rescue. However, reconstructing the buried voids in three dimensions (3D) and communicating the relevant information such as buried depth and void size to first responders remain significant challenges. In response, this study proposes a see-through technique by integrating ground-penetrating radar (GPR) with interactive augmented reality (AR). The contribution of this study is twofold. First, a new method is developed to process collected GPR data to reconstruct potential voids in disaster rubble in 3D and extract the buried depth and void size from the GPR data. The coordinates of void boundaries are extracted from multiple GPR scans to generate sparse point clouds. An improved alpha-shape method is exploited to reconstruct the 3D space beneath disaster rubble from the point clouds. Second, an interactive augmented reality interface is developed to enable first responders to visualize the voids in collapsed structures in 3D together with relevant information to assist urban search and rescue. The results from simulations and pilot experiments demonstrate the feasibility and potential of the proposed methods.

Most of the existing automated compliance checking (ACC) systems are unable to fully automatically convert building-code requirements, especially requirements that have hierarchically complex semantic and syntactic structures, into computer-processable forms. The state-of-the-art rule-based ACC methods that are able to deal with complex requirements are based on information extraction and transformation rules, which are inflexible when applied to different types of regulatory documents. More research is thus needed to develop a flexible method to automatically process and understand requirements to support the downstream tasks in ACC systems, such as information matching and compliance reasoning. To address this need, this paper proposes (1) a new representation of requirements, the requirement hierarchy, and (2) a deep learning-based method to automatically extract semantic relations between words from building-code sentences, which are used to transform the sentences into such hierarchies. The proposed method was evaluated using a corpus of sentences from multiple regulatory documents. It achieved high semantic relation and requirement hierarchy extraction performance.

Existing data-driven bridge deterioration prediction methods mostly learn from abstract inventory data from a single source to predict the future conditions of bridges. Bridge inventory data [e.g., the National Bridge Inventory (NBI) data] are undoubtedly important but are not enough. They mainly describe bridge conditions using abstract, aggregated condition ratings that do not contain detailed information about bridge deficiencies and maintenance actions, thus limiting the performance of data-driven deterioration prediction and its usefulness in supporting bridge maintenance decision making. Learning from the wealth of heterogeneous (i.e., structured and unstructured) bridge data from multiple sources opens an unprecedented opportunity for enhanced data-driven bridge deterioration prediction. Such data include structured NBI and National Bridge Elements (NBE) data, structured traffic and weather data, and unstructured textual bridge inspection reports. To capitalize on this opportunity, this paper proposes a novel bridge data analytics framework, which allows for the extraction, integration, and analysis of structured and unstructured bridge data from different sources. At the cornerstone of this framework is a proposed deep learning–based bridge deterioration prediction method for analyzing and learning from the integrated bridge data to predict bridge deterioration. The proposed method includes three primary components: manifold learning for embedding the integrated bridge data into a low-dimensional dense space, cost-sensitive learning for modulating the misclassification costs to address the class imbalance in the data, and recurrent neural networks for learning from the embedded and balanced data from past years to predict the conditions of the primary bridge components (decks, superstructures, and substructures) in the next year. The method was evaluated in predicting the condition ratings of the decks, superstructures, and substructures of 2,646 bridges in the state of Washington. It achieved an average macroprecision and macrorecall of 89.9% and 85.8%, which are 15.0% and 22.4% higher than those achieved by learning from only NBI data.

Satellite and street view images are widely used in various disciplines as a source of information for understanding the built environment. In natural hazard engineering, high-quality building inventory data sets are crucial for the simulation of hazard impacts and for supporting decision-making. Screening the building stocks to gather the information for simulation and to detect potential structural defects that are vulnerable to natural hazards is a time-consuming and labor-intensive task. This paper presents an automated method for extracting building information through the use of satellite and street view images. The method is built upon a novel transformer-based deep neural network we developed. Specifically, a multidomain learning approach is employed to develop a single compact model for multiple image-based deep learning information extraction tasks using multiple data sources (e.g., satellite and street view images). Our multidomain Vision Transformer is designed as a unified architecture that can be effectively deployed for multiple classification tasks. The effectiveness of the proposed approach is demonstrated in a case study in which we use pretrained models to collect regional-scale building information that is related to natural hazard risks.

Automated specification energy compliance checking aims to check the compliance of building designs captured in building information models (BIMs) with energy requirements from contract specifications. However, automated extraction of requirements from specifications to support automated compliance checking is challenging because of the text complexities of the specifications, including hierarchically complex text structures, incomplete sentence structures, and a variety of levels of development (LODs). To address this challenge, this paper proposes a semantic information extraction method to extract building energy requirements automatically from specifications. Specifically, three new submethods are proposed to deal with these complexities: a domain-specific text splitting and stitching method, an incompleteness-aware sequential dependency extraction method, and a detail-aware LOD extraction method. The proposed information extraction method was tested in extracting thermal-insulation and lighting-power requirements from MasterFormat specifications. A performance of 96.8% recall and 97.6% precision was achieved on the testing data, which indicates the effectiveness of the proposed method.

Manual building damage inspection in a postearthquake environment is typically resource-consuming and prone to limitations based on subjectivity. A bilevel thresholding–based iterative framework is proposed to automate the delineation of concrete-spalling using three-dimensional (3D) point-cloud data. Point-level surface variation was used for damage point characterization, and two stopping conditions were defined for process automation. Synthetic building element data with varying point distribution and damage region characteristics were used for quantitative analysis and comparison with the state-of-the-art iterative refinement analysis. Comparative analysis demonstrated that the proposed algorithm rendered damaged region detection with improved completeness and correctness. In this study, use of Matthews correlation coefficient (MCC) and mean generalized intersection over union (mGIoU) metrics for performance evaluation is proposed. For low-noise rectangular-wall samples used, an average increase of 20% and 55% MCC value compared with generalized iterative refinement analysis was observed for Stopping conditions I and II, respectively. Similarly, an average increase of 31% and 21% mGIoU was observed. Furthermore, the proposed algorithm has the potential to generalize better compared with the state-of-the-art because it does not require tuning or training using ground-truth data.

Three-dimensional (3D) truck information, e.g., geometry, orientation, and position, can enable various smart construction applications such as monitoring earthwork, enhancing construction safety, and promoting productivity. Whereas stereo cameras have been explored extensively, the use of monocular vision (MV) for object 3D reconstruction still lacks substantial documentation. This study advances the field of MV-enabled 3D truck reconstruction by formulating it as an optimization problem. First, the general geometry of trucks was conceptualized and used to form a truck parametric model (TPM). Then the TPM was rendered by a computer graphics engine to generate synthetic views of the truck. Finally, an optimization algorithm is proposed to calibrate variables of the TPM progressively to maximize the alignment of the synthetic views with a target truck image. The proposed approach, called Mono-Truck, was evaluated by both lab tests and field experiments. The lab tests demonstrated an average error of 10.1%, 6.7 mm, and 0.7° in estimating the truck’s dimensions, position, and orientation, respectively. In the field experiments, Mono-Truck performed well compared with the baseline. This study contributes to the knowledge body by opening a new avenue to the monocular 3D truck reconstruction problem from an optimization perspective. The proposed approach can be generalized further to other types of construction machinery (e.g., excavators, cranes, and bulldozers) for their 3D reconstruction and smart applications.