Journal of Computing in Civil Engineering

Digitalization of real-world traffic scenes is a fundamental task in development of digital twins of road transportation. However, the existing digitalization approaches are either expensive in equipment costs or inapplicable to collect granular level data of traffic scenes. This study proposed a vision-based method for real-time digitalization of traffic scenes through modeling and merging the road infrastructure (static components) and road users (dynamic components) progressively. Specifically, the former is reconstructed by leveraging unmanned aerial vehicles (UAVs) and structure from motion; and the latter is digitized via using roadside surveillance videos and a new reconstruction process through applying deep learning and view geometry. Last, the digital model of the traffic scene is built by merging the digital models of static and dynamic components. A field experiment was performed to evaluate the performance of the proposed method. The results showed that the traffic scene can be successfully digitalized by the proposed method with promising accuracy, thus signifying the method’s potential for the development of the digital twins of road transportation in support of intelligent transportation applications.

Indoor localization is a prerequisite for autonomous robot applications in the construction industry. However, traditional localization techniques rely on low-level features and do not exploit construction-related semantics. They also are sensitive to environmental factors such as illumination and reflection rate, and therefore suffer from unexpected drifts and failures. This study proposes a pose graph relocalization framework that utilizes object-level landmarks to enhance a traditional visual localization system. The proposed framework builds an object landmark dictionary from Building Information Model (BIM) as prior knowledge. Then a multimodal deep neural network (DNN) is proposed to realize 3D object detection in real time, followed by instance-level object association with false-positive rejection, and relative pose estimation with outlier removal. Finally, a keyframe-based graph optimization is performed to rectify the drifts of traditional visual localization. The proposed framework was validated using a mobile platform with red-green-blue-depth (RGB-D) and inertial sensors, and the test scene was an indoor office environment with furnishing elements. The object detection model achieved 62.9% mean average precision (mAP). The relocalization technique reduced translational drifts by 64.67% and rotational drifts by 41.59% compared with traditional visual–inertial odometry.

Transit bus systems are an essential part of public transportation systems and they improve the accessibility of the communities and economical activities. On-street assets, namely bus stops, are key elements of public bus transit systems, but their condition assessment mostly depends on manual efforts that are labor-intensive and expensive. This research proposes a novel system for detection and assessment of the public transit bus stops using captured videos by on-board cameras. This system utilizes deep convolutional neural networks (DCNNs) to detect bus stop assets and semantically segment pixels in the corresponding video frames. Then, the results of these two modules are further processed to assess the connectivity of the bus stop mobility pads and accessibility level of the bus stops on snowy days. The experimental results showed promising performance of the proposed methods, which have potentials for practical applications.

Visual inspection is a common approach for collecting data over time on transportation infrastructure. However, the evaluation method in visual inspections mainly depends on a subjective metric, as well as the experience of the individual performing the task. State-space models (SSMs) enable quantifying the uncertainty associated with the inspectors when modeling the degradation of bridges based on visual inspection data. The main limitation in the existing SSM is the assumption that each inspector is unbiased, due to the high number of inspectors, which makes the problem computationally demanding for optimization approaches and prohibitive for sampling-based Bayesian estimation methods. The contributions of this paper are to enable the estimation of the inspector bias and formulate a new analytical framework that allows the estimation of the inspectors’ biases and variances using Bayesian updating. The performance of the analytical framework is verified using synthetic data where the true values are known, and validated using data from the network of bridges in Quebec province, Canada. The analyses have shown that the analytical framework has enabled reducing the computational time required for estimating the inspectors’ uncertainty and is adequate for the estimation of the inspectors’ uncertainty while maintaining a comparable performance to the gradient-based framework.

Construction safety regulations and standards contain a massive number of fall protection requirements with respect to different equipment, facilities, and operations. Automated field compliance checking aims to detect field violations of construction safety regulations for improved compliance and safety. Recent research efforts focused on automated tracking of labor and equipment toward improved violation detection and safety compliance. However, extracting and modeling safety requirements for supporting automated violation detection or safety alert systems remains highly manual. Toward addressing this gap, information extraction provides an opportunity to automatically extract requirements from construction safety regulations for comparisons with field information to detect violations (or predict and prevent violations before they occur). However, existing information extraction methods are limited in terms of their scalability and/or accuracy. To address this need, this paper proposes a deep learning–based information extraction method for automatically extracting named entities describing fall protection requirements [e.g., scaffold, horizontal direction, or 1.82 m (6 ft)] from construction safety regulations and resolving referential ambiguities. The proposed information extraction method consists of three main submethods: (1) a deep learning–based method to recognize entities from the regulations, (2) a deep learning–based method to recognize referential ambiguities in the extracted entities, and (3) a named entity normalization method to resolve these ambiguities. The proposed method was implemented and tested on 20 selected Occupational Safety and Health Administration (OSHA) sections related to fall protection. An overall information extraction precision, recall, and $F\text{-}1$ measure of 93.2%, 89.6%, and 91.1% were obtained, which indicates good information extraction performance.

Abstract

Transportation assets, including retaining walls, noise barriers, rumble strips, guardrails, guardrail anchors, and central cable barriers, are important roadside safety hardware and geotechnical structures. They need to be inventoried accurately to support asset management and ensure roadway safety. There are detection methods for some of these transportation assets, which however lack flexibility and multiclass abilities. Moreover, the potential of neural networks has not been fully utilized although some detection methods have used deep learning. This paper for the first time proposes a multiclass transportation asset detection and pixel-wise segmentation model on two-dimensional images, based on region-based convolutional neural network (Mask-RCNN) with a feature pyramid network (FPN). The scale diversity and intensive continual appearance of transportation assets are identified as the main challenges, tending to result in numerous false-positive detections. A methodology with self-attention mechanisms based on the generic region of interest extractor (GRoIE) model along with the intersection over the minimum area merging (IoMA-Merging) postprocessing algorithm was then proposed. The evaluation outcomes demonstrated that our proposed methodology, including GRoIE-global context (GRoIE-GC) with IoMA-Merging achieves the best performance with a significant improvement over baseline. The precision increased by 10.0% on detection and 10.7% on segmentation. This proposed methodology will consequently improve the accuracy of asset inventory.

Abstract

This section serves as an illustration on how we could build a unified transportation inventory as a practical application based on the discrete detection results from our methodology. More details about the methodology can be found in the following sections. For simplicity, the demo in this section is only done with detected bounding boxes of a single object. After detection and segmentation, a tracking algorithm can play an important role in associating detection on the same object across different frames. In this way, we could recognize the distinct objects in the video frames with the help of a tracking algorithm. We also find it feasible to estimate the geographical and geometric information of the object of interest. With high-accuracy and high-frequency global positioning system (GPS) information recorded by our sensing vehicle, it is practical to locate the object and estimate the width in the physical world. Height information is also accessible with either camera calibration or combination of image and light detection and ranging (LiDAR) data.

Urban excavation is an indispensable process for many construction activities such as road paving, house building, and pipe rehabilitation. However, the everincreasing complexity of underground utilities (e.g., water mains, gas lines, and sewage pipes) in urban environments challenges the safety of urban excavation, posing tremendous risks of potential collision and damage accidents. By obtaining real-time excavation information and high-fidelity simulation to evaluate safety risks, digital twin (DT) has an unexplored potential to improve urban excavation safety (UES). This research aims to investigate how a DT for urban excavation can be developed and used to improve UES. First, a multisensory solution is proposed to equip the physical excavators with the capability to precisely estimate their three-dimensional (3D) poses based on the kinematic model and social spider algorithm (SSA). Second, a building information model (BIM) of buried utilities and a 3D model of the excavators are integrated to form a dynamic virtual model that mirrors the actual excavation process. Third, based on the physical-virtual coupling DT, a real-time safety control method is proposed to proactively monitor urban excavation, dynamically assess collision risk, and timely warn against unsafe behaviors. A system prototype was developed and applied in a case study in Shandong, China. Results show that the system can precisely twin the pose of the excavator, increasing the estimation accuracy of the translation by at least 4.0 cm. The system can display the dynamic spatial position of the excavator and the buried pipes in 3D and automatically guide the excavator to operate safely in real-time, thereby avoiding potential collision accidents.

There has been a large increase in the deployment of drones in construction, and it is expected that there will be more interaction between construction professionals and drones on construction sites. Working in close proximity to these flying robots raises novel occupational safety and health issues that might make the construction industry more dangerous than before. While there has been a significant amount of research about how drone technology can be used for specific construction-related tasks, further research is required to empirically evaluate the potential safety impacts of working with or near drones on the jobsite. This study employs virtual reality technology to study the health and safety challenges of construction professionals working at different distances from drones. A user-centered experimental approach was adopted, and various subjective and objective measures were used to assess the impact of working with drones at different distances on the physiological, emotional, and attentional states of construction professionals, as well as their attitudes toward drones. The results revealed that drone presence has some physiological impacts and reduces construction professionals’ attention on the tasks at hand. Moreover, the results suggested that drones operating at some distance cause more distraction than drones operating in close proximity. No evidence was found indicating that drones either cause significant psychological or emotional distress or affect construction professionals’ attitude toward them. The safety knowledge generated through this study will not only help in better understanding safe drone integration in construction but could also lead to formalizing comprehensive regulations for drone use in this industry.

Abstract

The rules in building codes are composed of language, logic structure, and the embedded knowledge that requires further articulation from rule experts. This greatly increases the complexity of rule interpretation. The conceptual graph (CG) is used as a graph-based representation method to capture such knowledge and transfer it to other actors in the compliance checking process. However, due to its subjectivity, it is difficult to ensure the quality of the CG to achieve minimal knowledge loss. This paper proposes a method using reusable CG patterns to reduce the knowledge loss caused by the opacity of concepts in rules, hence better facilitating knowledge transfer using CGs. The rules and their corresponding CGs are analyzed and compared. An assessment framework is used to assess CGs for correctness, completeness, complexity, and comprehensibility. A group of reusable CG patterns is then generated based on the analysis results of concepts in existing CGs. In a case study, rules selected from two types of building codes are used to validate the proposed methods. The results successfully demonstrate the effectiveness of the CG in facilitating knowledge transfer. Moreover, the comparison of quality between original CGs and improved CGs shows that using reusable CG patterns can effectively ease the creation of new CGs and improve its consistency.

Practical Applications

Automated code compliance checking requires close collaboration between experts and programmers to encode building codes in a computer-executable format for real-world implementation. The challenge is that experts possess the domain knowledge, while programmers are familiar with implementation. This communication gap between experts and programmers may result in inconsistencies or errors in the automated code checking system. To bridge this gap, a method that can facilitate communication between experts and programmers called the conceptual graph (CG) is used. However, the current CG still requires significant effort, and it may be subjectively interpreted. This paper introduces reusable CGs to improve clarity and reduce the effort needed. The paper elaborates how such reusable CGs may be developed and how it may be assessed in real situations. A validation case study is conducted on two types of building codes, and results indicate the reusable CG is a viable tool to improve clarity and reduce potential errors. The contributions of this paper will benefit the current state of automated code compliance checking by introducing a method to clearly represent concepts within building codes, thereby reducing potential rework arising from ambiguous contextual concepts in the building code.