Journal of Computing in Civil Engineering

Scanning is a key element for many use cases in the architectural, engineering, construction, and operation industry. It provides point clouds used for construction quality assurance, scan-to-building information modelling (BIM) workflows, and construction surveys. However, data acquisition using static laser scanners or photogrammetry methods is labor-intensive during scanning and postprocessing. Mobile scanners are conceptually the solution to this problem, given their potential to dramatically reduce onsite scanning effort and eliminate postprocessing work. However, current mobile mapping devices are limited to producing point clouds of relatively low resolution. In this paper, we propose a dense three-dimensional (3D) reconstruction pipeline for improving the resolution of point clouds, suitable for handheld scanners comprised of a color camera and a lidar. We fuse time-synchronized and spatially registered images and lidar sweeps using a spatial artificial intelligence (AI) method into dense scans of higher resolution, which are then used for progressive reconstruction. The novelty of our approach is that we first increase the precision and density of a bunch of individual lidar scans by inferring additional geometric constraints coming from predicted feature maps in the corresponding images. Then, we automatically register these scans together, thus reconstructing the scene progressively in an odometric manner. We built a prototypic scanner, implemented our reconstruction pipeline as a software package, and tested the whole in both indoor and outdoor case studies. The results showed that our method provided an overall noise reduction in point clouds by 11% and increased their density around six times.

Mixed reality has been envisioned as an interactive and engaging pedagogical tool for providing experiential learning experiences and potentially enhancing the acquisition of technical competencies in construction engineering education. However, to achieve seamless learning interactions and automated learning assessments, it is pertinent that the mixed reality environments are intelligent, proactive, and adaptive to students’ learning needs. With the potentials of artificial intelligence for promoting interactive, assistive, and self-reliant learning environments and the professed effectiveness of deep learning in other domains, this study explores an approach to developing a smart mixed reality environment for technical skills acquisition in construction engineering education. The study is based on the usability assessment of a previously developed mixed reality environment for learning sensing technologies such as laser scanners in the construction industry. In this study, long short-term memory (LSTM) and a hybrid LSTM and convolutional neural networks (CNN) models were trained with augmented eye-tracking data to predict students’ learning interaction difficulties, cognitive development, and experience levels. This was achieved by using predefined labels obtained from think-aloud protocols and demographic questionnaires during laser scanning activities within a mixed reality learning environment. The proposed models performed well in recognizing interaction difficulties, experienced levels, and cognitive development with F1 scores of 95.95%, 98.52%, and 99.49% respectively. The hybrid CNN-LSTM models demonstrated improved performance with an accuracy of at least 20% higher than the LSTM models but at a higher inference time. The efficacy of the models for detecting the required classes, and the potentials of the adopted data augmentation techniques for eye-tracking data were further reported. However, as the model performance increased with data size, the computational cost also increased. This study sets precedence for exploring the applications of artificial intelligence for mixed reality learning environments in construction engineering education.

Dense three-dimensional (3D) point clouds collected from rapidly evolving data acquisition techniques such as light detection and ranging (lidar) and structure from motion (SfM) multiview stereo (MVS) photogrammetry contain detailed geometric information of a scene suitable for a wide variety of applications. Among the many processes within a typical point cloud processing workflow, segmentation is often a crucial step to group points with similar attributes to support more advanced modeling and analysis. Segmenting large point cloud data sets (i.e., hundreds of millions to billions of points) can be extremely time consuming and tedious with current tools, which primarily rely on significant manual effort. While many automated methods have been proposed, the practicality, scalability, and versatility of these approaches remain a bottleneck stifling processing of large data sets. To overcome these challenges, this paper introduces a novel, generalized segmentation framework called Vo-Norvana, which incorporates a new voxelization technique, a normal variation analysis considering the positioning uncertainty of the point cloud, and a custom region growing process for clustering. The proposed framework was tested with several large-volume data sets collected in diverse scene types using several data acquisition platforms including terrestrial lidar, mobile lidar, airborne lidar, and drone-based SfM-MVS photogrammetry. In evaluating the accuracy of models generated from Vo-Norvana against manual segmentation, the average error of the position, orientation, and dimensions are 2.7 mm, 0.083°, and 0.9 mm, respectively. Over 0.2 million points per second and 36 thousand voxels per second can be achieved when segmenting an airborne lidar data set containing over 639 million points to about 1 million segments.

The digital twin (DT) approach is an enabler for data-driven decision making in architecture, engineering, construction, and operations. Various open data models that can potentially support the DT developments, at different scales and application domains, can be found in the literature. However, many implementations are based on organization-specific information management processes and proprietary data models, hindering interoperability. This article presents the process and information management approaches developed to generate a federated open data model supporting DT applications. The business process modeling notation and transaction and interaction modeling techniques are applied to formalize the federated DT data modeling framework, organized in three main phases: requirements definition, federation, validation and improvement. The proposed framework is developed adopting the cross-disciplinary and multiscale principles. A validation on the development of the federated building-level DT data model for the West Cambridge Campus DT research facility is conducted. The federated data model is used to enable DT-based asset management applications at the building and built environment levels.

Robotic building inspection is gaining popularity as a way to increase the security, productivity, and cost-effectiveness of traditional inspection tasks. Despite the development of numerous building inspection robotic platforms, their motions still require manual control. To facilitate full automation, there is a need to explore autonomous navigation strategies for building inspection robots. Although different autonomous navigation strategies have been developed in the robotics field, few of them are suitable for building structural inspection behavior. In accordance with the responsibilities of professional inspectors, the robot is required to follow the structural components within a desired distance and dynamically avoid obstacles to conduct in-depth scanning. This navigation task becomes more difficult when providing a smooth following path in special building scenarios, such as narrow corners. Motivated by this need, the present study aimed to explore autonomous navigation for building inspection robots. To save the cost of map construction, local navigation strategies, which control the robots’ travel in unknown environments, were targeted. Specifically, the objective is to develop a robust fuzzy logic controller (FLC) for wall-following behavior. The inputs are the distances within the designed interval ranges, which were measured with a 360° laser. The membership functions and the decision-making rules were designed based on robot and camera configurations, building designs, and structural inspection criteria. The outputs are the real-time angular and linear velocities. Tested in both simulation and real-world environments, the proposed FLC is able to (1) find the wall, follow the wall, conduct self-turning, and avoid obstacles in unknown building scenarios, (2) prevent wavy motions, and (3) prevent path deviations for arbitrary surfaces. The results can be employed to perform daily building inspection featuring autonomous navigation. In conclusion, the limitations of FLC are given for future study.

Tower cranes are generally used in modern construction projects due to their exclusive and flexible performance for material transportation in complex workspaces. Although traditional deep learning vision-based methods have proven their effectiveness in construction site monitoring, a shortage of realistic training images and inefficiencies in them have been an obstacle in achieving effective tower crane monitoring. To deal with this issue, this paper presents a database-free approach for timely productivity monitoring of tower crane operations. Using a synthetic model, the authors construct a research framework to effectively generate training images on deep learning–powered object detection. Additionally, an enhanced tracking algorithm collaborating with unsupervised clustering-driven postprocessing enables stable performance for tracking resources. Further, logical ontology-based activity recognition constructed on domain-specific knowledge takes place to estimate time-to-time operation of the tower crane. The curtainwall installation was selected to represent the dynamicity and variability of tower crane operations for which it has been difficult to apply traditional deep learning technologies. Results displayed that the intended framework successfully produced 300,000 training images in 1 h, about 500 times faster than human resources, and an enhanced tracker performed well even in the case of unsatisfactory detection results. It is also proved that a performance of activity analysis was acceptable to 93.58% accuracy. The results demonstrate that the proposed framework has proven remarkable data generation speed with little manual input and good performance of operational-level activity analysis. These findings can automate the monitoring process of tower cranes’ operations and the productivity of curtainwall installation, assisting construction project decision making. Furthermore, such efficient methods are less time-consuming and can handle background scene changes effortlessly, building customized models in less time and to less cost, thus buffering the practical gap of applying vision-based technologies into visually dynamic construction sites.

The traffic speed deflectometer (TSD) can collect network-level deflection data in a cost-effective and timely manner. However, an automated feature extraction method is needed to interpret such large amounts of data. Basis pursuit (BP) is one such technique: BP can sparsely decompose TSD measurements over a given basis or set of bases of the signal vector space that can represent a particular signal feature. For instance, the TSD surface deflection estimates can be reconstructed as a combination of wavelets, which represent continuously varying features like changes in pavement properties, plus pulses representing the response from structurally weak spots. Yet, the denoised measurements estimated by BP may either be riddled with several false positives (spikes that mismatch real weak spots) or be constructed out of true positive features with damped amplitude. This paper presents reweighed ${\mathrm{L}}_1$ minimization (RWL1), an enhancement to BP to both correct the dampening and discard false positives. This paper introduces RWL1 with examples from simulated data to show its advantage over vanilla BP denoising, plus a demonstration featuring real TSD measurements from a networkwide survey to demonstrate RWL1’s potential as an exploratory analysis tool to detect structurally weak locations within the pavement network worthy of further investigation at the project level.

Recent advancements in the areas of computer vision and deep learning have broadened the scope of vision-based autonomous condition assessment of civil infrastructure. However, a review of available literature suggests that most of the existing vision-based inspection techniques rely only on color information, due to the immediate availability of inexpensive high-resolution cameras. Regular cameras translate a 3D scene to a 2D space, which leads to a loss of information vis-à-vis distance and scale. This imposes a barrier to the realization of the full potential of vision-based techniques. In this regard, the structural health monitoring community is yet to benefit from the new opportunities that commercially-available low-cost depth sensors offer. This study aims at filling this knowledge gap by incorporating depth fusion into an encoder-decoder-based semantic segmentation model. Advanced computer graphics approaches are exploited to generate a database of paired RGB and depth images representing various damage categories that are commonly observed in reinforced concrete (RC) buildings, namely, spalling, spalling with exposed rebars, and severely buckled rebars. A number of encoding techniques are explored for representing the depth data. Additionally, various schemes for the data-level, feature-level, and decision-level fusions of RGB and depth data are investigated to identify the best fusion strategy. Overall, it was observed that feature-level fusion is the most effective and can enhance the performance of deep learning-based damage segmentation algorithms by up to 25% without any appreciable increase in the computation time. Moreover, a novel volumetric damage quantification approach is introduced, which is robust against perspective distortion. This study is believed to advance the frontiers of infrastructure resilience and maintenance.

Integrating drones into already dangerous construction sites could expose workers to additional risks and make construction sites more dangerous than before. Several studies have identified various safety challenges of using drones on jobsites. However, limited studies have focused on educating construction workers about the safety challenges of the drone presence in their working environment. This study is the first effort to develop a 360-degree virtual reality (360VR) environment providing a semi-immersive training experience about drone applications in construction, potential safety challenges of working with or near drones, and proposed countermeasures for their safe integration. A repeated measures study design was used to assess the effectiveness of the 360VR training, followed by a system usability assessment. The results indicate that the developed 360VR training significantly improved users’ safety knowledge about drones and that the 360VR platform proved to be a sufficiently usable delivery mechanism for this training. The main contribution of this study is to provide a better understanding of 360VR capabilities, as an emerging learning technology, to create virtual learning environments for construction education and training, and, in particular, enhancing trainees’ knowledge about working safely with drones.