Computing in Civil Engineering 2023: Data, Sensing, and Analytics

Bridge inspection images capture a wealth of information and details about bridge conditions. This study proposes a method to interpret on-site bridge inspection images by generating human-readable descriptive sentences. The resulting text can be formed into a bridge inspection report to aid and expedite the bridge inspection process for bridge engineers; and the extracted information can be further exploited to support bridge deterioration prediction and maintenance decision making. This is, however, a challenging task that combines computer vision and natural language processing. First, it not only requires object detection/segmentation from the bridge inspection images but also demands a grasp of the relationships between the recognized objects. Second, human-readable sentences need to be generated based on the extracted information from the images. Third, the available bridge image-text data pairs, which can be used for training, are quite limited and highly noisy. To address these gaps, this paper proposes a deep learning-based model for generating free-form human-readable descriptive sentences of the bridge conditions, which leverages bootstrapping language-image pre-training (BLIP) and its vision-language pre-training data from the web. This paper discusses the proposed model and its performance results.

Implementing geometric digital twins for roads can increase productivity and help streamline predictive maintenance. However, the related costs outweigh foreseen financial gain due to the manual work needed in scene recognition and modelling. The foremost step in automatically creating such digital twins is segmenting objects in 3D point clouds. Researchers have done extensive work to tackle this challenge, with deep learning solutions often preferable for solving multi-class problems. Yet, only several large-sized road objects get high-quality detection results. Other essential but rather smaller objects represent a further challenge. We utilise common properties of roads and road assets in a lightweight deep learning network training and data pre-processing to facilitate a broader range of highly detected objects. We show how these priors improve detection performance by a significant margin. Our work contributes to the improved automatic detection of most road object types, enabling automatic digitisation for roads and reducing related costs.

Construction tasks involve extensive and repetitive physically demanding activities, which results in a higher risk of work-related musculoskeletal disorders (WMSDs) compared to other industries. Thus, assessing construction workers’ postural ergonomics during construction occupational tasks is critical to evaluating workers’ postural stresses and reducing the potential risk of WMSDs. Recent developments in machine learning-based computer vision methods have attracted an increasing attention of construction researchers as it is an effective tool for assessing postural ergonomics. However, existing computer vision-based ergonomic assessments in the construction research field are still mainly based on multiple-view motion tracking systems or fixed-position stand-alone cameras. These types of motion-tracking methods are limited by resource-expensive and serious occlusion issues. As an alternative approach, this paper proposes an economical and scalable machine learning enabled egocentric postural ergonomic assessment (EPEA) system that integrates the fisheye camera mounted to the hard hat to identify and assess workers’ postural ergonomics via a convolutional 3D pose estimation neural network. We tested the EPEA with multiple construction operational tasks. Results show that the EPEA can correctly identify users’ key joint parts and classify different postures. The results confirmed the usability and feasibility of the proposed system, and it has the potential to help construction workers to identify the potential WMSDs risks during repetitive and forceful construction works.

With the increasing awareness of sustainable development, reusing and recycling the waste materials are gaining more and more importance on construction sites. The semantic segmentation techniques provide a powerful tool to identify the construction recycling materials automatically for their subsequent management. Among these segmentation techniques, recently proposed transformer-based architecture could understand long-range dependencies and learn highly expressive representations. This paper presented the transformer-based semantic segmentation for recycling materials in construction. It started with an evaluation of two state-of-the-art transformer-based architecture types. The selected architecture types included Twins Transformer and K-Net. Further, the ensemble learning strategy of model averaging was implemented to improve the segmentation performance. The idea of model averaging was to average different checkpoint weights for each transformer-based architecture along a single training trajectory. To evaluate the segmentation performance, a novel dataset containing five classes of recycling materials on construction sites was created and employed as a benchmark. The evaluation results indicated the ensemble models of Twins Transformer and K-Net achieved the mean Intersection over Union (mIoU) of 80.5% and 82.2%, separately, which demonstrated superior performance on segmenting recycling materials in construction. Our segmentation model won the first place in VIMS-IAARC Joint Datathon 2022 Competition.

The growing emphasis on smart cities and digital twins has heightened the significance of 3D geospatial information. However, 3D urban building modeling, considered a significant object of interest, typically involves time-consuming manual labor. Several studies have been conducted to reconstruct 3D urban buildings using multidimensional and multitemporal. However, several constraints, such as data availability, view angles, occlusions, and texture detail, affect the accuracy and completeness of 3D models. Furthermore, inconsistencies across different datasets pose difficulties in comparing results across studies and developing generalized methods. In this study, our objective aims to utilize neural rendering techniques to reconstruct a 3D model of buildings in urban areas using imagery captured from unmanned aerial vehicle (UAV). Neural radiance field (NeRF) synthesized a continuous scene of unknown direction for areas not covered by the UAV images. As a result, the proposed method provided a more comprehensive view to supplement the occlusions in the photogrammetry-based point cloud. The results were quantitatively evaluated and indicated that the proposed method could be a feasible complementary solution for photogrammetry-based 3D building modeling.

4D (3D + time) building information modeling (BIM) brings important benefits to the practice of project planning and controls in construction. Despite well-documented benefits, the level of technical effort necessary to create 4D BIM by manually tying in BIM elements with schedule activities has been a significant barrier to its wider adoption. To address it, this paper explores the possibility of identifying <object, location, responsible party, means&method> in each schedule activity based on ASTM’s Uniformat classification and then automatically mapping it to the most relevant BIM elements in its corresponding work location. Specifically, the feasibility of training a transformer-based natural language processing model to infer Uniformat classification of a schedule activity is explored. Results from testing the proposed model on four real-word project schedules show the potential for a path forward toward automated 4D BIM creation. Insights from the performance of this model and a path forward for model improvements are discussed.

An automated intelligent building code compliance methodology using a flexible and adaptable BIM authoring tool is presented. The method is designed to use minimal computing resources in different contexts of code applications. The innovation utilizes the complete corpus of the building code as input for compliance checking. Current research developments use specific sections of the code that focus only on parts of the building code without making inferences to subsections or other building codes. The approach uses natural language processing techniques to generate an easy-to-process output [simplified comma-separated value (CSV) file] for rapid interpretation and further manipulation for review and modifications of the generated ruleset, thereby increasing the accuracy of the rule when needed. Simultaneously, the approach prevents the recreation of rulesets when inferred sections were previously generated, to ensure the integrity of IBC sections and subsections structure. This paper presents the data extraction and processing approach that generates computer-readable rules using building code paragraphs. Future work will incorporate methods to fully process tables and figures to generate a comprehensive check and inference in compliance.

Poor quality of bid documents impacts bidders’ cost estimates and bid price, and further leads to claims/dispute during the project implementation. As the problems and ambiguities in the bid documents are addressed by the pre-bid request for information (RFI), pre-bid RFI analysis helps determine major ambiguities in the bid documents that should be clarified for enhancing bid documents’ quality. This study uses pre-bid RFI as the principal data for obtaining good quality bid documents. Natural language processing (NLP) is used to pre-process/transform unstructured raw text data, and machine learning-driven classifier is applied for the classification of collected pre-bid RFI. The proposed method can automatically identify the critical pre-bid RFI that can lead to significant revision in the original bid documents. This study can contribute to efficient pre-bid RFI management that facilitates bidding process and can improve bid document quality for similar projects to be carried out in the future.

Existing maintenance management systems of healthcare facilities tend to have vague facility failure classifications and rely on manual request interpretations and task assignments, which are time-consuming and human error-prone. To improve its effectiveness, in this paper, the authors utilize natural language processing (NLP) techniques to automate the interrogation of the textual data of maintenance work orders (MWOs), to reduce manual efforts, and to improve efficiency. Five word-splitters (i.e., FoolNLTK, Jieba, Stanford NLP, SnowNLP, and THULAC) and four machine learning algorithms (i.e., decision tree, random forest, eXtreme Gradient Boosting, and support vector machines) were configured to automate failure identification and staff assignment from MWOs. Experimental results on multiple MWOs datasets showed a 0.80 accuracy for failure identification and a 0.83 accuracy for staff assignment were achieved, indicating a promising multiclass classification performance, which can help transit maintenance staff assignment from a labor-intensive process to a more automated one, improving overall efficiency.

We present an automated scan-to-BIM pipeline that simplifies the 3D building object recognition problem into a 2D recognition problem. We used the Habitat Matterport 3D Dataset (HM3D) for training wall detection model. The weakly supervised learning is conducted since we used the noisy depth-projected annotation. We isolated individual building levels and projected the points to 2D along the Z-axis (up/down). The architectural components’ recognition system detects walls within the plan view projection of the indoor point cloud. We compare the performance metric of validation on noisy annotation with human-labeled annotation and analyze the wall inference results from visualization. We assume the human-labeled annotation as ground truth and noisy annotation is prediction to calculate the average precision. The average precision values are compared to the neural network performance. We anticipate this experiment can provide a feasible weak supervision method for simplifying 3D digital model creation from scan data.

Investigating road conditions is an essential task to maintain service life, ensure safety, and preserve smoothness. Digital road surface information is gaining interest as the main database for planning, design, and inspection. However, the implementation of this still confronts several challenges, such as automated surface extraction, typical geometric features, large scene size, and so on. Although active research on point cloud-based automated road surface extraction has addressed this issue, previous studies mainly relied on experts’ perceptions to decide on the result produced by the selected semantic classification and estimated value, consequently requiring trial and error. This study proposes a U-net image segmentation model-aided method to automatically extract 3D point cloud road surface in typical circumstances generated by UAV images. The proposed method starts by developing a road curb detecting model for geometric 3D spatial data with U-net. Then, the road surface is derived by using geometric feature extraction and machine learning classification algorithms. The outcome is assessed by comparing it with manually classified points and the surface produced by a method from a previous study. The proposed method is capable of extracting road surfaces while reducing human involvement, elevating the road extraction from an unstructured 3D point cloud to a more fully automated approach.

While helpful for engineering applications, digital models representing the as-is status of the built environment are rarely available and costly to create using conventional methods. Commonly, editable and preferably parametric model geometries are preferred over less easy-to-process, triangulated meshes where possible; additional semantic information beyond the geometry is required in almost any case. We propose an end-to-end method starting from conventional laser-scanned point clouds including RGB color information: the captured data is processed using semantic and instance segmentation and model fitting first to identify semantic clusters and object instances, and then selected structural and MEP elements are reconstructed using geometric primitives and procedural geometric operations such as sweeps to generate meaningful, ready-to-use models. We describe all steps individually, along with a prototypical implementation in which we use state-of-the-art segmentation and reconstruction methods on a real-world dataset collected by the authors. Intermediate and final results are showcased and critically discussed.

Departments of transportation (DOTs) must regularly inspect and maintain culverts to prevent their failure. However, DOTs cannot conduct culvert inspections on a regular basis due to their limited maintenance budgets. In this study, we proposed an intelligent approach for inspecting the condition of culverts. We developed a hybrid model based on the eXtreme Gradient Boosting (XGBoost) algorithm to predict culvert condition ratings using culvert inventory data from the states of Utah, Vermont, Ohio, and Colorado. We applied a swarm intelligence algorithm—Grey Wolf Optimizer (GWO)—to fine-tune the XGBoost model’s hyper-parameters and enhance the model’s prediction accuracy. Based on the results, the GWO-XGBoost algorithm outperformed the previous algorithms employed in the literature for predicting culvert condition ratings. This study showed that DOTs could minimize culvert inspection costs while maximizing their culvert network quality by taking advantage of predictive analytics models trained on their current inventory datasets.

Green buildings (GB) aim to minimize the environmental impact of construction and operation activities. The adoption of building information modeling (BIM), the Internet of Things (IoT), and data mining (DM) presents promising opportunities for managing GBs with high energy efficiency. However, research on integrating these digital technologies to enhance GB energy resilience requires further improvement. This study reviews the applications of these three digital technologies in GB. Subsequently, an integration framework is constructed using semantic technologies to enable data exchange among BIM, IoT, and DM. Additionally, this paper illustrates how digital technologies can improve energy resilience in GB. Finally, the limitations and challenges of enhancing GB energy resilience are discussed. This research provides valuable insights into the practical application of digital technologies to advance energy efficiency in GBs. The proposed conceptual integration framework is a foundation for a digital solution that improves GB energy resilience.

Modeling, predicting, and controlling the interactions between humans and civil infrastructure systems can simultaneously improve the operational efficiency of infrastructure systems and the satisfaction of infrastructure users. The first step toward achieving this goal is to model human-to-infrastructure interaction, which in most cases is driven by human movements (e.g., moving from an origin location to a destination requires using the transportation infrastructure connecting the two). To this end, this paper aims to conduct a machine learning-based data-driven analysis to rank the importance of factors influencing human movement purposes, thereby identifying highly influential factors to support subsequent human-to-infrastructure interaction modeling. The research methodology included: (1) representing movement instances using spatial and land use, temporal, and demographic features; and (2) conducting feature ranking per movement purpose type using the logistic regression algorithm. As a preliminary work, this paper focuses on presenting the research methodology, and analyzing and discussing the feature ranking results.

During the COVID-19 pandemic, engineering control emerged as an efficient method for isolating people from hazards through physical or mechanical means. To enhance engineering control, understanding the relationship between occupants’ movement and building morphology before and during the pandemic is crucial. Although previous research has explored the impact of building elements on occupants’ route choices and indoor movements in emergencies, everyday scenarios remain understudied. This study addresses this gap by analyzing a Wi-Fi dataset to track occupants’ movements and a building dataset to obtain morphological features of selected academic buildings. Using the K-means clustering method, the relationship between 12 building features and occupancy was examined. Trajectory data at different pandemic stages was visualized to emphasize changes in occupants’ movements following engineering control measures. The results reveal that building features, such as floor area, corridor area, and room number, significantly influence occupants’ indoor movement.

With the rapid expansion of underground railway systems and the increase in public transportation ridership, short-term metro passenger flow forecasting has become one of the prevalent topics in transportation planning, infrastructure construction and improvement, and underground railway monitoring and management. This paper proposes a short-term metro passenger prediction approach based on long short-term memory neural network (LSTM NN). The big data of historical metro passenger flow, the holiday information, and the characteristic of stations are taken into consideration. The prediction performance is evaluated by mean absolute error (MAE) and root mean squared error (RMSE). Experiments are conducted based on Hong Kong Mass Transit Railway (MTR) data, and the result indicates that the proposed model has a higher accuracy than baseline models. The novel model provides an option for stakeholders to achieve a high-performance prediction of metro passenger flow. By obtaining future passenger flow data in advance, the authorities can make appropriate decisions on real-time underground railway system management and deliver timely responses to unexpected events.

Thanks to emerging technologies and increased computational power, a new wave of smart buildings is gaining attraction with a specific focus on the personal experience of individual occupants. Real-time monitoring and estimation of human states are key to achieving individual satisfaction. This paper proposes a framework to predict and track the real-time state of each individual and assist with the decision-making of room assignment and environment control. The core idea of the framework is to distinguish individuals by an idea of the personal database named human digital ID (DID). Two case studies are given to demonstrate the adoption of the framework. They show how different indoor well-being such as thermal comfort and work engagement can be incorporated into the framework. The results show improvement in the occupants’ well-being after adopting the framework. The framework is scalable in every aspect and can serve as a fundamental concept for future smart buildings.

Field-of-view calibration is essential for establishing the relationship between 2D image coordinates inside the camera and 3D real-world coordinates of the traffic scene. For many surveillance-based traffic applications, the field-of-view calibration involves precisely extracting the three orthogonal vanishing points. However, many traffic scenes lack parallel lines along all three dominant directions, making it difficult to successfully calibrate using the present methods. This study proposed a novel method for estimating the three mutually orthogonal vanishing points in traffic scenes. To determine the dominant directions of the real-world coordinate frame, this method exploits the visual features of both the road environment and moving vehicles and avoids the need for parameter tuning through trial and error in different scenarios. To evaluate the performance of the proposed method, laboratory tests were conducted. The outcomes demonstrated the potential of the method in traffic scenes with scarce parallel line features.

Design reviews are critical for construction projects to reduce costly reworks and future conflicts. However, this is a challenging task due to uncertainties during the initial stages of a project, which can lead to numerous requests for information (RFIs). With the recent advancements in language models and computer vision, a large volume of historical RFIs can be leveraged to aid design reviews. This study proposes a novel framework using natural language processing, ChatGPT API, and computer vision techniques to identify the RFIs from previous projects that are more likely to reoccur in the project under review. The framework was tested using RFI data from 19 healthcare construction projects, and a web application was used to evaluate user experiences with the tool. Successful implementation of the proposed framework could reduce the number of RFIs, change orders, rework by contractors, and the likelihood of time and cost overruns for construction projects.

Computational thinking-supported educational training can equip the construction workforce with the necessary skills and knowledge to implement sensing technologies and perform sensor data analytics aimed at enhancing construction operations. However, construction students struggle to understand the computational concepts and workflows required to translate low-level sensor data into knowledge for supporting decisions. This study compared the cognitive load of construction students performing similar data analytics tasks using an end-user programming environment (EUP) to conventional MS Excel. Comparative analysis using descriptive and inferential statistics demonstrated that participants using EUP perceived lower levels of cognitive loads and more positive experiences than those who used the conventional techniques. The study’s findings will advance the understanding of the potential cognitive effects of adopting EUPs for construction sensor data analytics. This study contributes to the cognitive theory of multimedia learning by illustrating how multimodal programming environments can influence learners’ cognitive demands.

We present statistical modeling of wind speed and direction in the New York Bight region using a Weibull distribution for wind speed and a mixture of von Mises distributions for wind direction. The historic wind data for the years 2019−2021 with a 6-min sampling frequency are divided into two half-yearly intervals based on wind power, representing different seasonal behaviors. The parameters of von Mises distributions are estimated using four different metaheuristic optimization algorithms. The suitability of the distributions is judged based on the Pearson correlation coefficient, which indicates good fits for the proposed models. The results demonstrate the flexibility of the proposed models in representing the probability density function of wind direction regimes in zones with several modes or prevailing wind directions. This research provides valuable insights into the evaluation process of wind energy resources of the New York/New Jersey Bight region, including wind farm layout optimization.

Compared to other industries, the construction sector shows low productivity worldwide. However, holistic, data-oriented methods for investigating potential bottlenecks within the as-performed construction stage are scarce. Our research presents an approach to acquiring raw data from job sites and its subsequent processing to high-level information. First, images were captured over a period of one year in high frequency using multiple crane cameras. Second, an end-to-end deep learning based approach was developed to derive and link information about construction activities, covering the classification and localization of specific on-site objects. This information was subsequently integrated into a knowledge graph. Finally, additional data sources like the weather were exploited to interpret different on-site scenarios. We demonstrate that construction-related activities like working times can be detected. The presented approach provides a significant step toward exposing correlations on construction sites by using multiple data processing steps and showcases the possibility of identifying process patterns.

Machine learning model evaluations in academic literature tend to fixate on summary performance metrics. However, the performance of a model architecture depends heavily on the context it is evaluated in. Superficial summary performance metrics fail to encapsulate this context and provide readers with virtually no understanding or ability to predict performance in new contexts. Slicing is a type of fine-grained machine learning evaluation, where data is separated into subsets and the performance on each subset is evaluated. Here, we demonstrate slice-based evaluation on a computer vision task, excavator detection in 2D color images. Critical slices are identified using metadata augmentation and feature-space clustering. Slices are created based on features including lighting, excavator color, weather, distance from camera, occlusion, view perspective, number of excavators, presence of other equipment, and environment. Meaningful performance trends identified using slice-based evaluation provide readers with insight about the task’s inherent hardness and training dataset imbalance. Slice-based evaluation should become standard practice in reporting machine learning method results in the academic literature.

Current façade safety inspections are done manually by relying on visual cues and have proven to be labor-intensive and unsafe. Deep learning (DL) techniques, particularly, convolutional neural network-based approaches, demonstrated great success in automated defect detection on surfaces. Yet, these models need large amounts of images for high performance. Collecting and labeling a large number of façade defect images is expensive and time-consuming, inducing data scarcity and imbalanced dataset problems. Previous studies aimed to improve DL models’ accuracy with various methods but by training them with imbalanced and relatively small façade defect datasets. This study introduces a data augmentation approach using a deep convolutional generative adversarial network (DCGAN) to generate synthetic images of façade defects, addressing these issues. We evaluated the model using Fréchet inception distance (FID) and visual inspection. Our study provides a data augmentation method to generate a much-needed dataset for training automated façade defect classification models.

While driving is a mentally demanding task, driving on roads that include maintaining and repairing construction activities can heighten the complexity of this task. Within work zones, drivers may fail to properly process road construction signs, which can highly threaten road construction workers’ safety. However, drivers’ control over the vehicle and the inhibitory control ability are important to prevent accidents. While previous studies investigated inhibitory control among drivers, they lack natural settings with the presence of an auditory stimulus. To bridge this gap, this study investigated expert and novice drivers’ neural activity and inhibitory control while intruding into a work zone equipped with auditory intrusion alert technology. Results showed that experts had higher neural activity within the prefrontal cortex while showing better response inhibition. Such knowledge can increase road safety professionals’ knowledge regarding the interventions that are needed to increase work zones’ safety.

A truck-mounted attenuator (TMA) is designed to reduce the impacts of crashes and protect workers in mobile work zones. Because of the speed difference between the TMA truck (usually stationary or with very low speed) and surrounding traffic flows, the probability of TMA being crashed into is relatively high. In Indiana, 152 TMA-related accidents were recorded in crash reports between 2016 and 2021. A natural language processing approach was adopted in this study to identify the latent themes of TMA-related crashes, and the co-occurrence of themes that led to these crashes. More specifically, the proposed method started with extracting n-grams (i.e., unigrams, bigrams, and trigrams) from each TMA-related crash report. Then the correlation between each n-gram was calculated and visualized by the network topology. Next, two hypotheses to reduce the impacts of TMA-related accidents were proposed based on the network topology. The hypotheses were: (1) the implementation of autonomous TMA (ATMA) can reduce the number of injuries; and (2) trajectory planning and controlling at the merging area can reduce the number of accidents. The findings of the study include the factors leading to TMA-related crashes and the potential solutions to reduce these TMA-related crashes with the consideration of the recent development of connected and autonomous vehicles (CAV) and related technologies.

Work zones for road construction, maintenance, and enhancement activities affect traffic safety and reduce road network efficiency. This study aims to investigate the effect of work zone layout on traffic safety by designing and conducting a driving simulator experiment. Twenty participants were recruited to drive through three different scenarios, including a scenario without heavy equipment, a scenario with heavy equipment near the closed lane at the beginning of the work zone (position 1), and a scenario with heavy equipment away from the closed lane at the end of the work zone (position 2). The driving behaviors of the participants, including longitudinal velocity and lateral offset, were monitored and collected. The results suggest that heavy equipment in construction work zone should be placed at position 2 from a safety perspective.

In the practical implementation of road design, engineers must make multiple decisions during the design process, which is often neither formalized nor automated, thereby resulting in a repetitive cycle of design, analysis, and modification processes when designing and constructing road projects. Researchers in the architecture, engineering, and construction domain have explored design optimization from many perspectives. This study presents a systematic review of existing literature on road design optimization, summarizing the status quo of existing studies in terms of optimization disciplines, objectives, and methods. The review highlights that existing research on road design optimization has mainly focused on optimizing traffic network, geometry design, and material performance with certain objectives, using various approaches. Future topics could include optimization methods that integrate more analysis processes from different disciplines and incorporate multidisciplinary expertise into building information modeling (BIM) with parametric representation and rich semantic information.

Technological advancement transforms construction jobsites into more intelligent systems. Among the technologies, robotics relies on environmental perception, such as the motion and dynamics of interacting objects; a digital twin expands its capabilities by collecting real-time data of the physical twin, including spatial and physical properties. Given booming attention and efforts in such technologies, there lacks a non-invasive approach to collect jobsite objects’ 3D location and orientation, which is a required step for physically based modeling. As an initial effort, this paper proposes a vision-based approach to estimate the 6-DoF object pose of construction jobsite objects from a single image while leveraging deep learning. Tests are performed on a brick and a concrete block of cuboid shape. The evaluation against ground truth data, collected by an RGB-D camera, presents a certain potential for utilizing a non-invasive perception approach to collect jobsite objects’ advanced kinematic data for extended capabilities of intelligent systems.

Crash scene reconstruction is essential for reverse-engineering the factors of a crash scene to determine the cause of a crash. It requires automated information extraction (IE) from the textual crash report narratives and their formalization in a computer-processable representation. Natural language processing (NLP) is a powerful computational tool to process texts. This paper presents a dependency parsing (DP)-based NLP system for automated IE of crash events information from crash report narratives. DP-based rules and patterns were leveraged for defining relations between subjects and objects to support the IE algorithm. The proposed IE system was tested on 50 reports collected from the Southeast Michigan Council of Governments (SEMCOG) traffic crash database, which achieved an overall 94.9% precision, 90.2% recall, and 92.5% F1-score. A parallel experiment was conducted with ChatGPT to extract information from crash narratives, where an 88.0% precision and 88.0% recall were obtained.

The queue length is a critical performance indicator for traffic networks, traditionally measured using sensors such as induction loops, piezoelectric sensors, and pneumatic tubes. However, these sensors often fail to provide accurate information about traffic flow during periods of congestion, leading to potential inaccuracies in measuring queue length. To overcome these challenges, this paper proposes an image processing technique combining object detection with You Only Look Once (YOLO) followed by simple online and realtime tracking with deep association metric (DeepSORT) to estimate the queue length and delay. To test the performance of the given framework, 50 samples of 1, 5, 10, and 15 min long videos were randomly extracted from Utah highways. The proposed method estimated the queue length and queue delay with 72.56% and 69.08% accuracy, respectively. This study has significant implications for estimating ramp parameters and improving ramp performance.

In order to prevent scaffolding accidents at construction sites, field managers have manually inspected safety regulations. A 3D reconstruction technology enables effective safety management through the automatic monitoring of scaffolds. However, it is difficult to obtain meaningful data for scaffold monitoring in the actual installation process because a safety net is installed. To address this issue, we propose a sensor fusion-based point cloud registration method considering the installation of safety nets for scaffold 3D reconstruction. The method consists of three steps: (1) camera and mobile LiDAR calibration, (2) identification of scaffold installation phases with deep learning-based 2D detection, and (3) registration of point cloud data without safety nets. The proposed method has demonstrated the potential to generate registered point clouds without safety nets for scaffold 3D reconstruction.

Deep learning recognition models have been widely studied to recognize construction objects from site images. These methods require high volumes of quality data from human-made annotations to achieve moderate performance. Yet, manual annotation of images is time-consuming and error-prone, limiting ground-truth quality and performance of the resulting recognition models. To address such inefficiencies, automatic data generation and labeling of synthetic images have been recently explored. The quality of this type of data and how it can be effectively incorporated into a machine learning training pipeline still deserved further evaluation such that these models can scale up to real-world applications such as computer vision-based construction progress monitoring. This paper aims to re-evaluate synthetic image generation using physics-based simulation environments and 3D BIM. Using experimental results, insights are shared on how the quality of synthetic data can impact the performance of the trained recognition models where synthetic images with repetitive architectural or MEP patterns or 4D BIM with low-LOD engineering disciplines are used and when real and synthetic images are integrated into the same training pipeline. A path forward for improving the performance of the synthetic data and specifically mitigating the impacts of low LODs in BIM engineering discipline are discussed.

There were some technical limitations, but there was an advancement in the photogrammetric approach for point cloud generation recently. Thus, the authors attempted to verify an advancement of the photogrammetric approach for the geometric quality assessment of a modular house and documented the challenges and lessons learned from developing a 3D point cloud model of an existing modular house using an existing commercially available software that uses photogrammetric approach. The documented lessons learned are: (1) obstructions by existing objects on the site; (2) insufficient overlapping of pictures; (3) low-quality GPS data; (4) movement of objects/workers; (5) different exposures; and (6) issues with combining photos taken by UAV and a handheld camera, and others. The findings will help practitioners to avoid the same mistakes/difficulties and service providers and scholars on the photogrammetric approach to conduct further research and improve the technology by resolving the raised issues.

Modular construction has recently gained interest as a transformative construction method. In this method, a large portion of the construction is performed inside factories, where processes are fast-paced and interdependent; therefore, any deviation from the schedule can delay the production. Such deviations are frequent in modular factories due to the labor-intensive nature of the tasks. This propagation of delays can be mitigated by continuously monitoring each process; however, current manual monitoring methods are laborious, and recently proposed contact sensor-based methods are intrusive to the work. In addition, recent computer vision-based monitoring methods inside factories are limited to detection algorithms that fail to provide the pixel-level accuracy required for assembly progress monitoring in highly occluded factory scenes, and they require a large number of manual annotations. Therefore, this paper proposes a method to monitor the installation of subassemblies in modular construction factories using mask R-CNN instance segmentation and improves the data efficiency of the model using a copy-paste augmentation method. This method was validated on the CCTV videos captured from a modular construction factory in the US, resulting in a 9% mAP improvement in segmentation.

In construction environments, transient construction objects such as moving vehicles, debris, materials, and construction tools can pose safety hazards and impede visibility. Monitoring their movement in video frames is crucial for effective project management, but it is challenging due to the complexity and dynamism of construction sites. This study proposes a two-step method to address this challenge by detecting, tracking, and segmenting transient construction objects in video frames. A synthetic dataset is used to train a deep learning-based detection model, and coarse and fine tracking models are then applied to track and segment objects based on detection results. The preliminary results reveal the significant performance of the proposed detection model, achieving an average precision of 75.75% at an IoU threshold of 0.5. It is demonstrated that the proposed detection method enables the detection of transient construction objects using synthetic datasets, reducing the need for manually annotating additional construction-related datasets. Besides, the produced segmentation results provide detailed information about the location and shape of objects, enabling enhanced safety control and analysis.

Accurate volume measurements of bulk construction materials are crucial to create reliable cost estimates and schedules for construction projects. The traditional method of bulk volume measurements uses surveying GPS to capture the coordinates of perimeter and surface points at appropriate locations of the bulk pile. These geographical points are then used to establish a triangulated irregular network of the bulk pile surface. While the surveying GPS method is a convenient approach to quickly estimate the bulk volume, its accuracy is questionable because the limited amount of perimeter and surface points captured on the pile is not able to precisely represent its actual geometry. In addition, this method requires a GPS operator to physically climb onto the bulk pile. This paper presents the workflow and findings of using the UAS-mounted LIDAR approach. A customized commercial UAS with a professional LIDAR sensor was used to capture the entire surface of a construction material pile at low altitudes, and the results from the LIDAR sensor were compared with volume measurements using the surveying GPS method. Using UAS-mounted LIDAR will not only significantly improve the work efficiency for construction bulk volume measurements but also provide highly accurate results compared with the traditional surveying GPS method.

Vision-based sensors have been widely used in reality capture and the corresponding scene understanding tasks such as object detection. Given the increasing complexity of built environments, geometric features from the raw scanning data can become too vague for effective object detection. One example challenge is stacked object recognition. Previous methods propose to use high-resolution sensors to capture more detailed geometry information to highlight the boundaries between adjacent objects, which increases the deployment cost and computing needs. This paper proposes a novel data augmentation and voting method for stacked object detection with only low-cost sparse LiDAR sensors. First, a modified Lidar Odometry and Mapping (LOAM) method is used to register and augment raw point cloud data from multiple scans in real time. Then a voxel-based density voting method is applied to centralize the points in an enhanced scan for more accurate clustering. Finally, the clustered points are grouped and applied to generate 3D bounding boxes for object boundary identification. A pilot test was performed to show the improved results of the proposed methods for both reality capture and clustering.

Due to the shift towards green coal industries, the decrease in fly ash production has affected concrete industries. It is crucial for them to identify the availability of fly ash in landfills/storage facilities to meet their demand if the supply decreases. However, there is no standard procedure to quantify fly ash in existing landfills. Previous research used UAVs and laser scanning to estimate stockpile volume but didn’t address the feasibility of the technologies on a broader scale. This paper aims to develop a standard procedure for using UAVs and laser scanning to estimate fly ash availability and analyze their feasibility in terms of cost, safety, quality, and ease of use. The study compares the results to select the best method for stockpile volume estimation. Findings will provide a guideline for industry professionals to choose a suitable technology for quantitative estimation of fly ash on a large scale and encourage research into emerging technologies.

Digital twins (DTs) can support the operation and maintenance process of bridges by providing a digital model representing the actual asset in reality. The underlying semantic-geometric model of bridges can be created from point cloud data (PCD), obtained by laser scanning or photogrammetry. The bridge PCD, however, needs to be processed and abstracted to a parametric model to handle geometric updates. Today, this process is conducted manually which in turn increases the geometric modeling costs. This paper aims to automate semantic segmentation and parametric modeling as essential steps in the geometric modeling of bridges. The point cloud of bridges is semantically segmented first through a deep-learning model. The value of parameters is then extracted by a heuristic optimization algorithm. Finally, the model of the entire bridge is created. The results of the paper show that the geometric modeling process of bridges can be automated to a large extent through computational methods.

Mobile lidar is a remote sensing technique adopted by several transportation agencies to collect dense 3D point clouds with high accuracy and efficiency on a regular basis for a wide variety of applications (e.g., asset management, civil design, etc.). Leveraging mobile lidar technology can substantially improve the conventional procedure of asset management. However, there is still a need for automatic tools to process these 3D point clouds effectively and efficiently. In this study, we developed an automatic workflow to extract curbs and localize curb ramps in large point cloud datasets. The proposed method consists of three steps: Vo-SmoG ground filtering followed by refinement for preserving curbs, curb detection based on the sudden elevation change and linearity of the curbs, and curb ramp localization leveraging the context provided by the curb lines. It was evaluated both qualitatively and quantitatively with a representative mobile lidar dataset, resulting in recall, precision, and F-1 scores of 72.4%. Beyond extracting curbs and curb ramps, the proposed approach can be potentially used for further analyses such as feature characterization and point cloud classification for other assets and objects of interest.

The fire hydrant is one of the critical infrastructure assets which provide water supply to firefighters during the outbreak of fires. Preventive maintenance of fire hydrants is necessary to inspect them for any damage and assess the need for further maintenance actions to keep the fire hydrants in good condition. However, due to the large number and geographical spread of fire hydrants, the conventional inspection process requires extensive labor and time. Therefore, an automated framework to detect and reconstruct fire hydrants’ three-dimensional (3D) models is proposed in this paper. The automated framework could save time and labor for fire hydrant inspections. Also, the reconstructed 3D fire hydrants models could assist in making maintenance decisions by providing a clear view of fire hydrants. The proposed framework consists of four modules: (1) fire hydrant data collection; (2) fire hydrant detection using computer vision; (3) fire hydrant image cropping; and (4) 3D reconstruction of the fire hydrants. To improve the 3D reconstruction quality, different cropping sizes and the number of images were tested for the fire hydrant 3D reconstruction. The number of points related to fire hydrants is used as the indicator to evaluate 3D reconstruction quality. The findings of this study show that x2 cropping sizes and using 32 images can improve the 3D reconstruction quality.

A low-cost 2D LiDAR sensor and a near real-time data processing algorithm are used in this study to capture the as-built geometry of the freshly printed layers for inline and automated inspection. The accuracy of the proposed system is evaluated under different conditions and layer dimensions. Based on the obtained results, significant improvements were observed after a statistical approach was used for outlier detection and elimination. Maximum error values of less than 1.1 mm were achieved for inline layer width and height measurements using the proposed approach.

The disability-related inequity has been identified in the built environment, causing discomfort in their essential routines. Current legislation focuses on building design and construction but not on the actual experiences of those with disabilities, like wheelchair users. These occupants encounter significant issues such as seat discomfort and unhealthy air quality due to their lower breathing zone. Although there are some research efforts on wheelchair user monitoring, there is limited information on the personal environmental conditions of wheelchair users, especially regarding their navigation in buildings. In this paper, we used WheelCom, an open source based toolkit to identify breathing and seat discomfort of a wheelchair user to assess actual perception and experience. We recruited a wheelchair user and installed WheelCom on the participant’s wheelchair to follow the typical trajectory. Our outcomes would motivate architects, engineers, planners, and policymakers to rethink the accessibility and inclusion of buildings from the perspective of wheelchair users.

Although computer vision technology has shown great potential, its reliability can significantly degrade in the target domain where the model is applied. Collecting and labeling training data from the target domain can address this issue; however, it is a tedious and time-consuming task. To address this issue, this paper presents a novel method generating training data for construction site monitoring. The proposed method consists of extracting moving objects, classifying each region by comparing their features to the target classes, and then assigning class labels. The newly labeled data is copied and pasted to a clear background of the target domain where its foregrounds are removed by image inpainting. Experiments were conducted on construction site videos captured in far-field monitoring environments. The proposed method can significantly reduce the amount of effort required for data collection and labeling, thereby increasing the efficiency of developing robust computer vision models for construction site monitoring.

Human activity recognition (HAR) using machine learning has shown tremendous promise in detecting construction workers’ activities. HAR has many applications in human-robot interaction research to enable robots’ understanding of human counterparts’ activities. However, many existing HAR approaches lack robustness, generalizability, and adaptability. This paper proposes a transfer learning methodology for activity recognition of construction workers that requires orders of magnitude less data and compute time for comparable or better classification accuracy. The developed algorithm transfers features from a model pre-trained by the original authors and fine-tunes them for the downstream task of activity recognition in construction. The model was pre-trained on Kinetics-400, a large-scale video-based human activity recognition dataset with 400 distinct classes. The model was fine-tuned and tested using videos captured from manual material handling (MMH) activities found on YouTube. Results indicate that the fine-tuned model can recognize distinct MMH tasks in a robust and adaptive manner, which is crucial for the widespread deployment of collaborative robots in construction.

Understanding the capacity of urban stormwater infrastructure systems before the disaster and evaluating the impact of their possible disruption during the disaster is essential to assess the vulnerability and resilience of interconnected infrastructure systems in urban communities. This paper proposes a novel digital twin modeling framework to identify the up-to-date condition of urban drainage infrastructure systems and then map it into the virtual model for risk analysis. Building upon the two different sources of images—(1) participatory sensing obtained from mobile devices; and (2) large-scale publicly available street-level visual dataset—we first detect the drainage infrastructure in cities and then estimate their geographic location and as-is condition information through deep learning architecture. The identified spatio-temporal information is then fetched into the associated virtual replica toward a digital twin modeling, and finally the associated risk is analyzed. Case studies were conducted in Houston, TX, which demonstrates the update of infrastructure condition and mapping, eventually evaluating the vulnerability of urban stormwater infrastructure systems in Houston, TX. The proposed digital twin approach for urban drainage infrastructure systems has the great potential to improve proactive hazard mitigation and data-driven decision-making in prioritizing the infrastructure improvements for a resilient built environment.

Conventional spatial information is mainly presented as exocentric perspectives; that is, information is displayed in front of a user, analogous to third person viewpoint. Exocentric perspective presents the location of one object relative to the location of other objects. The recent advancements in mixed reality have expanded the border of spatial information communication by condensing intuitive visualization, localization, and interaction methods into portable devices. The implementation of optical see through (OST) head mounted displays (HMD) and reality capture technologies enable spatial models (e.g., point cloud) to be projected directly at a 1:1 scale on top of the real building, creating an illusion of “seeing through walls.” This will give the user an egocentric reference of spatial information, which presents location information relative to the user’s own body, and from which the user can plan paths to the desired destination without ingestion and conversion. However, the uncertainties remain in how these applications can improve the efficiency of spatial awareness especially when one has access to both new and conventional spatial perspectives. This paper develops an augmented reality-based navigation system to improve spatial awareness via real-time reality capture and geometric alignment technologies for providing both exocentric and egocentric perspectives. Then a human-subject experiment was performed in a real two-story building at the University of Florida. Corresponding spatial information was presented with pre-scanned point cloud and mesh models using a virtual scene built with Unity game engine and then visualized with a HoloLens 2 HMD in both perspectives. The findings confirmed that the proposed egocentric visualization of building helped achieve better wayfinding performance, and advantages in cognitive load and situational awareness.

Timely progress update in the project schedule is essential for managing the critical path and expected project completion date. With the advent of AI-powered construction progress monitoring, inferring up-to-date progress information from visual data (images/videos) has become very much possible. Principally, these methods compare as-built reality models (point clouds) with 4D BIM to estimate element-wise or schedule-activity-wise project progress. Estimated progress is then updated in the project schedules. However, the 4D BIM creation process is time-consuming, and this schedule-linked model often becomes outdated after construction begins. Currently, there is no method to automatically align project schedules and reality models for progress updates without an updated 4D BIM. Therefore, this research proposes an automatic alignment method between project schedules and reality models with or without a 3D BIM. In particular, the reality models are first aligned to the world coordinate system using a 3D BIM or ground control points. Next, two-step point cloud segmentation is employed to detect the progress associated with a specific location, object, and task (L-O-T). Similarly, location, object, and task (L-O-T) information is inferred from each schedule-activity through natural language processing (NLP)-based information extraction. Later L-O-Ts are matched through a distance-based matching technique to link progress information with schedule activities. The method is applied in a precast building construction project, and the preliminary results confirm its applicability for automated progress updates.

Smart city projects aim to enhance public safety and disaster management through the use of technology such as CCTV (closed-circuit television) cameras. However, sharing CCTV surveillance areas across local and central governments can be challenging due to the different models and physical environment surrounding each camera. In this study, we propose a method to extract camera parameters and surveillance areas for creating a visible coverage map of CCTV without the initial value of IOP (interior orientation parameters). The EOP (exterior orientation parameters) and focal length of CCTV cameras were extracted using point cloud data obtained through an MMS (mobile mapping system) using perspective projection algorithms. The surveillance areas were then vectorized utilizing a combination of DBSCAN (density-based spatial clustering of applications with noise) and convex hull algorithms, with the SMRF (simple morphological filter) algorithm used to account for occlusion areas. Manual verification was utilized to ascertain the ground truth of the surveillance coverage. Comparing the ground truth with the estimated surveillance area vector data, an IoU (Intersection of Union) of over 85% was recorded, demonstrating the validity of the proposed method for creating accurate CCTV visibility areas. Our study presents a method for developing CCTV surveillance zones in smart city initiatives, which can facilitate the exchange of vector data between local and central governments. Our results offer a holistic perspective on a situation, which can aid in improving public safety and disaster management.

Imagery is a standard modality of visual data capture on construction sites for documenting construction progress. Assimilating the data from multiple disjointed 2D images into a single 3D format enhances visualization and scene understanding and increases the data usability for tasks like quantity estimation and progress tracking. Two popular methods for 3D reconstruction are structure-from-motion (SfM)-based photogrammetry and neural radiance fields (NeRF), a neural network-based technique in computer vision. In this paper, we compare the spatial geometric accuracy of 3D reconstruction from 360° videos of construction sites using the SfM library called Colmap and NeRF. Our experiments show that 3D reconstruction from conventional photogrammetry is sharper than NeRF and more accurate in capturing object details and boundaries.

Robotic labors are increasingly incorporated in construction to team up with human workers and relieve them from physically exerting and hazardous work. Compared with traditional communication tools (e.g., control interfaces), speech is a natural and powerful one to efficiently communicate workers’ intents to robots without interrupting the work at hand. However, few studies systematically investigate the existing speech understanding technologies for construction applications. By following a systematic review methodology, this paper (1) gives an overview of the heated debate topics in recent 10 years, (2) presents a taxonomy of speech understanding studies, and (3) discusses the research gaps and future work for deploying speech-enabled collaborative robots in the noisy and unstructured construction contexts. The findings are expected to reveal the development trends of the speech understanding technologies, direct future research efforts, and advance human-robot collaborative construction applications.

Despite the potential of robotic systems for automating the construction industry, the role of human operators remains essential for the success of these systems in complex and dynamic environments. However, current human-robot interfaces are often limited to low-level interactions that require constant micromanaging of robot movements. To address this limitation, this study proposes a deictic gesture-based interface that enables high-level task specification for construction robots. To evaluate the user experience and task performance of the proposed interface, we conducted a laboratory experiment with six human subjects who interacted with the robot to make openings in drywall panels. The results show that the proposed interface significantly reduced mental demand and effort levels among the participants compared to the conventional joystick interface. Moreover, task performance using the proposed interface was comparable in accuracy and efficiency to that achieved with the joystick interface. These findings highlight the potential of the proposed deictic gesture-based interface to facilitate intuitive human-robot interaction and precise operation of construction robots, particularly in situations where as-planned building models are not readily available.

As the construction industry faces the challenges of a worker shortage and low productivity rate, there is a growing interest in using human-robot collaboration (HRC) in construction. HRC allows for a combination of the accuracy and repeatability of robots with the flexibility and intelligence of human workers. To take advantage of its potential benefits, it is important for construction workers who are not robotic experts to interact easily with robots through intuitive and natural user interfaces. Even though many studies have been performed on HRC using natural language, little research has been conducted on this topic in construction. This paper conducts natural language understanding of language instructions for pick-and-place operations in construction using the language model Joint BERT. Experimental results show high accuracy on intent classification and slot filling tasks, allowing the robot to perform tasks accurately for a given natural language instruction.

The construction industry is facing problems with low productivity rates, safety concerns, and labor shortages, leading to the adoption of robotic technologies. While robots can replace tasks requiring repeatability, power, speed, and precision, tasks requiring intelligence, flexibility, and problem-solving still need skilled human workers. For effective human-robot collaboration on highly dynamic and uncertain construction job sites, robots must be designed and controlled to earn workers’ trust. To identify trust mechanisms facilitating collaboration, this study aims to analyze human physiological/psychological responses to construction robots. Using wearable sensors, this study measured responses during virtual reality experiments in which humans worked collaboratively with robots. The results showed that robot movement speed, proximity distance, and worker engagement significantly affected human responses, but the robot’s direction of approach did not. These findings can improve collaborative task design and guidelines for improving safety and productivity in human-robot collaboration from a human-centered perspective.

Robotic teleoperation is regarded as a promising method of assisting workers in performing construction activities while avoiding potentially dangerous conditions. However, the control interface of conventional teleoperation systems relies on physical devices, such as a keyboard and joystick, to input control commands, which cannot be intuitively mapped to robot movement, and thus affecting the teleoperation performance of non-expert users. In this research, we present a novel human-robot interface to map human arm movement to robotic arm motion for intuitive teleoperation. Firstly, the human arm is tracked in real-time through a motion capturing system, that is, OptiTrack. The 3D location data is then streamed to robot operating system (ROS) and is used for motion planning of the robotic arm in MoveIt. Finally, the interface is demonstrated in a teleoperated pick-and-place task through robotic simulation, and the usability of the developed framework is compared to that of a joystick-based teleoperation system.

Advances in robotics enable the implementation of collaborative robots in hazardous, repetitive, and demanding construction tasks to improve safety and productivity. Accurate and reliable human motion prediction is required to achieve smooth human-robot collaboration (HRC). However, many deep-learning-based models only consider observed movement to predict human motion while neglecting the interactions between humans and their surroundings. This study proposes a context-aware deep learning model, integrating observed movement and context information (i.e., locations of assigned tasks) into a long short-term memory network with an encoder-decoder architecture to predict a sequence of human motion in 3D. A pilot experiment was conducted, and the proposed model achieves an average displacement error of 0.15 m. The results show that incorporating task contextual information improves the accuracy of human motion prediction by 6.25%, which could augment the perception and reasoning capability of collaborative robots for improved HRC in construction.

Remotely operated vehicles (ROVs) are widely used in subsea navigation and operation tasks. Currently, ROV operations are primarily conducted through traditional control kiosks and feedback methods, with the camera display and joysticks control on a surface vessel. This kind of control method resulted in an extremely high training barrier. The marine and offshore sectors are dominated by male workers due to the harsh working conditions. Besides, subsea engineering also poses strict requirements on age, because of the mental and physical load of certain tasks. Therefore, this paper proposes a human body motion and hand gesture control method in virtual reality (VR) for ROV teleoperation, aiming to lower the training barrier, improve the work condition, and reduce mental load during the tasks. Specifically, human body motion parameters, mainly motion and orientation, are recorded in VR and mapped with ROV locomotion control via closed-loop control. A series of scripts are developed to recognize human gestures as functional control, such as light on/off or ROV up/down control. Finally, a configuration UI panel is designed for personnel adjustment. A case study was also presented to verify the effectiveness of the system. With this control system, the operator could teleoperate ROV in a more comfortable and safer environment in a more intuitive manner.

Robot teleoperation has gained popularity in a variety of construction applications. A significant challenge of robot teleoperation is the disconnection between the robotic perception and the human operator’s sensory processes, creating a sensorimotor mismatch in motor-intensive activities. As a result, recent robot teleoperation literature has started investigating haptic interactions in the feedback and control loop. Haptics, including the feeling of force, motion, and vibrations, have been proven helpful for accurately perceiving task status and physical systems with heavy motions and enabling precision operations. However, the current haptic simulation methods are oversimplified and hard to provide a comprehensive, realistic, and complete simulation of the physical interaction that occurs in typical construction operations. This study aims to test a comprehensive, high-fidelity physical rendering method that simulates complete real-world physical processes via the physics engine. The proposed method captures all categories of physical interaction in typical motor-intensive construction tasks, including weight force, texture force, inertia force, and impact force. The result shows that the proposed method builds a digital twin model of real physical processes to facilitate intuitive robot teleoperation.

In teleoperation, it is crucial to monitor human operators’ cognitive workload. This is because the movement of the end-effector of the robot in a distance is ultimately determined by the decisions, controls, and commands of the human operators. There are various factors that affect cognitive load, and among them, time pressure or schedule pressure is the common stress that human workers usually experience in construction to meet the project completion deadline. They have to make up for it due to factors that delay the schedule during the construction project (e.g., weather, clients’ change orders). It is hard to expect that there will not be time pressure even when teleoperating construction robots since it is known that teleoperation takes longer to accomplish the same amount of work as onboarding. Such time pressure typically affects human workers’ cognitive load and work performance in various contexts. There is still a lack of knowledge about operators’ perspective when it comes to teleoperating a robot for a challenging task at a construction site under time constraints. Therefore, it is necessary to examine the cognitive load of the operator and overall human-robot interaction performance with time pressure. This study takes account of different levels of time pressure at a construction site rather than a binary situation, that is, with or without time pressure. The experimental results showed that cognitive load increased and performance decreased when excessive time pressure was given, whereas cognitive load decreased and performance increased when reasonable time pressure was applied.

Construction technology researchers and forward-thinking companies are experimenting with collaborative robots (aka cobots), powered by artificial intelligence (AI), to explore various automation scenarios as part of the digital transformation of the industry. Intelligent cobots are expected to be the dominant type of robots in the future of work in construction. However, the black-box nature of AI-powered cobots and unknown technical and psychological aspects of introducing them to job sites are precursors to trust challenges. By analyzing the results of semi-structured interviews with construction practitioners using grounded theory, this paper investigates the characteristics of trustworthy AI-powered cobots in construction. The study found that while the key trust factors identified in a systematic literature review—conducted previously by the authors—resonated with the field experts and end users, other factors such as financial considerations and the uncertainty associated with change were also significant barriers against trusting AI-powered cobots in construction.

Artificial intelligence (AI) is becoming more prevalent in the construction bidding process, assisting human decision-makers. However, little is known about the tactical shifts that may arise from AI participation in the market, particularly regarding bid pricing. This study aims to predict the strategic decisions AI bidders may make and their impact on bid pricing when they become dominant players in the construction bidding market. An experiment was conducted in which AI bidders competed repeatedly in an environment that simulates the decision-making process in the construction bidding phase. AI bidders were built with Q-learning algorithms, which is a popular reinforcement learning algorithm in repetitive games. Bid notice data from public construction projects in the Washington Department of Transportation (WSDOT) was given to the AI bidders, who set bid prices based on prior bidding experiences. As a result of repeated competition and learning, it was found that the AI bidders gradually learn to cooperate rather than to compete with each other, sustaining higher bid prices compared to human bidders. The study suggests the possibility of collusive behavior by AI bidders in a scenario where they are the dominant participants in the construction bidding process. These findings highlight the need to monitor and regulate the AI participants to prevent anti-competitive behavior in the construction bidding market.

The growing socio-technical influence of artificial intelligence (AI) is increasing in various dimensions of our daily lives. The monitoring and decision-making role of smart home AI agents could go beyond simple operations, and a lack of users’ trust could lead to the underuse or misuse of their capabilities. As such, lying at the intersection of trust in human-AI interactions and smart home ecosystems, this paper attempts to explore the driving factors of building trustworthy AI-powered smart home ecosystems. To this end, through a systematic literature review, we have identified the characteristics of human-AI trust in intelligent environments, factors that can influence users’ trust in AI, features that can be implemented to improve users’ trust, and challenges and opportunities that need to be considered. The findings can help develop a pathway for building long-term users’ trust in AI-powered smart homes for integrated human-AI interactions.

The current delivery practices of airport projects are traditionally outdated, lacking innovative and modern techniques which provide overall better cost, schedule, among other performance metrics. Design-build (DB) is an alternative and viable option for the effective delivery of airport projects. Previous studies have lacked in focusing on studying the determinants that affect the success of DB airport projects. This study aims to fill this gap. The goal of this research is to explore these determinants by computationally analyzing the perception of relevant experts. This goal is achieved by: (1) developing a survey to gather valid data on the likelihood of occurrence, impact on cost, and impact on schedule of 34 risk factors perceived in airport DB delivery as compared to design-bid-build (DBB); (2) clustering the data using the K-means technique; and (3) analyzing the clusters and providing insights from the emerging trends. Results indicate three clusters classified with low, moderate, and high-risk factors in terms of DB delivery as compared to DBB. The inability to fast-track delivery poses the least risk. Ineffective risk management was considered to be an example of a moderate risk. And inappropriate pricing agreement method was determined to be the riskiest. Ultimately, this study adds to the body of knowledge identifying key risks that affect the delivery of DB airport projects.

This research explores the applicability of blockchain technology for collaborative risk assessment of capital projects through prototype development. Risk assessment is a collaborative process where project participants get together to assess and manage potential project risks. Different systems have been developed to support this collaborative risk assessment process. However, such systems are governed in a centralized manner and do not protect the confidentiality of individual inputs from the stakeholders. Moreover, the security and transparency of the data stored can become an issue as well. The research explains the development of a blockchain prototype to address the problems stated above. A private permissioned blockchain network has been configured so that risk-related information can be transacted in a secure, transparent, and trustable way. Smart contract algorithms have been developed to receive risk inputs from multiple stakeholders and update an aggregated output. The system architecture and the functionalities of the prototype are described in detail, and each step of the risk assessment process has been demonstrated on the prototype system. The research discusses the limitations of the current prototype and future work that can be done to improve collaborative risk assessment.

Adaptive signalized intersections can significantly enhance the traffic characteristics of transportation networks. This research aims to utilize a cooperative game theory approach, Shapley value, to optimize the green times of signalized intersections in a network. This is achieved through simulating the current traffic characteristic of a network, then applying Shapley value to calculate the optimized green times of phases in the intersection based on their marginal contribution to the overall network performance. The proposed model is developed and simulated on PTV VISSIM. The optimization module is coded via Python through the integrated COM interface. The proposed approach is developed and tested on a case study in Cairo, Egypt, presenting its ability to optimize the overall traffic characteristics of a network. The model’s outcome is compared to optimized fixed time signal. The proposed approach improved the overall network performance, rather than achieving local optimal solutions at subsets of the network.

The use of drones for construction applications is on the rise, but it also brings about new safety challenges for human workers in an already hazardous environment. To address these safety challenges, one potential solution is to allow workers to actively communicate with drones rather than passively observing them. To explore this idea, a series of gesture-based safety commands were developed. This would enable workers to communicate their safety needs to drones while working in close proximity. A pilot study was conducted with a small group of users to evaluate the feasibility of using gesture communication modality for safe human-drone interaction in a virtual construction scenario. User feedback was also gathered, providing valuable insights for future studies and improvements. The results of the study show that gesture-based communication has great potential for ensuring safety during human-drone interaction in construction environments.

When operating a construction robot (e.g., excavator), the operator’s unsafe behavior directly affects the safety risk (e.g., underground utility damage occurrence during excavation process). Operators’ behavior is greatly influenced by the surrounding environment and further the communication with other coworkers (i.e., spotter), and thus there is a need for studying human factors during work by investigating how the operator-spotter interaction affects the operator when performing the task in a challenging work environment. In this paper, we investigate how the different levels of environmental complexities and operator-spotter communication channels affect operators’ performance and the accident occurrence during work. A human-centered experiment is designed and conducted in environmentally realistic scenarios based on immersive virtual reality. The task of operating a virtual excavator as well as the interaction between the operator and a spotter are performed in realistic jobsites in which a series of environmental stimuli are modeled and simulated. Operators’ cognitive responses and work performance are assessed by subjective evaluations and instrument-based measurements (i.e., eye-tracking). This study can establish a better understanding about the effectiveness of between-worker communication and worker-to-robot interaction during robot operation as well as the influence of environmental visual and auditory stimuli on the teleoperator.

Construction robots have great potential to serve as assistants to relieve construction workers from repetitive and physically demanding tasks. It is essential for robots to understand and predict human intention in order to adapt their motion to ensure smooth human-robot collaboration. This study proposes a long short-term memory model-based multi-task learning framework to simultaneously predict multi-level human intention in assembly tasks, including high-level actions and objects, and low-level body movements, from observed body movements and associated assembly components extracted from videos. The proposed models were trained and tested using 54 videos collected with nine participants performing six assembly tasks, achieving an accuracy of 82% and 98% in action and object prediction, respectively, and an average displacement error of 8.71 pixels in pose prediction. The incorporation of work context significantly improves the accuracy of object prediction by 11.36%, with the performance of other two tasks increasing slightly.

With the advancement of robotic technologies, an increasing number of robots are envisioned to be deployed in various human-occupied environments such as construction job sites. Although navigation in a static environment is well studied, navigation in such a dynamic environment is still a challenging problem. In particular, construction environments present many challenges as they are often densely populated with both static and dynamic objects (e.g., stocked materials, workers, and equipment). Thus, robots in the construction environment should be capable of adapting to frequent layout changes and dynamic human movements and producing efficient paths in unknown, dynamic environments accordingly. Given the safety-critical nature of the construction environment, it is also critical not to collide, distract, or interfere with workers when robots maneuver around human workers in the workplace. To accommodate this need, this study proposes a Reinforcement Learning (RL)-based navigation model that enables robots to adhere to appropriate safety proxemic considerations aligned with the social work convention of construction workplaces while following a globally planned trajectory. The result showed that our model could secure the respective minimum distance in contrast to the baseline model, which suffered in complex environments. These findings will contribute to building future construction mobile robots with social intelligence that can produce socially compliant and safe behaviors.

Bridge inspections are characterized by their labor-intensive nature and inherent risks, relying predominantly on engineers’ visual analysis. Although the integration of drones has alleviated the safety concerns associated with human labor, the accurate identification of defects in vital elements continues to necessitate inspectors’ specialized knowledge. Aggregating multi-inspector experiences can improve the localization of critical defects. The challenge lies in capturing and explaining drone trajectories into reusable and explainable strategies. This paper presents a framework to capture inspectors’ strategies by analyzing drone control in bridge inspection simulations. It gathers and scrutinizes inspectors’ drone control histories to understand their intentions. Due to the vast search space of inspection strategies in dynamic, uncertain contexts, imitation and reinforcement learning are utilized to learn reusability and explainability. Experiments demonstrate that drone trajectories aligned with bridge elements can explain inspection knowledge. Inspectors with explainable patterns, such as the human attention between the different spans inside the span, achieve better defect detection performance (correlation coefficient of 0.5). This framework promotes inspector-drone collaboration that adaptively supports human inspectors, resulting in more reliable inspections.

Design for disassembly (DfD) is a strategy that facilitates material recovery by proactively identifying key elements of a building that can be deconstructed in the future. Many factors influence the success of DfD, including connection type, use of lightweight components, and government policies. Many studies have focused on these factors, and several tools have been proposed to support DfD implementation. To better understand the nature of these factors and tools, this study presents a literature review composed of two parts. The first investigates critical success factors (CSFs) influencing DfD in the construction industry using a people, process, and technology (PPT) methodology. The second part focuses on DfD computer-aided tools, and the state-of-the-art in the development of these tools is discussed, along with their main contributions. In conclusion, 27 DfD critical success factors are identified along with several tools that support its implementation.

The construction industry has increasingly adopted digital twin (DT) technology to support planning and design, construction, and operation. Modular DTs offer a flexible and scalable solution that can adapt to the changing requirements of the construction industry. However, it is unclear how modules of DTs are defined and structured for the built environment. This paper defines modularity in DTs as technical, process-oriented, spatial, and life-cycle-oriented modularity. A literature review is conducted to analyze current DT approaches in the built environment, considering the modularity of their implementations. Information containers are identified as a viable way to implement modular DT. A comparison of their implementation based on the Asset Administration Shell (AAS) of Industry 4.0 and information container for linked document delivery (ICDD) is provided. The paper contributes comprehensive and modular approaches for implementing modular DTs, offering valuable insights for future research and practical implementation.

Workplace safety, particularly electrical hazards, is crucial in the construction industry, with electrical issues among the top causes of construction fatalities. Energy storage systems (ESS) and microgrids pose significant risks to workers lacking proper knowledge of large-scale batteries. Virtual reality (VR) has proven to be effective in improving specific knowledge and skills in various domains and has demonstrated potential for effective training, especially when access to ESS facilities is limited. To achieve educational effectiveness comparable to hands-on training, VR modules must consider specific design features. This study assessed an immersive VR module for training electrical workers in battery assembly and connection. Participants completed identical tasks twice: once in VR and once with physical components. Researchers recorded and analyzed participants’ behavior, comparing the efficiency of VR and physical training through video coding analysis to identify crucial VR design features impacting educational effectiveness. While physical training currently demonstrates greater accuracy and efficiency, there is potential for VR training to improve through focused design enhancements. Addressing challenges and implementing design improvements will enhance VR training’s value across diverse fields.

Suspended ceiling systems are complex and heterogeneous due to the combination of different components. Therefore, it is hard for robots to navigate in these environments without expert designed robot morphology. However, the cost of robot design is high since ceiling systems vary from building to building. Currently, some studies demonstrate that robots can evolve like creatures so that they can adapt themselves to different environments through evolutionary strategies. Inspired by the assembly of LEGO bricks, we applied graph grammar methods to optimize robot design in suspended ceiling systems. The basic idea is that the robot can find the optimized structures assembled from elementary components for themselves without any human intervention. Robots were trained in four common ceiling environments reflecting the influence of typical terrain and obstructions (e.g., ducts). Results show that different suspended ceiling system significantly affects the configuration of robots and robots can successfully evolve specific shapes to improve their acclimatization. This paper marks the first attempt at performing robot evolution in the context of facilities management and is expected to evoke future discussions in robot design in more civil engineering tasks.

Designers usually assess only a few house design alternatives simulating their energy use, due to time and cost constraints. This study aims at developing a machine learning (ML)-based generative design (GD) framework to automate the design process of Florida’s detached residences while optimizing their energy performance. An artificial neural network (ANN) model was developed using the machine learning platform TensorFlow along with some Python-based Keras libraries based on a big dataset of around 17,000 newly constructed detached residences in Florida between the years 2009 and 2021. The GD framework was established using Autodesk Dynamo and Autodesk Revit internal generative design tool that uses the Non-Dominated Sorting Genetic Algorithm (NSGA-II). The ANN model was created to predict the required capacities of the cooling and heating systems in detached houses considering 10 independent variables relating to the house geometry and its energy performance. The ANN was then integrated within the GD framework for performance evaluation purposes. The findings of this study included a fully automated design of energy-efficient detached houses greatly reducing the financial strain and time consumed by designers/developers using traditional techniques.

Leveraging deep learning models to automatically detect structural steel elements at the pixel level from visual data can support many potential use cases for construction projects, including automated construction progress monitoring, quality control, safety monitoring, and perception systems for construction robotics. Many researchers have been applying deep learning models to detect various construction entities on sites. However, limited research has focused on detecting structural steel elements at the pixel level. This research focused on implementing U-Net, a state-of-the-art deep learning model for semantic segmentation, to segment steel beams, girders, and columns from construction images. All images were captured by a drone from a three-story steel building construction site. A data augmentation process was performed to increase the quantity of the training data. Two pre-trained convolutional neural networks (CNNs), VGG16 and ResNet50, were applied to the U-Net as backbones, respectively, for transfer learning purposes. The U-Net models with different backbones were trained using different sizes of the training data, and the performance was analyzed and compared. Four metrics were used to quantitatively analyze the model performance: precision, recall, F1-score, and Intersection over Union (IoU). This research found that the transfer learning technique improved the model performance by 2%−6%, depending on the analyzed metric. The data augmentation also slightly boosted the model performance. However, both transfer learning and data augmentation increased the overfitting issue. The future work includes combining the segmented steel elements with the as-designed building information modeling (BIM) for automated construction progress monitoring.

This study explores the potential of vision-based control of robots for construction applications using a transfer learning approach. Repetitive and arduous construction tasks cause musculoskeletal disorders in human workers which can be performed safely by robots. Unlike manufacturing processes, construction tasks are complex. Collaborative robots utilize human intelligence to adapt to the complexities of the construction site. Interaction between humans and robots is a challenge because of the limited infrastructure and computational training of construction workers. Vision-based hand gesture control is implemented using transfer learning by using a pre-trained hand detection model called Mediapipe Hands and using its output as features to train another artificial neural net (ANN) to classify different hand gestures. After optimizing the model parameters, 99.43% training and 99.11% validation accuracies were achieved. The results of validation performed through experiments in a simulated environment show the potential of using hand gestures for the control of construction robots.

As university buildings become more complex, navigating them and finding rooms and exits, especially for newcomers, has become increasingly difficult. This study aimed to explore the literature on indoor navigation technologies, examine the use of as-built 3D scans in indoor navigation, and develop a prototype application to assist users in locating facilities such as restrooms or exits during emergencies in a university building. The study analyzed data, evaluated the perceptions of students who visited the building, and developed a prototype application based on a 3D scan of a university building. The study used a Faro laser scanner to scan the indoor hallway of the building and collected 3D point cloud data, which was converted into a mesh model and an area target using the Vuforia engine. The study then exported the mesh model and area target to a Unity application to develop the prototype indoor navigation application using the Vuforia engine and Unity.

Human-robot collaboration (HRC) is an emerging solution for the construction industry’s productivity and safety challenges. The seamless HRC requires robots to understand the structure of construction tasks. Nevertheless, the implicit, dynamic construction task flow poses a non-trivial challenge. To address this challenge, a vision-based multi-granularity task’s primitive learning method is proposed. This study seeks to enhance the mutual understanding between workers and robots by determining which granularity level is best for the tasks’ understanding. Results show that the intermediate level has the best compromise between classification performance and embedded task knowledge. The outcomes will improve the smoothness of a HRC team.

The working environment of construction workers is often hazardous and dynamic, leading not only to decline in worker productivity but also fatal accidents. Monitoring of workers’ behavior has thus gained increasing attention in the construction community for hazard monitoring, ergonomic analysis, and productivity estimation. Egocentric action recognition is robust and promising in identifying, localizing, and tracking workers’ actions. However, the insufficiency of publicly available datasets designated for egocentric construction workers’ action recognition hampers the training and evaluation of existing and newly developed deep-learning models. In this regard, this paper introduces ICON-Pose, the first open dataset built specifically for estimating construction workers’ poses via egocentric view. ICON-Pose offers hundreds of egocentric images and corresponding 2D workers’ body joints in 38 actions, categorized in 10 basic construction tasks, including “connect,” “cover,” “cut,” “dig,” “finish,” “place,” “position,” “spray,” “spread,” and “others.” ICON-Pose with the proposed pose estimation model demonstrates the ability in accurately depicting diverse construction workers’ poses as well as the robustness in describing unique construction poses. The proposed dataset is expected to invigorate and support artificial intelligence research in construction workers’ behavior tracking and has the potential of serving as a benchmark dataset for subsequent analysis.

Regular indoor building inspections and maintenance are essential to prevent property damage and sustain its condition and functionality for the intended occupants. Traditional inspection is heavily labor-intensive, time-consuming, experience-based, and error-prone. With the growing development of multi-sensor and artificial intelligence (AI)-empowered unmanned ground vehicles (UGV), inspection activities can be automated and digitalized. However, there is a lack of knowledge on how to incorporate them effectively for streamlined inspection of indoor building systems. This paper proposes a workflow of integrating robots, multimodal imagery sensors (e.g., infrared camera, LiDAR, RGB camera), and AI-based data analytics to support real-time detection and assessment of defects within indoor building environments. In addition, the detected imagery anomalies will be geo-registered and located into a 3D building model, to support further analysis and evaluation of building conditions. The successful implementation of the envisioned intelligent robotic sensing system can lead to a significantly more efficient and responsive diagnosis of indoor building systems.

The construction industry has been experiencing a severe welder shortage in the US. Therefore, the development of welding robotics that can move around the job site and perform welding is in critical need. Moreover, welding robotics removes health-related issues associated with harmful fumes, elevated temperatures, and strenuous physical labor. It will also reduce the risk of injuries. An autonomous robot operating on a construction site must be able to detect obstacles, navigate to welding material, and then perform welding on the desired material. This paper focuses on developing an integrated unmanned ground vehicle (UGV) system with an automated robotic welding system to provide mobility for the construction site. The autonomous system maps and navigates with collision avoidance using a 3D light detection and ranging (LiDAR) sensor. When the robot has reached the desired position where the target welding material is located, the robotic welding system automatically performs the welding process using visual sensors. A hardware and software integration between the UGV and the robotic welding system is also proposed. The results demonstrate the accuracy and effectiveness of the proposed method.

Real-time monitoring of indoor air quality (IAQ) is significant for ensuring occupants’ health and comfort. While smart sensing technologies were used for IAQ monitoring, there is still a gap of quantitatively assessing IAQ and visualizing its interactions with the surrounding physical world. Therefore, this study proposes a novel real-time IAQ monitoring and visualization system. This system specifically integrates an unmanned ground vehicle (UGV), an indoor localization system, and the Internet of Things (IoT). First, an IoT sensing unit was developed to dynamically measure the concentration of different indoor air pollutants. Second, a UGV was configured with the capability of full autonomy for providing mobility to the sensing unit. Third, an indoor localization system was developed using ultra-wideband technology to track the sensing unit. Fourth, a cloud-based web server was created to establish communication for data transmission. Fifth, a 2D visualization interface was generated to visualize the indoor air condition and its interactions with the physical world. The proposed system was validated in a real-world application at the authors’ institution to test its applicability and performance. The proposed system achieved promising performance regarding (1) full navigation autonomy to sense the indoor environment, (2) reliably monitoring and quantifying the indoor air condition, (3) accurately localizing the sensing unit, and (4) intuitively visualizing the IAQ with consideration of both temporal and spatial characteristics of indoor air condition. This study contributes to the body of knowledge by enhancing existing practices of IAQ monitoring using a cost-effective and mobile robotic system and indoor localization technology.

Since 1851, an agenda for automation in construction has shifted to respond to global conflicts and disasters. Over the past 70 years, tools, machines, and robotic manipulators have been developed to enhance human capabilities and perform difficult and repetitive tasks or dangerous tasks. From the 2000s to present day, our agenda for automation in construction has shifted again in response to global housing crisis, labor shortages, and digital technology disruptions. These advancements toward automation in construction make it possible to produce affordable housing of better quality more efficiently and safer traditional means of construction requiring extensive human labor. This paper builds upon an existing survey of automation in construction from 1851 to present day by defining a contemporary agenda for the continued development of three distinct approaches to automation in construction categorized as: (1) off-site automated prefabrication, (2) on-site single purpose automation, and (3) on-site automated factory.

Robotic construction and 3D printing are revolutionizing the construction industry by offering numerous advantages over traditional building methods. Automated construction with robotics is faster, more efficient, and more precise, resulting in cost savings and shorter project timelines. Moreover, the use of robots reduces the reliance on manual labor, improving worker safety and reducing the risk of injuries. Additionally, the robotic approach ensures a more consistent and uniform end product with fewer variations or defects compared to traditional methods. This paper presents an eight-step guideline for automatically constructing reinforced concrete beams using building information modeling (BIM) and 3D printing. This paper serves as a stepping-stone toward increased automation in construction, providing valuable insights for optimizing construction processes. By integrating BIM and 3D printing, construction projects can benefit from improved efficiency, reduced costs, and enhanced quality, propelling the industry toward a more automated future.

Advances in construction robotics represent a potential shift in building design and construction. In general, construction robotics are usually deployed directly onto construction sites without systematically evaluating the design constructability for robotic applications. Literature on constructability suggests that ignoring it during design will cause rework, inefficiency, and higher cost. Although previous studies have widely discussed design constructability, they mainly focus on traditional human craft-based construction methods. Whereas a gap still exists in design constructability assessment for construction robotics. This paper presents an initial analytical framework for constructability assessment for construction robotics during the design phase. Specifically, we summarize factors that impact robotic constructability based on robotic features, design features, work constraints, and piloted an automated constructability checking system for robotics. Additionally, this study takes CANVAS, a drywall finishing robot, as case study to create a framework in simulation environment, and the results demonstrate the potential value of the proposed framework.

This study gathered data into a construction robot schema (CRS) with an initial data structure that can be used to collect and exchange various construction robots’ information based on the data requirements of construction planners for robotics operations. To develop the CRS, the study conducted a systematic literature review using the Web of Science database to filter and identify relevant papers which were published from 2018 to 2022. Based on 279 eligible papers, the study identified significant information which involved data requirements of the construction domain on robotics using Nvivo software. To structure the information, the study summarized the information into parameters then categorized, defined, matched data types, and exemplified for these parameters. All the parameters were grouped into four categories, including ontological properties, operational requirements, activity, and safety. As a result, CRS supports data structure, including four categories and 35 parameters with corresponding definitions, data types, examples, and references.

Recent advances in small unmanned aerial systems (sUAS), photogrammetry, and flight path planning enabled autonomous surveying and mapping. However, most automatic flight path plannings are designed for horizontal planar surfaces. There is vertical path planning for building surfaces with specified building widths and heights. However, there is vertical path planning that creates paths based on the actual geometry of the surfaces to be surveyed, which is required for vertical and sloped surfaces or terrains. This paper presents a robust three-dimensional (3D) UAS trajectory planning to capture the images of sloped surfaces using different flight configurations with waypoints. The flight configurations include flight height, image overlap, and camera angles. A rock surface site in NC was used as a case study to create and validate a 3D flight path.

In recent years, the construction industry has increasingly adopted the use of four-legged robots. These robots offer several advantages, including time-saving, improved accessibility to dangerous or inaccessible areas, and cost reduction. However, integrating these robots into the workplace introduces new hazards. Unfortunately, no research has been conducted to assess the risks associated with four-legged robots in construction. To have a comprehensive understanding of the four-legged robot integration in construction workplaces, it is crucial to consider their impact on health and safety. To this end, a study was conducted using virtual reality (VR) visualization techniques to simulate conceptual scenarios that required human interaction with four-legged robots at proximal and distant distances. Few construction safety experts observed these scenarios and ranked the potential safety challenges of four-legged robots. The study provides valuable insight into the potential safety challenges posed by four-legged robots and can assist in ensuring their safe integration on construction sites.

Powered exoskeletons have the potential to reduce the physical demands on construction workers and enhance their abilities, yet adoption of this technology has been limited in the US construction sector. To that end, this study aimed to identify the barriers to the adoption of powered exoskeletons in the US construction industry. Firstly, a literature review was conducted to identify commercially available powered exoskeletons suitable for construction. Concurrently, questionnaires were developed and distributed among construction practitioners to understand the challenges associated with the implementation of powered exoskeletons in the US construction industry. The results showed that concerns about usability and productivity gains were the main barriers to the adoption of exoskeletons in the construction sector. The findings of this study provide valuable insights for improving the adoption and implementation of powered exoskeletons in the US construction industry, which could enhance worker safety and productivity.

The adoption of collaborative robots in construction is one major step toward achieving intelligent and automated construction to improve productivity and safety. Many construction tasks require physical human-robot collaboration (HRC), where workers and robots collaborate side-by-side in a common workspace. Such close-distance interactions may cause worker resistance, hindering successful HRC implementation. This study aims to understand workers’ acceptance of HRC via experimental studies. Experiments on human-robot collaborative wood assembly were performed, where participants were tasked to connect wood pieces, and a robot was programmed to place pieces according to design. Two designs with different complexity levels were given. Surveys adapted from technology acceptance model were collected before and after the experiments to investigate individual perceptions and acceptance of robots. The results show that gender and complexity of tasks have a great impact on the acceptance of robots. Finally, the implications for the future development of HRC in construction were discussed.

NASA’s Artemis Program includes a plan to build a sustainable habitat on the lunar surface along with missions for Mars exploration. With the advanced technology in teleoperation, such tasks outside of the Earth have become feasible. To conduct the surveying, excavation, and site preparation toward the habitat construction in the future, we should prepare against many challenging and uncertain issues, such as microgravity, regolith dust, and extreme environments. Under those inherent environmental constraints, a robust teleoperation system plays a key role in mission tasks on the lunar surface. In a remote-control system, a time delay occurs inevitably in data processing or signal transmission. This latency affects operators’ mental workload, situational awareness, decision-making, and in turn, work performance. Therefore, there is a need to better understand uncertainties due to latency in advance to reduce unexpected or uncontrolled situations. This paper examines the impacts of time delay during space teleoperation and discusses future research directions. The capabilities of teleoperation for knowledge-based work under latency conditions are evaluated based on the technical limitations and requirements. The outcome of this paper is expected to provide foundational knowledge to minimize operators’ mental workload and improve work performance in long-distance robot-assisted teleoperation.

Physical interactions between humans and construction robots must inevitably occur under a capacity-based task allocation scheme for successful implementation of a human-robot collaboration (HRC) based workflow. For example, in staging and installing components, a robot can complete physically challenging material handling tasks and a human worker can then creatively manipulate the materials for installation. When materials are handed over from robots to humans, tandem manipulation of objects and physical interaction is necessary. However, this poses several challenges to human workers’ safety, as the objects might fall or break, if the robot incorrectly perceives the human worker’s grip state. This paper considers object handovers between robots and humans and explores how to build a safe tandem HRC framework by letting the robot imitate the mutual physical state adaptation dynamics inherent in human co-workers. To build such a human physical state-aware robot controller, this paper first proposes the use of a haptic glove-based sensing system to capture the grip strength and gesture of construction workers simultaneously. Secondly, the programming by demonstration (PbD) method is used to automatically program a robot through a one-shot demonstration of natural handover processes without requiring the demonstrators (i.e., construction workers) to have computational or programming expertise. The proposed method outperforms prior robot-to-human object handover studies in handling eight construction materials of random shape, dimensions, and weight distribution. In addition, to enable such construction HRC implementations to readily comply with global safety standards, the proposed method is implemented to adhere to ISO 15066:2016 guidelines.

Globally, building construction is complex and advanced in performance. The improvement of the economy and social benefits of building delivery depends critically on applying automation and robotic technology throughout all stages of construction. Therefore, this paper aims to identify the benefits of applying automation and robotics technologies to deliver building projects. A systematic review of the literature was utilized to identify the benefits. A total of 43 papers from conferences and journal articles from Scopus databases were utilized for the study. Findings revealed encouraging design specifications, less dependence on direct labour, increased productivity, and reduced duration of project delivery towards delivering construction projects within schedule and at a required quality standard. This study contributes to the construction automation and robotics field by providing evidence of the benefits of the application of construction automation and robotics for building delivery to stakeholders in the construction industry and policymakers.

As building automation becomes more prevalent, smart buildings will be integrated into smart cities. Using artificial intelligence (AI) is expected to increase efficiency and the automation level throughout the whole life cycle of building information modeling (BIM). Especially for the management phase of a building, facility managers are recognizing the value of machine learning and robotics for the automation of different maintenance tasks. In the fire safety management field, documentation of fire safety equipment (FSE) is required due to recurring maintenance work, system changes, and relocations. This study concentrates on the automatic detection of inspection tags on FSE using a YOLO network and its deployment on a mobile robot. The goal is to create a fully autonomous inspection mission for a legged robot by executing predefined routes, identifying FSE, and extracting necessary information, such as the next maintenance date tags.

Work-related musculoskeletal disorders pose a significant threat to the health, safety, and retention of skilled workers in the construction industry. Construction trades, such as floor layers, perform physically demanding work involving lifting materials and maintaining awkward postures, triggering low back disorders. Active back-support exoskeletons have emerged as potential preventive technology. This paper evaluates the suitability of a commercially available active back-support exoskeleton for a simulated flooring task by assessing measures such as range of motion, perceived level of exertion, and cognitive load. No significant difference was observed in the comparison of these measures with and without the active exoskeleton. However, the use of the exoskeleton triggered slight reductions in the range of motion of the back, perceived exertion, and mental demand. These preliminary findings could motivate more studies into the design and evaluation of similar technologies for improving construction workforce health and safety.

Introducing robots to future construction sites will impose extra uncertainties and necessitate workers’ situational awareness (SA) of them. While previous literature has suggested that system errors, trust changes, and time pressure may affect SA, the linkage between these factors and workers’ SA in the future construction industry is understudied. Therefore, this study aimed to fill the research gap by simulating a future bricklaying worker-robot collaborative task where participants experienced robot errors and time pressure during the interaction. The results indicated that robot errors significantly impacted subjects’ trust in robots. However, under time pressure in time-critical construction tasks, workers were tended to recover their reduced trust in the faulty robots (sometimes over-trust) and reduce their situational awareness. The contributions of this study lie in providing insights into the importance of SA in future jobsites and the need for investigating effective strategies for better preparing future workers.