Journal of Computing in Civil Engineering

A digital twin (DT) can be defined as a multiphysics, multiscale model in which a digital model, such as a building information model (BIM), is updated based on data obtained from a physical system, such as sensor data, results from probabilistic simulations, and material/structural models. This study describes sensor data integration within a BIM as the first critical step toward the implementation of DTs to support structural health monitoring (SHM). In particular, the study defines a methodological approach used to integrate the as-built geometry of existing buildings, as well as their material properties and sensor data into a digital model to assist in accessing sensor data to assess a building’s structural performance. A mass-timber structural system consisting of post-tensioned cross-laminated timber (CLT) self-centering shear walls at the George W. Peavy Forest Science Center (“Peavy Hall”) at Oregon State University was used as a case study to test the proposed approach. The BIM of the shear walls was developed using a Scan-to-BIM approach by converting light detection and ranging point clouds into a BIM. Sensors in the building recorded environmental and structural parameters influencing the long-term performance of the shear walls. Measurands included relative humidity, air and wood temperature, wood moisture content, displacements, and deformations of shear walls. The precise placement of these sensors and the possibility to associate the measured parameters of these entities within a BIM is hypothesized to assist with data management by adding a spatial element to data and analysis results. In addition, the integration into the IFC-BIM platform of a material- and phenomena-specific warning tool allows to promptly identify areas of concern in the monitored building. This can support facility managers in planning inspection and maintenance activities and eventually could lead to the prolonged service life of a building.

Visual inspection is one of the main approaches for annual bridge inspection. Light detection and ranging (LiDAR) scanning is a new technology, which is beneficial because it collects the point clouds and the third dimension of the scanned objects. Deep learning (DL)-based methods have attracted researchers’ attention for concrete surface defect detection. However, no point cloud–based DL method currently is available for semantic segmentation of bridge surface defects without converting the data set into other representations, which results in increasing the size of the data set. Moreover, most of the current point cloud–based concrete surface defect detection methods focus on only one type of defect. On the other hand, a data set plays a key role in DL. Therefore, the lack of publicly available point cloud data sets for bridge surface defects is one of the reasons for the lack of studies in this area. To address these issues, this paper created a publicly available point cloud data set for concrete bridge surface defect detection, and developed a point cloud–based semantic segmentation DL method to detect different types of concrete surface defects. Surface Normal Enhanced PointNet++ (SNEPointNet++) was developed for semantic segmentation of concrete bridge surface defects (i.e., cracks and spalls). SNEPointNet++ focuses on two main characteristics related to surface defects (i.e., normal vector and depth) and considers the issues related to the data set (i.e., imbalanced data set). The data set, which was collected from four concrete bridges and classified into three classes (cracks, spalls, and no defect), is made available for other researchers. The model was trained and evaluated using 60% and 20% of the data set, respectively. Testing on the remaining part of the data set resulted in 93% and 92% recall for cracks and spalls, respectively. Spalls of the segments deeper than 7 cm (severe spalls) can be detected with 99% recall.

Abstract

Health care ensures the health and safety of people, and as such requires effective management of facilities. Unfortunately, current approaches to health care facilities management (HFM) are based on an “if it breaks, we fix it” approach. There is still no effective real-time monitoring of health care facilities, causing a lack of effective preventive maintenance. We propose digital twin (DT), which can monitor equipment and systems in real time and enable bidirectional coordination, to address the shortcomings of current health care facilities management processes. DT is the virtual representation of a physical asset in all instances and in real time. This study aims to identify the shortcomings in HFM and propose a DT prototype to improve HFM. The study adopts a mixed methodology including a literature review, a case study integrating interviews, and scenario planning. The study identified basic DT concepts and data dynamics. The study identified the adopted DT components including virtual design/modeling technologies, sensors, data analytics techniques, communication networks, and mobile devices, including the components integration and data flow management of the DT platform. Functional implementations for scenario deployment were also developed. The functional implementations addressed the following scenarios: monitoring the performance of the HVAC system using pressure criteria, monitoring indoor air quality (IAQ), monitoring the performance of pulsed light for deactivating bacteria and viruses and radio frequency (RF), as they affect the performance of the magnetic resonance imaging (MRI) machine. This paper concludes by highlighting the benefits and limitations of DT application in health care facilities management.

Practical Applications

This study focused on DT-based health care facilities management. The paper elaborated on health care facilities management and identified some of the various shortcomings. Some of the shortcomings include a lack of effective real-time updates on the performance of the facilities and their system, lack of control of a physical system from the virtual model, and lack of building information modeling (BIM) integration with facilities management systems. The study further identified some FM software and its shortcomings. The findings from the study were supported using a case study approach that involved some health care facilities management. The facilities management and DT experts were used to help develop four scenarios that addressed some of the identified HFM issues identified in a different study. Subsequently, functional specifications explaining the step-by-step implementation processes of the proposed scenarios were developed. Although the study is still at its planning and development stage, potential benefits and limitations based on the proposed DT system were identified, including the contributions of the study. The study provides potential applications of the models discussed, and the implementation of the proposed system is currently in progress.

Modular construction can improve construction performance (i.e., cost, schedule, and safety) by prefabricating modules at an off-site facility and installing them at a construction site. However, when defects of modules are not easily repairable on the construction site, they cause additional cost overruns and delays due to long lead times of refabrication and reshipment. Thus, quality assessment of modular components at the fabrication facility before shipment is very important. The current inspection practices rely on manual measurement, which can be imprecise, labor-intensive, and time-consuming. To address this issue, some research efforts are made on the module inspection techniques (e.g., estimates of geometric properties and surface quality) using laser-scanned data. The accuracy of these techniques relies on the quality (i.e., coverage and resolution) of the scan data. However, ensuring the consistent quality of data is a major challenge as there is little to no research on optimal scan planning for modular components. Therefore, this paper proposes a model-based 3D scan planning method for modular components that ensures user-specified scan quality. Given a 3D computer-aided design (CAD) or building information modeling (BIM) model, scanner property, and user’s quality requirement, this method automatically computes the input parameters for the laser scanner (i.e., angular step and field of view) and optimal scan positions. It also predicts the scan quality and shows the areas that will not meet the user requirement due to geometric constraints (i.e., self-occluded surfaces). This study was validated through two case studies using two modular-sized structures in a fabrication facility. The results showed that the scan planner is able to accurately predict the scanning quality and ensure that the output scan will meet the user quality requirement.

The heating, ventilation, and air conditioning (HVAC) system is one of the most complex parts in a building design, which requires a high level of specialty to interpret and analyze during building energy modeling. From an energy simulation perspective, significant efforts are required in extracting HVAC information from the two-dimensional (2D) mechanical drawings or three-dimensional (3D) design models and manually inputting the data into the energy models. This tedious, error-prone, and time-consuming process has hindered high productivity in energy analysis. Automatically transforming HVAC data already contained in building information modeling (BIM) into building energy modeling (BEM) can significantly accelerate this process and improve efficiency during design iterations. To automate this process, the authors proposed a new algorithmic method by leveraging the state-of-the-art data-driven reverse engineering algorithm development (D-READ) method and the invariant signatures of HVAC objects. Following the proposed method, an algorithm was developed using a 2-story residential building model and a 2-story office building model. It successfully transformed HVAC data from BIM to EnergyPlus input file following the Industry Foundation Classes (IFC) standard. It was tested on two testing models with different HVAC systems [i.e., a 4-story office building model with a boiler radiator system and a 2-story clinic building with a variable air volume (VAV) system], which achieved 97.5% and 98.7% transformation accuracy compared with evaluation models manually created in commercially available software, respectively. Results also showed a satisfactory precision ($<9.6\%$ error) in total energy consumption by the algorithmically generated model when compared with the evaluation model. This work provides a new IFC-based approach to address the research gap of HVAC interoperability between BIM and BEM and supports better accessibility compared with a proprietary workflow. It builds a solid step for realizing seamless and fully-automated HVAC information transformation between BIM and BEM, for complete BIM-BEM interoperability.

Tag-based visual-inertial localization is a lightweight method for enabling autonomous data collection missions of low-cost unmanned aerial vehicles (UAVs) in indoor construction environments. However, finding the optimal tag configuration (i.e., number, size, and location) on dynamic construction sites remains challenging. This work proposes a perception-aware genetic algorithm-based tag placement planner (PGA-TaPP) to determine the optimal tag configuration using four-dimensional (4D) building information models (BIM), considering the project progress, safety requirements, and UAV’s localizability. The proposed method provides a 4D plan for tag placement by maximizing the localizability in user-specified regions of interest (ROIs) while limiting the installation costs. Localizability is quantified using the Fisher information matrix (FIM) and encapsulated in navigable grids. The experimental results show the effectiveness of our method in finding an optimal 4D tag placement plan for the robust localization of UAVs on under-construction indoor sites.

Geometric features, such as normal and curvature, have been prominently used for point cloud-based unsupervised spall classification. In addition, some researchers use hand-crafted features (e.g., out-of-plane distance, eccentricity, principal curvatures in 2D slices). These features perform well in low noise settings; however, the performance tapers down significantly when the quality of point clouds is affected by factors such as higher noise and inconsistent point-to-point spacing. Instead of relying purely on handcrafted features, the research presented in this paper investigates the potential for combining domain knowledge with deep learning to automatically learn better quality defect-sensitive features for point cloud-based spall classification. Specifically, generic three dimensional (3D) shape and 3D neighborhood features have been encoded as inputs to a deep autoencoder network for self-supervised spall classification from point clouds. Overall, this approach only resulted in marginal improvement over classification results from the current state-of-the-art unsupervised approach that uses handcrafted features. However, significant improvement in the results were observed in datasets that had higher noise levels. Given that noise is pervasive in datasets from outdoor settings like civil infrastructure, this added robustness to noise improves the reliability of point cloud-based condition assessment for concrete bridges.

Image segmentation is an essential step in vision sensing and image processing. It enables the understanding of the object’s classes, spatial locations, and extents in the scene, which can be used to support a wide range of construction applications such as progress monitoring, safety management, and productivity analysis. The recent ground-breaking achievements of deep learning-based approaches for semantic segmentation are at the cost of expensive large-scale training datasets annotated at the pixel level. Although building information modeling (BIM) has been leveraged to alleviate labeling costs using automatically generated, color-coded images as semantic labels, the differences between the BIM models and the real-world scenes make it difficult to apply networks trained on BIM-generated labels to real images. Furthermore, it takes nontrivial efforts to reduce such differences. To address these problems, this paper proposes a weakly supervised segmentation approach that uses inexpensive image-level labels. The missing boundary information in image-level labels is compensated by BIM-extracted object information. The proposed method consists of three modules: (1) detect initial object locations from image-level labels; (2) extract object information from BIM as prior knowledge; and (3) incorporate the prior knowledge into the network to enhance the detected object locations. Three extensive experiments are designed to evaluate the effectiveness of the proposed method. Results show that the proposed method substantially improves the detected object areas by using prior knowledge of target objects from BIM and outperforms the state-of-the-art weakly supervised methods.

Data sharing between different organizations is critical in supporting decision making in civil infrastructure engineering projects (e.g., transportation projects). Understanding the characteristics of data-sharing-related factors in civil infrastructure engineering is crucial for the civil engineering community to identify the priorities in promoting data sharing. The ASCE Data Sensing and Analysis (DSA) Committee initiated an investigation on data-sharing barriers in civil infrastructure engineering. This work produced (1) a classification of data-collection and data-use contexts in the domain of civil infrastructure engineering; (2) a comprehensive classification of data-sharing barriers and data-sharing promotion strategies in this domain; and (3) a synthesis of the opinions about the current practices, future vision, barriers, and data-sharing promotion strategies from civil infrastructure professionals and researchers using an online survey. The survey results revealed that (1) more data-sharing activities are occurring within the organizations, such as academia, government agencies, and for-profit corporations, compared with data exchanges among these communities; (2) “if there is a cost sharing in the data sharing,” is the condition that makes data providers more willing to share data with others; and (3) compared with technology improvements, the protocol of data sharing in the business domain has more potential in promoting data sharing. Meanwhile, this research identified the top five priorities among 20 data-sharing promotion strategies, which are protocol in industry and organization, data-standard regulation, data validity, data security protection, and large-scale data-sharing technology. These five priorities play critical roles in a roadmap showing how policy, business models, and technical solutions should cooperate to promote data sharing in civil infrastructure engineering.

Human motor skills are critical for executing a variety range of tasks in construction. Traditional hands-on training is resource and labor intensive, whereas virtual training, such as video demonstrations, cannot provide trainees with egocentric kinesthetic or proprioceptive experience such as muscular engagement. It is important to develop remote training methods that can provide rich sensory feedback and leverage the trainee’s proprioception. This paper proposes a novel remote motor skill training system that can transfer experts’ kinematic and kinesthetic experience, including both positional and force experience, to novice trainees by using virtual reality (VR) and a robot arm without the physical presence of the experts. The system uses VR to simulate virtual operation scenarios and interactions to provide an immersive operation experience. The robotic system records experts’ kinematic and kinesthetic patterns and trains novices with perceptual learning. The system design was demonstrated with a welding training task. A welding simulator was built with a Unity engine and a seven-degrees-of-freedom robot arm, which provided high-fidelity welding experience and could actively guide welding trainees. It was found that the welding simulator was resilient to external disturbance and provided accurate feedback and guidance. The proposed system contributes to the design of a more embodied remote motor skill training method.