Benchmarking Data-Driven Site Characterization

The special collection “Benchmarking Data-Driven Site Characterization” is available in the ASCE Library (https://ascelibrary.org/ajrua6/benchmarking_data_driven_characterization).

Data-centric geotechnics is an emerging field that advocates a “digital first” agenda (Phoon et al. 2022a). One aspect of this agenda is the development of methods that make sense of actual data. Perhaps more importantly, it calls for the industry to develop methods without imposing thresholds on quantity, quality, and/or other “ugly” data attributes as prerequisites for exploring data-informed decision-making. In short, the industry should engage in the spirit of offering our data an opportunity to speak for itself.

The end goal of any analysis is to improve decision-making in geotechnical engineering, which is always related to a project carried out at a specific site. It is a given that the role of engineers in the design process is indispensable because they take ultimate responsibility for the project. Improving an engineer’s site-specific knowledge of the ground conditions will surely enhance this role. It is natural for data-driven site characterization (DDSC) to attract the most attention in data-centric geotechnics (Phoon et al. 2022b).

One purpose of DDSC is to produce a three-dimensional (3D) stratigraphic map of the subsurface volume below a full-scale project site and to estimate the governing engineering properties and soil type at each spatial point based on site investigation data at the target site and relevant big indirect data (BID) from other sites, neighboring or otherwise. However, this is a significant challenge because the “ugly” attributes of actual site data are MUSIC-3X [multivariate, uncertain and unique, sparse, incomplete, and potentially corrupted with 3D spatial variability (3X)]. The widely held perception that data sparsity is a strong hindrance to DDSC is incomplete in terms of the scope of the challenge. In particular, site uniqueness is a fundamental feature of geotechnical engineering, but there is no fully satisfactory data-driven method to characterize this “U” attribute (also called intersite variability) in BID. For example, Clause 2.4.5.2(10) of Eurocode 7 (CEN 2004), “Characteristic Values of Geotechnical Parameters” highlights the importance of accounting for “U” but does not provide a quantitative method to do so: “If statistical methods are employed in the selection of characteristic values for ground properties, such methods should differentiate between local and regional sampling and should allow the use of a priori knowledge of comparable ground properties.” The jury is still out on many important questions pertaining to the kinds of data sets that will provide better support for decision-making at a specific site. Do we use the entire BID or a subset of “similar” sites (a quasi-regional cluster) (Phoon and Ching 2022)? Do we collect only high-quality data that are limited in quantity or only lower-quality data in larger quantity? Do we combine them? When should they not be combined?

DDSC will fill an existing gap in Building Information Modeling (BIM) where a digital model for the subsurface is largely missing. An exciting prospect is risk- and reliability-informed design at the systems level using BIM as a platform that may eventually evolve into a digital twin to support smart infrastructure life-cycle management. As such, research on DDSC methods has been active in recent years. Because site-specific data are incomplete and sparse, these DDSC methods—Bayesian machine learning being the most popular—are mainly probabilistic . However, there are other methods (Phoon and Zhang 2023). At present, it is not known which method is more appropriate and/or more efficient for given circumstances. The answer must largely depend on the attributes of the training data and the desired outcomes of DDSC. To promote the development of DDSC methods that are applicable to routine projects in a more purposeful way, one important step is to create reference geotechnical data sets and standard performance metrics to train and measure different DDSC methods on a uniform basis. This benchmark testing, or benchmarking, is widely used in machine learning (ML) to support unbiased and competitive evaluation of emerging ML methods (Thiyagalingam et al. 2022). In a relatively early attempt at establishing a benchmarking framework for DDSC, Phoon et al. (2022c) recommended the following considerations for benchmark examples: (1) they must be 3D and realistic in scale, stratigraphy, and properties; (2) they must cover a sufficient range of ground conditions; (3) they must restrict the training data set to data that engineers have at their disposal (MUSIC-3X or other data attributes); and (4) they must select validation data sets and performance metrics that showcase the value of DDSC to decision-makers [to meet the “fit for (and transform) practice” requirement in data-centric geotechnics].

Phoon and Zhang (2023) classified value according to “ML advantage” and “ML supremacy.” For the former, the ML method is effective but not indispensable. Hence, its value to practice is incremental. For the latter, the ML method can solve problems where conventional methods are found to be impractical or ineffective. Hence, its value to practice is significant. A simple “value” matrix is also proposed to guide future research: (1) Type 1 (incremental value) involving available data and existing conventional applications, (2) Type 2a (potentially high value) involving available data and new applications, (3) Type 2b (high value) involving new data and existing applications, and (4) Type 3 (disruptive value) involving new data and completely novel applications.

This special collection contains seven papers. Some of the research was presented at the second TC304 Forum “Data-Driven Modeling for Geotechnical Spatial Variability” organized by Tokyo City University and the Japanese Geotechnical Society. The Forum took place online on July 3, 2021. Its purpose was to compare the performance of five data-driven methods in site characterization using standard benchmark examples and performance metrics (including computational cost).

Phoon (2023) reviews reliability applications and real-world databases that are mature and already in use or promising new developments in data-centric geotechnics that may transform practice in the near future. The paper situates geotechnical reliability as one step toward a broader and deeper exploitation of data for decision-making: (1) at a real-time system-wide scale, (2) in hybrid intelligence (human, machine) mode, (3) for a specific project [e.g., “precision construction” in Phoon (2018)], and (4) to address complex design goals beyond safety and economy (e.g., sustainability, resilience). This state-of-the-art review can be regarded as a timely update to National Research Council (1995) and as reopening the conversation with a focus on what is useful for practice and what has changed in a material way that increases or decreases the need for probabilistic methods in practice. The National Research Council report, “Probabilistic Methods in Geotechnical Engineering,” has been influential, but its impact on the broader community has been mainly restricted to the load and resistance factor design (LRFD). Phoon (2023) argues that reliability or other uncertainty quantification methods have become more rather than less important since their inception in the 1960s because they exploit data in more systematic and comprehensive than deterministic ways, which are critically needed to address complex new challenges (ASCE 2019). Data infrastructure is now considered to be as important as physical infrastructure. This is not well appreciated by most engineers, although it is pivotal to digital transformation (Phoon et al. 2022a). Given the growing ubiquity of digital technologies, this state-of-the-art review concludes that the basis for decision-making will shift from a data-informed physics approach to a data-driven approach that may or may not be based on physics.

The following five papers present the benchmarking results of different DDSC methods based on the virtual ground examples in Phoon et al. (2022c). Lyu et al. (2023) apply a DDSC method based on Bayesian compressive sampling/sensing (BCS) for benchmarking and implement a user-friendly in-house software known as Analytics of Sparse Spatial Data (ASSD-BCS). A new benchmark example based on an actual cone penetration test (CPT) data set collected at a site in Adelaide, Australia (Jaksa 1995) is also proposed. Mavritsakis et al. (2023) propose a Bayesian site characterization framework (BaySiC) and report its benchmarking results. In BaySiC, a vector random field model containing two CPT parameters (tip resistance and sleeve friction) is used. The random field is assumed to be normally distributed. Cross-correlation between tip resistance and sleeve friction and spatial correlations with depth are considered. The hyperparameters of the model are estimated using the Hamiltonian Markov Chain Monte Carlo (MCMC) method. The authors formulate two models using BaySiC, aggregated model and split, and report that the former is effective in predicting tip resistance while the latter is effective in identifying material type. Tomizawa and Yoshida (2022) implement Gaussian process regression (GPR) using multiple Gaussian random fields (GPR-MR) for DDSC. GPR has recently attracted renewed attention and is being actively studied for DDSC (Yoshida et al. 2021; Ching et al. 2023). In GPR-MR, three autocorrelation functions (or kernel functions), Gaussian, Markovian, and Whittle-Mattern, are considered and the best one is selected based on the Akaike information criterion (AIC) and Bayesian information criterion (BIC). Ching and Yoshida (2023) compare the performance of two DDSC methods, sparse Bayesian learning (SBL) and GPR-MR, through benchmarking. Bayesian evidence is computed in the process of hyperparameter tuning and used to compare the performance of the two DDSC methods. The authors also propose a two-step algorithm that applies clustering (cluster analysis) for stratification before spatial interpolation. Shuku and Phoon (2023a) propose a new benchmark example, small virtual ground 5 (S-VG5) based on a more heterogeneous virtual ground than the four in Phoon et al. (2022c). The 3D virtual ground is created by a category-based approach that generates complex geological structures through iterations of simple mathematical rules such as a Markov chain. An alternate approach is to apply a graphical Markov random field model (Shuku and Phoon 2023b). In addition, the authors compare three DDSC methods, Glasso, Glasso-BFs (Glasso with basis functions), and GPR, from a Bayesian perspective. This allows the methodological similarities and differences to be explained in a theoretically consistent way.

Finally, Hu et al. (2023) present a framework to assess the effectiveness of a sampling layout in a site investigation program based on uncertainty reduction. The authors propose an index called expected reduced variance (ERV) for this purpose. If ERV is large, the investigation program is effective. Crude Monte Carlo and bootstrap methods are compared in evaluating the statistical uncertainty of ERV, and it is reported that the bootstrap method is computationally more efficient than crude Monte Carlo. Two design examples (shallow foundation and slope) are presented to demonstrate the proposed method.

The guest editors thank Professor Michael Beer, editor of the ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, for his encouragement and support to complete this special collection. The efforts of the authors and reviewers are also deeply appreciated.