Metaresearching Structural Engineering Using Text Mining: Trend Identifications and Knowledge Gap Discoveries

The LDA topic model is a generative probabilistic model that focuses on identifying key topics from a collection of textual documents (Blei et al. 2003). Several previous studies (e.g., Steyvers and Griffiths 2007; Amado et al. 2018) demonstrated how this model can be used to examine the content of scientific text documents. This is because LDA is capable of significantly reducing the number of dimensions (i.e., words) in a document while still retaining the essential relationships between all the dimensions and their key topics in the underlying documents (Blei et al. 2003). As such, the LDA topic model was extensively applied to address a wide range of problems (e.g., Amado et al. 2018). LDA was originally developed based on the classical probabilistic latent semantic analysis model introduced by Hofmann (1999). The basic concept behind this model is that, for any collection of textual documents, each document contains a large number of words. Based on the probability of co-occurrence of different words within the same document, considering all documents, different topics (each made up of different sub-sets of words) can be identified. The words comprising these topics are weighted according to their statistical distribution within each topic (Blei et al. 2003). As such, each document can be represented by the combination of (latent) topics. Such representation significantly reduces the complexity of handling a massive number of textual documents, thus facilitating discovering key topics through their latent inherent statistical relationships (Hofmann and Chisholm 2016).

The LDA model initially defines a number of topics, $K$, where each topic, $k$, contains a distribution of words, ${\psi }_k$, that is evaluated based on a Dirichlet distribution ($\beta$). Based on these topics, each document, $d$, is analyzed by sampling a document topic distribution, ${\theta }_d$, over $K$ topics, where ${\theta }_d$ is quantified from another Dirichlet distribution ($\alpha$) to assign a topic for each word, $w_{di}$, in $d$. For each $w_{di}$ within each $d$, LDA first assigns a particular topic, $Z_{di}$, that belongs to $K$ based on the multinomial distribution (${\theta }_d$), and subsequently, $w_{di}$ is selected from multinomial distribution ${\psi }_{Zdi}$. The full details of Dirichlet and multinomial distributions can be found in Minka (2000) and Correa (2001), respectively.

LDA utilizes several inference algorithms to evaluate the document topic distribution, $\theta$, and word distribution, $\psi$. One of these algorithms is the Gibbs sampling algorithm that was originally introduced by Geman and Geman (1984) and utilized to discover topics by Griffiths and Steyvers (2004). This algorithm assigns a value to each word in the model, $w_{di}$, and estimates the probability of assigning $w_{di}$ to each topic, $k$. Although other algorithms [e.g., variational expectation maximization, introduced by Blei et al. (2003)] can also estimate $\theta$ and $\psi$ efficiently, this study utilizes the Gibbs sampling algorithm, as several studies have demonstrated that similar key topics can be inferred, regardless of the algorithm that is used (Hofmann and Chisholm 2016).

It should be noted that the current study does not focus on capturing how a specific topic evolves over time, and dynamic topic modeling approaches (e.g., Blei and Lafferty 2006; Wang and McCallum 2006) are neither within the scope nor interest of the analysis presented herein. Instead, the current study considers all topics over the 26-year studied period, focusing on the overall variation within that time window. Therefore, within its scope and objectives, the current study considers the approach by Griffiths and Steyvers (2004) to be the most appropriate through adopting the label-unaware method and subsequently aggregating the results per label.

The current study utilizes a non-joint modeling approach to understand the overall temporal distribution of the identified topics within all papers published over the period from 1991 to 2016 in the considered journals. In this respect, the proportion of topic $k$ distribution over time, $t$, for all the articles, ${\theta }_k^t$ is evaluated following the approach developed by Gatti et al. (2015)where $M$ = total number of articles, ${\theta }_{dk}$ = proportion of topic $k$ in document $d$, $t_d$ = publication year of document $d$ and $t$ = index of years. The proportion of topic $k$ for all the articles in journal $j$, ${\theta }_k^j$ (Gatti et al. 2015) is the following:where $j_d$ = journal of document $d$, while $\nabla (j_d=j)$ = one if ($j_d=j$) is true and zero otherwise. The proportion of topic $k$ distribution in journal $j$ over time $t$, ${\theta }_k^{jt}$, is quantified as the following:where $\nabla (t_d=t,j_d=j)$ is the number of articles for journal $j$ in year $t$. As such, ${\theta }_k^{jt}$ indicates that articles are aggregated for a particular journal $j$ and for the topic $k$ and year $t$ of interest.

The two key journals in the field of structural engineering research, Journal of Structural Engineering (JSE) and Engineering Structures (ES), were selected in this study. Both journals are top tiered in the field, according to Science Citation Index (SCI), and comprehensively contribute to structural engineering research in all its subfields. In addition to the qualitative (e.g., reputation) and quantitative (e.g., impact factor) attributes, unlike subdiscipline-focused journals (e.g., ACI, Wind Engineering, and Earthquake Spectra), the former are the two main journals covering the breadth of structural engineering subfields—which is key for the current metaresearch study.

During the data collection, the inclusion of an article was stratified using the following criteria: (1) articles published from 1991 to 2016; (2) articles with abstract containing technical information; (3) articles with abstracts more than 10 words; and (4) articles listed under “Technical Paper” and “Research Paper” categories in JSE and ES, respectively. In this respect, the abstract dataset is based on a total of $M=\mathrm{11,027}$ articles that span 26 years. The number of articles for each journal is shown yearly and cumulatively in Figs. 1(a and b), respectively. As shown in the figures, the annual number of articles published in JSE is almost constant, while the same number in ES has increased in general over time with a dramatic boost starting in 2006.

The presence of linguistic noise is a common problem that negatively influences any statistical analysis within the context of TM (Salloum et al. 2018). This linguistic noise is mainly attributed to the variations in case types (e.g., DESIGN and design), word forms (e.g., experiment and experimental), and the presence of common words (e.g., the, and, of) and special characters (e.g., punctuation). As such, the raw abstract dataset was further processed through five steps (Zhang et al. 2015): (1) tokenization, where all the abstract words were separated into tokens; (2) treatment, where a standard filter stop list in natural language processing was utilized to remove common words; (3) transformation, where all the characters were structured in a lowercase format; (4) stemming, where all the affixes were removed to return words to their word stem; and (5) cleaning, where the abstract dataset was treated to remove all words with more than 25 or less than 4 characters. These preprocessing steps minimized the abstract dataset from 35,579 raw words to 7,795 clean words. Fig. 2 shows a comparison between the raw and clean abstract datasets through word clouds, where the size of each word is in proportion to its probability of occurrence. As shown in Fig. 2, while the raw dataset (i.e., 35,579 words) contains several common words with very high frequency (e.g., the, and, of), the clean dataset (i.e., 7,795 words) has several high-frequency words that are related to structural engineering (e.g., concrete, steel, model, design), demonstrating the importance of these preprocessing steps in enhancing the dataset quality for meaningful and non-trivial analyses.

The LDA model requires some basic input parameters to apply an inference algorithm on the underlying abstract dataset. First, this model requires the hyperparameters $\beta$ and $\alpha$ that control both the mean shape and sparsity of ${\psi }_k$ and ${\theta }_d$, respectively, from the underlying Dirichlet distribution, as discussed earlier. Typically, larger $\beta$ and $\alpha$ result in uniform topic distributions, while their smaller counterparts yield sparser topic distributions (Griffiths and Steyvers 2004). Since key topics of structural engineering research are relatively well identified, small values of $\beta =0.01$ and $\alpha =5/\mathrm{number}$ of topics, were assumed in the current study to follow sparse topic distributions (Griffiths and Steyvers 2004).

Second, the optimum number of key topics is one of the main challenges in topic modeling. To address this challenge, perplexity analysis was performed to quantify how well the LDA model can predict the key topics of the abstract dataset (Blei et al. 2003). Perplexity is a standard performance measure for natural language statistical models. More specifically, for a given number of topics, the LDA model is developed, and the synthesized word distributions, represented by the corresponding topics, are compared to the actual topic mixtures, or distribution of words in the documents comprising the dataset (Blei et al. 2003). In this respect, the dataset was divided into 9,925 (90%) and 1,102 (10%) articles for training and testing, respectively. Afterward, for the training dataset of $\mathrm{1,102}$ articles, the perplexity was evaluated (Blei et al. 2003) as follows:where $W_d$ and $N_d$ = word collection and number of words in document $d$, respectively. Fig. 3 shows the sensitivity of the perplexity to the number of topics, $K$, from 10 to 160. As shown in Fig. 3, although the minimum perplexity is attained at $K$ value of 160, the current study adopts a $K$ value of 40 in order to deal with a practical number of topics, especially considering the small difference in the perplexity value at $K=40$ versus that at $K=160$.

Eq. (1) was utilized to evaluate the overall temporal variation of each topic inclusion in the articles published over the 26-year period considered, as shown in Fig. 5 and presented in Appendix II. In this figure, the 40 topics identified earlier are presented in order (i.e., from Topic No. 1 to Topic No. 40) from the bottom to the top to show their corresponding probability of inclusion in the JSE and ES abstract dataset from 1991 to 2016. The level of inclusion of any topic (e.g., Topic No. 20) is represented by the area between this topic and the previous topic (i.e., Topic No. 19), as shown in Fig. 5. Fig. 6 shows the temporal variation of five sample technical topics: Topic No. 11 (systems dynamic response) “impact, force, model, dynamic, effect, mass, velocity, response, etc.”; Topic No. 12 (structural design) “code, limit, factor, design, standard, practice, criteria, specification, etc.”; Topic No. 16 (systems control) “sensor, active, hybrid, control, linear, system, friction, passive, etc.”; Topic No. 35 (nonlinear finite element analysis) “model, software, nonlinear, geometry, material, numerical, finite, element, etc.”; and Topic No. 40 (composite structures) “steel, beam, slab, strengthening, connector, fiber, bond, retrofit, etc.” As shown in Fig. 6, the inclusion of Topic No. 40 (composite structures) has been increasing, where its topic probability increased from 0.016 in 1991 to 0.028 in 2016, while inclusions of Topics No. 12 (structural design) and 35 (nonlinear finite element analysis) have been decreasing with topic probabilities of 0.038 (in 1991) to 0.032 (in 2016) and 0.043 (in 1991) to 0.029 (in 2016), respectively. Some topics have also been experiencing either consistent (e.g., Topic No. 11 systems dynamic response) or sporadic (e.g., Topic No. 16 systems control) inclusions, as also shown in Fig. 6.

To quantify the inclusion increase (or decrease) of each topic within the articles published between two temporal windows (years), a contribution index, $C_k$, was calculated for each topic using Eq. (5) (Gatti et al. 2015)where $C_k>1$ indicates that the inclusion of topic $k$ has increased from (1991–1995) to (2012–2016), whereas $C_k<1$ indicates that topic $k$ has declined within the same two temporal windows. Fig. 7 shows the calculated $C_k$ values for all 40 topics, whereas Figs. 8(a and b) present the temporal variation of only five technical topics with the highest and lowest $C_k$ values to show the temporal variation of the topic inclusions, respectively, from 1991 to 2016 within the considered structural engineering research literature. As shown in Figs. 7 and 8, Topic No. 5 (reinforced concrete columns testing), Topic No. 25 (material properties), Topic No. 28 (earthquake/seismic engineering), Topic No. 31 (fire testing/analysis), and Topic No. 32 (masonry testing) have been experiencing inclusion increase, whereas Topic No. 7 (soil-structure interaction), Topic No. 20 (plate analysis), Topic No. 24 (bridge engineering), Topic No. 27. (prestressed structures), and Topic No. 30 (buckling) have all been experiencing inclusion reduction.

To further investigate the inclusion proportion of each topic over time within both journals (Fig. 9 and Appendix III), Eq. (3) was used. Figs. 9(a and b) demonstrate that topic distribution and proportion of JSE and ES, respectively, are almost consistent over time, confirming that both journals possessed very similar temporal author interests and research compositions. The figure also shows that some topics [e.g., Topic No. 24 (bridge engineering) in JSE and Topic No. 14 (vibration effects) in ES] have been declining over the last 15 years despite the significant increase in the number of research publications within the two journals. This decrease might be attributed to the lack of researchers with such interests, natural variation in research scopes in the last two decades, emergence of specialized journals in bridge engineering and vibration effects, or a combination of all such effects. Overall, the trend demonstrates that some topics have been decreasing as other topics have been the focus of more articles. For example, Topic No. 37 (uncertainty and reliability) in JSE and Topic No. 22 (structural health monitoring) in ES have been experiencing increasing inclusion, as shown in Figs. 9(a and b), respectively.

All the previously identified topics were inferred based on the 1991 to 2016 JSE and ES dataset, as discussed earlier and shown in Fig. 5. Therefore, these key topics might attenuate the influence of some topics that might have emerged only recently within the corresponding abstract dataset. Such attenuation might be attributed to the small number of publications possibly covering these emerging topics relative to the high volume of the total number of publications from 1991 to 2016. As such, to further evaluate the variation of research content and the potential of new emerging topics, the LDA model was applied on the abstract dataset from 2012 to 2016 only to infer the key topics within this temporal window, as shown in Fig. 10. This temporal window facilitates investigating the correlation between these key topics and their counterparts from 1991 to 2016, as shown earlier in Fig. 5. Fig. 10 shows that the inferred recent key topics are consistent and essentially the same over time, regardless of the temporal window considered. However, as shown in the figure, the main difference between the two temporal windows (i.e., from 1991 to 2016 and from 2012 to 2016) is the inclusion decrease of only two topics after 2012 (i.e., concrete creep and shrinkage and prestressed structures) and the inclusion increase in the same year of only one topic (i.e., shock effects).

Co-occurrence networks are generally used to analytically evaluate and visually present the potential relationships between people, organizations, and concepts or, as adopted herein, between entities present within a collection of textual documents (Wuchty and Almass 2005). As such, the current study develops a co-occurrence network of all words within the journal abstract dataset to investigate the interconnection between these words across all topics. This network comprises nodes, representing words, and links, representing the co-occurrence of different words within the same topic. The set of all nodes and links in the word co-occurrence network is {$N$,$L$}, and its adjacency matrix, $A$, is the following:

In Eq. (6), B = binary matrix of size ($V\times K$), and its elements $b_{vk}$ are given bywhere $b_{vk}=1$ indicates that word $v$ is an important component in topic $k$, where the threshold ${\psi }_k^v$ is taken as 0.05 (Manning and Schütze 1999; Hofmann and Chisholm 2016) in the current study. Therefore, according to Eq. (6), the adjacency matrix, $A$, has a size of $V\times V$ with $V=\mathrm{7,795}$, and its elements, $a_{vu}$, represent the number of topics that two words $v$ and $u$ tend to co-occur effectively when ${\psi }_k^v\ge 0.05$ and ${\psi }_k^u\ge 0.05$.

To visually investigate the structure of the co-occurrence network, the Yifan Hu (Hu 2005) graphical algorithm is adopted to the largest connected component of the network ($N=803$ nodes and $L=\mathrm{38,448}$ links). This algorithm utilizes the principle of repulsion and attraction strengths to facilitate rapid visual interpretation. More specifically, attraction draws linked words closer together, while repulsion forces unlinked words further apart from one another in proportion to the strength of such attraction and repulsion. Fig. 11 demonstrates the use of the Yifan Hu algorithm in highlighting the structure of the co-occurrence network, with the sizes of nodes scaled according to their number of direct links. As shown in this figure, some words are more interconnected relative to others within the same co-occurrence network. For example, Fig. 11 clearly demonstrates that, as expected, the words “reinforced” and “concrete” are more interconnected through several research topics compared to the words “reinforced” and “masonry”. The figure also shows some word clusters, revealing some research topics within the co-occurrence network (e.g., wind engineering, seismic isolation systems, and bridge engineering), thus showing the key bridge words connecting these topics. For example, the word “column” connects the “steel columns testing” and the “RC columns testing” topics. In addition, the co-occurrence network can be used as a tool to evaluate the conception distance (i.e., number of research studies) between topics. For example, Fig. 11 shows that some topics (e.g., bridge engineering and wind engineering) are more interconnected through several research studies as opposed to other topics (e.g., bridge engineering and seismic isolation systems) within structural engineering research, thus highlighting possible research opportunities in and knowledge gaps within the latter subfield intersection.

The current study further develops an interlinkage matrix, $S$, for all the 36 technical research topics within the dataset to visualize the interlinkages (and quantify their strengths) between these topics

In Eq. (8), C = binary matrix of size ($K\times M$), and its elements $c_{kd}$ is given bywhere $c_{kd}=1$ indicates that topic $k$ is an important component in document $d$. Therefore, according to Eq. (8), the interlinkage matrix, $S$, has a size of ($K\times K$) (i.e., $K=36$); and its elements, $s_{jk}$, represent the number of articles that two topics $j$ and $k$ tend to co-occur in considering the ${\psi }_d^j\ge 0.01$ and ${\psi }_d^k\ge 0.01$ thresholds. Fig. 12 shows the interlinkage matrix, $S$, normalized to the total number of articles (i.e., $M=\mathrm{11,027}$) to facilitate direct comparison between the strengths of the interlinkages between all topics. The visualization in Fig. 12 presents a valuable tool for researchers, journal editors and funding agencies/conference organizers, whereas the interlinkages and their strengths can be used to possibly identify knowledge gaps and future strategic directions. For example, Fig. 12 shows that Topic No. 1 (earthquake/seismic engineering) is highly interlinked to Topic No. 38 (analysis methods), Topic No. 34 (nonlinear modeling), and Topic No. 12 (structural design) through strength values of 0.44, 0.43, and 0.42, respectively. Conversely, the same topic (i.e., Topic No. 1) is essentially unlinked to Topic No. 31 (fire testing/analysis), Topic No. 24 (bridge engineering), and Topic No. 15 (wind engineering) through strength values of 0.06, 0.09, and 0.10, respectively, at least within the two journals. However, following an earthquake, for example, fires typically erupt for different reasons (e.g., due to the rupture of either electrical or gas lines) (i.e., Topics 1 and 31). As such, to provide an overall image of the current and possible future structural engineering research directions, the interlinkages between all 36 topics are shown in Appendix IV in the order of their strength. For example, as shown in Appendix IV, while most intersection areas essentially present Red Oceans (Kim and Mauborgne 2014) of well-established interlinked topics with a large number of publications [$s_{jk}>0.10$] (e.g., torsion effects and analysis methods), there are also a few remaining scattered Blue Ocean (Kim and Mauborgne 2014) opportunities within the 36 topics with $s_{jk}<0.10$ (e.g., the intersection of wind engineering and systems control) that might still require additional future research efforts from the structural engineering community.

Although adopting the approach by Griffiths and Steyvers (2004) facilitates evaluating the overall linkage strength between topics to discover knowledge gaps for future opportunities, integrating other data sources is a possible future extension of the current study. Such analysis would facilitate quantifying the growth/variation of research topics and evaluating the potential of new emerging topics over time. In such case, the dynamic topic model (Blei and Lafferty 2006) or the topic-over-time model (Wang and McCallum 2006) can be utilized to further validate the results presented in the current study.

At its core, Blue Ocean Strategy (Kim and Mauborgne 2014) refers to creating value through discovering new needs and thus uncontested spaces, Blue Oceans, instead of struggling to survive in established markets (spaces) that are swarming with vicious competition, Red Oceans (referring to shark-infested waters). The metaresearch analysis performed herein shows that the structural engineering research community continues to produce more journal articles, as shown in Fig. 1, albeit with the majority of these articles essentially within the same established focus areas within the 26-year period considered (Red Oceans). As such, it might be argued that the structural engineering research community is missing immense Blue Ocean opportunities of extending and branching out its knowledge beyond these established boundaries. For example, although emerging, our research output falls short in terms of addressing some key societal needs including for example topics such as resilience, sustainability, and lifecycle design (Bonstrom and Corotis 2014; Cimellaro et al. 2016; Lounis and McAllister 2016; Salem et al. 2017). Such topics were not identified with any appreciable percentage over the 26-year study period, or even when only the latest 5 years within that time window were considered. Studies addressing such societal needs are more relevant now than ever before because of the complexity of the challenges being experienced by different structural systems including those related to climate change-induced hazards, aging populations, deteriorating infrastructure, and natural/anthropogenic interdependent hazards and subsequent systemic (cascade-type) risks (Ezzeldin and El-Dakhakhni 2019).

Notwithstanding the opportunities identified through the weak interlinkage of some established structural engineering topics, the real Blue Oceans are, in the authors’ view, elsewhere. When one considers other engineering disciplines (e.g., mechanical and electrical engineering), it is easy to examine the growing impacts of idea fusions at the interfaces of different disciplines that disrupt entire well-established engineering fields. In our field, for instance, considering intelligent structural systems, to name but one major opportunity, such systems adapt their characteristics to mitigate excessive response and monitor their own conditions and are envisioned to also perform self-diagnosis and possibly even self-repair (Holnicki-Szulc and Soares 2013). The structural analysis, design, construction, and future upgrades of such systems create numerous multidisciplinary research opportunities at the interfaces between structural engineering and multiple emerging cross-cutting technologies. Such technologies, already disrupting the world around us, include data fusion, sensing, robotics, data-driven models, machine learning, unmanned ground or aerial vehicles, adaptive controls, artificial intelligence, virtual and mixed reality visualization, bio-inspired materials and structures, and three-dimensional (3D) printing (Manyika et al. 2013).

Finally, it is important to highlight that viewing the breadth of structural engineering research through a metaresearch lens should not be considered as orthogonal to or refuting in-depth research necessary in structural engineering articles. In fact, metaresearch is key to minimizing research waste (Chalmers and Glasziou 2009) of funds, time, human resources, subsequent low/no-use publications, and non-transferable knowledge and to distinguishing between over-saturated and grand-challenge research areas in order to facilitate effective and efficient research. Furthermore, a Blue Ocean lens highlights breakthrough opportunities through either cross-pollinating ideas within structural engineering subfields or at the interfaces of structural engineering and other fields/technologies, opportunities that may be hidden under the plethora of research articles published every year in traditional areas.