Advanced searchCitation search
Skip main navigation
Close Drawer MenuOpen Drawer Menu
Previous article
Next article
Free access
Technical Papers
Jul 10, 2024

Monitoring Mental Fatigue of Construction Equipment Operators: A Smart Cushion–Based Method with Deep Learning Algorithms

Authors: Lei Wang [email protected], Heng Li https://orcid.org/0000-0002-3187-9041 [email protected], Haitao Wu https://orcid.org/0000-0002-4804-3806 [email protected], Yizhi Yao [email protected], Changyuan Yu [email protected], Waleed Umer https://orcid.org/0000-0003-2419-4172 [email protected], Dongliang Han [email protected], and Jie Ma https://orcid.org/0000-0002-0827-3842 [email protected]Author Affiliations
Publication: Journal of Management in Engineering
Volume 40, Issue 5
https://doi.org/10.1061/JMENEA.MEENG-5913
PDF

Abstract

Construction equipment operators (CEOs), who are required to work in seated positions for prolonged periods, often develop excessive mental fatigue, causing human error-related accidents, lower productivity, and psychological illnesses. However, the current practice for assessing fatigue is limited on construction sites. Previous studies utilizing smartwatches, electroencephalography, or eye-tracking technologies are intrusive and not convenient since they require operators to wear special devices, while vision-based solutions are sensitive to lighting conditions and have serious privacy concerns. There is a demand for continuously and accurately monitoring CEOs’ mental fatigue levels without causing discomfort and aversion. This study introduces a noninvasive and noncontact smart cushion method to bridge the knowledge gap. We first developed a smart cushion system incorporating optical fiber sensors to collect human heartbeat and respiration data. Then, we adopted the Bidirectional Long-Short-Term Memory (BiLSTM) model to recognize fatigue states. An experiment was conducted in which data was collected from 16 subjects engaged in simulated excavation tasks. Experimental results demonstrate the feasibility of the proposed method, and the BiLSTM model obtained an accuracy of 94.0%. The proposed smart cushion method could also be convenient for understanding ergonomic risks resulting from prolonged sitting, a grave occupational health and safety problem that plagues various industries.

Practical Applications

This study presents a smart cushion–based framework to continuously monitor the mental fatigue states of construction equipment operators (CEOs) during daily work. The proposed solution has clear advantages: it (1) is nonintrusive since it no longer requires sensors attached to the skin of operators, (2) is not sensitive to dynamic lighting conditions and does not generate privacy concerns, (3) is easy-to-use since it can be placed on the operator’s seat or seatback and does not need additional power. Experimental results also demonstrated that the proposed Gaussian mixture model and Bidirectional Long-Short-Term Memory model can achieve effective data processing and accurate fatigue recognition. The proposed system can provide construction managers with a quantitative and reliable assessment tool to measure CEOs’ mental workloads and support the intervention (e.g., worker shifts and breaks). If wirelessly connected to a smartphone, the smart cushion can conveniently provide early warnings to operators and their managers to alert the operators at stake to take breaks/rests to avoid mental fatigue-related ill consequences. Moreover, the proposed solution could be used to study and prevent the risks resulting from prolonged sitting, which is a grave occupational health and safety problem plaguing many industries.

Introduction

Heavy equipment, such as excavators, cranes, scrapers, and trucks, is indispensable in nearly every construction project. Unfortunately, it brings significant safety concerns due to its vast size and broad coverage area. The Bureau of Labor Statistics found that construction and extraction occupations had the second most fatalities (1,056) in 2022 in the US (BLS 2022). Struck by moving vehicles, struck by moving objects, and falls from a height accounted for around two-thirds of fatalities in 2022 in the UK (HSE 2022). The mental fatigue of construction equipment operators (CEOs) is one of the prevailing factors in the above accidents since it contributes to operators’ cognitive failures and operation errors (Fang and Cho 2017; Hu and Lodewijks 2020). CEOs tend to experience mental fatigue in practices due to extended working hours and high workloads. Tam and Fung (2011) found that about 52.6% of invited crane operators did not have a break every working day, and 60.5% of tower crane operators will continue working even if fatigued. Consequently, continuous monitoring of CEOs’ mental fatigue remains an imperative concern within the industry (Li et al. 2019c, 2020, 2021a; Mehmood et al. 2022).
Various methods have been proposed for CEO mental fatigue monitoring, including survey questionnaires (Techera et al. 2018), wearable sensors (Li et al. 2019c, 2020, 2021a; Xing et al. 2019; Park et al. 2023; Lee et al. 2020), and vision-based methods (Giannakakis et al. 2017; Mehmood et al. 2022). Although existing methods have benefits, several limitations inhibit their use in practice. Questionnaire-based assessments, for instance, are limited by subjectivity, time-intensive data collection, and an inability to support continuous monitoring (Li et al. 2019b). Wearable sensor-based methods, e.g., electroencephalography (EEG) (Xing et al. 2019), smartwatch (Park et al. 2023), and eye-tracking devices (Li et al. 2019c), were widely adopted to collect neurological and biological data for monitoring mental fatigue status; however, they need to attach sensors to the human body, which might cause irritation and limit usage on construction sites for prolonged durations (Li et al. 2020). For example, EEG systems require various sensing units positioned on the head and forehead to capture electrical activity from various brain regions. CEOs may reject wearing these sensors. Vision-based methods that recognize mental fatigue through facial features (e.g., yawning, blinks) are sensitive to lighting conditions (Ansari et al. 2022a). More importantly, such methods introduce privacy concerns, given that facial data represents personal information (Nnaji and Karakhan 2020). Legal frameworks such as the General Data Protection Regulation (GDPR) emphasize safeguarding data privacy. In summary, an ongoing demand exists for a privacy-preserving, user-friendly, and nonintrusive method to monitor CEO mental fatigue status during operations.
Against this backdrop, this study introduces a novel approach utilizing a smart cushion–based system. The primary objectives are: (1) Proposing a smart cushion system that incorporates optical fiber sensors to collect operators’ heartbeat and respiration vital sign data, (2) developing efficient algorithms to recognize fatigued status from raw data accurately, and (3) conducting a laboratory test to evaluate the performance. We claim the following contributions to the current body of knowledge compared to existing works:
•
This study bridges the knowledge gap of existing studies by introducing a noninvasive, noncontact, privacy-preserving, and easy-to-use smart cushion method for recognizing CEOs’ mental fatigue status. The noninvasive, noncontact smart cushion sensor could be a convenient method for studying, managing, and preventing ergonomic risks resulting from prolonged sitting, which is a grave occupational health and safety problem that plagues various industries.
•
This study utilized the Gaussian mixture model (GMM) to identify multiple levels of mental fatigue, which can provide an adequate understanding of mental fatigue and support proactive interventions. Moreover, the Bidirectional Long-Short-Term Memory (BiLSTM) model was used to recognize fatigued status from the collected time-series data.
•
We conducted a laboratory test and evaluated the performance of the proposed method within the context of an excavation task. Experimental results demonstrate the feasibility of the proposed method, and the BiLSTM model achieved 94.0% accuracy.
The remainder of this study is structured as follows: We commence by providing an overview of related work, encompassing mental fatigue sensing methods and recognition techniques while identifying existing research gaps. Subsequently, we elucidate the methodology employed in this research, followed by a presentation of our findings. We then delve into our study’s theoretical contributions, managerial implications, and inherent limitations. Finally, we present our conclusions and future research directions.

Related Work

The advancements in sensing devices (e.g., visual cameras and biosensors) provide efficient methods to collect mental fatigue data from construction sites. The breakthrough of machine learning (ML), especially deep learning (DL) algorithms, allows construction managers to automatically recognize mental fatigue from a considerable amount of data. This section reviewed relevant works from these two parts: (1) mental fatigue sensing methods, and (2) mental fatigue recognition algorithms.

Mental Fatigue Sensing Methods in Construction

Mental fatigue is a subjective wakefulness state in which humans begin to feel mentally tired, drowsy, and sleepy, as experienced during sustained cognitive demands (Tran et al. 2020). Existing methods for monitoring mental fatigue can be divided into four categories: (1) subjective self-rating methods; (2) fitness-for-duty test methods; (3) wearable sensors-based methods; and (4) vision-based methods. The details of such classifications are shown in Table 1.
Table 1. Summary of prevailing methods for mental fatigue assessment
MethodExamplesCharacteristicsReference
Subjective self-rating methodStanford Sleepiness Scale (SSS)Lack of unified scales; Unprecise, unreliable, and biased resultsTechera et al. (2018) and Wang and Xu (2016)
NASA Task Load Index (NASA-TLX)
Karolinska Sleepiness Scale (KSS)
Fitness-for-duty test methodsPsychomotor vigilance task (PVT)Failed in continuous monitoringDawson et al. (2014), Lin et al. (2018), and Tyagi et al. (2009)
Working memory vigilance task
Wearable sensor-based methodsElectroencephalography (EEG)Relies on specific wearable sensors; IntrusiveLi et al. (2019b) and Xing et al. (2019)
Electrocardiography (ECG)Lee et al. (2022)
Eye movementsLi et al. (2019c, 2020, 2021a)
Vision-based methodsFacial featuresAffected by on-site lights; data privacy concernsGiannakakis et al. (2017) and Mehmood et al. (2022)
Subjective self-rating methods, which rely on individuals’ responses to predefined questions, have been widely employed for assessing the mental states of operators. For instance, Karolinska sleepiness scale (KSS) is a common tool used to gauge driver sleepiness in transportation (Wang and Xu 2016). Similarly, subjective methods have found application in estimating fatigue levels among construction industry workers through self-reporting scales (Techera et al. 2018). Low initial cost and ease of usage are advantages of these approaches. However, there is a lack of commonly acknowledged standards for evaluating fatigue states. Moreover, disparities often exist between how individuals perceive their state and believe they should feel, introducing imprecision and bias into the results obtained through these subjective approaches.
Fitness-for-duty tests seek to identify the onset of fatigue by assessing neuro-behavioral performance in standardized tasks, such as psychomotor vigilance tasks (PVTs). PVTs require participants to engage in screen-based activities and respond to visual stimuli by pressing buttons to measure reaction times and lapses during the experiment (Dawson et al. 2014). Software tools like SkyTesthave have been developed to ensure operators’ fitness. However, fatigue effects may not always manifest as noticeable performance declines (Hu and Lodewijks 2020). Furthermore, these methods cannot continuously monitor fatigue states during operational activities. Additionally, fitness-for-duty tasks, encompassing aspects like memory, attention, hand-eye coordination, and logical reasoning, are impractical for on-site evaluations.
Wearable sensor-based methods, such as smartwatches (Park et al. 2023), EEG (Xing et al. 2019, 2020), and electrocardiography (ECG) (Lee et al. 2022), have drawn increasing attention recently. EEG is usually regarded as the gold standard for assessing mental weariness (Hu and Lodewijks 2020), which uses an array of sensing units placed on the head and forehead to record waveforms with different frequency power bands of the brain, such as alpha, beta, theta, and delta. For example, Xing et al. (2019) used EEG signals to evaluate the emotional and mental state of high-altitude construction workers. The relationship between physical fatigue and mental fatigue was also investigated based on EEG signals. Previous studies show that the spectral power in the alpha and theta bands increases when the interviewee experiences fatigue status (Craig et al. 2012). In addition to EEG methods, eye-tracking-based fatigue detection has been tested by recent studies. For example, Li et al. (2020) adopted the equipment (termed Pupil) to collect participants’ related data, such as fixation duration, count, and dwell time. Eye movement parameters for fatigue detection can be divided into blink, saccade, and fixation metrics. However, fixations and saccades are task-dependent and easily influenced by interface design (Li et al. 2019a), hindering their practical implementations.
Additionally, EEG and eye-tracking methods are invasive, requiring participants to wear specific equipment (Li et al. 2020). EEG necessitates constant contact between multiple sensing units and the head/forehead, which may lead to discomfort and restrict prolonged usage on construction sites. Eye-tracking relies on specialized equipment, such as Pupil Lab’s eye trackers (Li et al. 2019c) or EyeLink II (Hasanzadeh et al. 2017). Operators, especially those working under extreme conditions (e.g., high temperatures and excessive sweating), may resist wearing these sensors. Vision-based methods can be affected by poor lighting conditions often encountered on construction sites (Ansari et al. 2022a). Moreover, vision-based methods encounter data privacy concerns as they involve recording workers’ facial cues (Mehmood et al. 2022). Therefore, developing a convenient, easy-to-use, privacy-preserving, and noninvasive way to detect CEOs’ mental fatigue during the operation stage can be very important.

Mental Fatigue Recognition Algorithms in Construction

Recently, numerous studies have utilized ML algorithms to recognize fatigued states from physiological data. For example, Li et al. (2020) employed a subsequence clustering method to identify multiple levels of operators’ mental fatigue and label relevant time-series data. Umer et al. (2022) tested various ML algorithms to classify heart rate variability (HRV) data. Similarly, Jebelli et al. (2018a) compared the performance of K-nearest neighbors (KNNs), support vector machines (SVM), and Gaussian discriminant analysis (GDA) and found that SVM obtained the highest accuracy of 80.32%. Matuz et al. (2022) further demonstrated SVM’s effectiveness as a classifier for HRV data. Our previous research also compared the performance of SVM, KNN, naive Bayesian (NB), and random forest (RF) classification models in processing heartbeat and respiration signals. Nevertheless, it is worth noting that ML classifiers necessitate manual feature engineering, a process often characterized by its time-consuming nature.
DL algorithms, e.g., convolutional neural networks (CNNs) and recurrent neural networks (RNNs), can learn end-to-end mapping from input to output and avoid manual feature engineering. Moreover, DL models, such as RNNs and Long Short-Term Memory Networks (LSTMs), are well suited to modeling temporal dependencies in time-series data. For example, Kim and Cho (2020) effectively employed the LSTM model to classify 13 distinct motions executed by construction workers, demonstrating an increase in accuracy from 82.39% to 94.73% compared to traditional ML algorithms. Antwi-Afari et al. (2022) compared the classification performance of three DL algorithms in handling the wearable insole sensor data. Yang et al. (2020) used the Bi-LSTM algorithm and achieved 98.6% classification accuracy when assessing the physical load levels of construction workers. In summary, previous studies regarding construction workers’ mental fatigue recognition usually adopt ML solutions, and DL received limited attention.

Points of Departure

The above works indicate that CEOs’ mental fatigue monitoring is essential for on-site safety management. The following two gaps are identified in terms of data collection and processing:
•
There is a demand for a convenient, easy-to-use, privacy-preserving, and noninvasive solution to collect CEOs’ physiological data. Existing studies using different wearable sensors are intrusive, and workers may refuse to wear these sensors in real-world applications, while vision-based methods have privacy concerns and are unreliable due to the challenging lighting conditions. Smart cushions could be an ideal solution to naturally collect data related to CEOs’ mental fatigue. Unfortunately, it has not been previously investigated in the construction industry. Although our previous work has demonstrated the accuracy of the smart cushion systems (Wang et al. 2023), there is still a demand for continuously monitoring CEOs’ mental fatigue since the previous work calibrated the mental fatigue state through subjective questionnaires and it is impractical to conduct periodic questionnaires on operators in practical applications.
•
Although DL has cutting-edge performance and robust skills in creating features in time-series data analysis, limited attempts have been made to explore DL-based mental fatigue recognition and classification. Moreover, previous studies viewed mental fatigue monitoring as a simple binary problem. However, mental fatigue is a gradual accumulation process, and simply defining it as fatigue or nonfatigue could not adequately characterize the fatigue changes during the construction process.
To this end, this study tends to develop a nonintrusive smart cushion system for capturing data and design efficient algorithms to recognize fatigued states from raw data.

Methodology

Overview of the Proposed Framework

This study proposed a smart cushion–based framework for privacy-preserving and nonintrusive mental fatigue monitoring of CEOs, which mainly contains two parts: (1) a smart cushion system design; and (2) DL model development. The proposed framework can monitor fatigue states continuously and naturally. More specifically, as shown in Fig. 1, the proposed cushion system can be conveniently deployed on the seat, making an ordinary chair in different construction equipment smart. While sitting/leaning on the cushion, an operator’s heart and breathing rates will induce subtle changes in the laser, detected by a photodetector. The DL model would automatically analyze the collected data. The smart cushion system and data processing details are presented in the following sections.
Fig. 1. Smart cushion–based mental fatigue monitoring framework.

Smart Cushion–Based Data Collection

Smart Cushion System Design

The periodic stretching and contraction of the diaphragm and chest muscles result in human respiration, while the periodic stretching and contraction of the heart result in a human heartbeat. Subjects’ mental fatigue levels can be reflected in the differences in HRV and respiration rate (RR) (Aryal et al. 2017; Chen et al. 2017; Martin et al. 2018). As shown in Fig. 2, the smart cushion system, which consists of an optical fiber-based interferometer, circuit board, and data acquisition card, was designed to collect HRV and RR during operation processes. The Mach-Zehnder interferometer, composed of two 3db couplers and multisection single-mode fiber, was employed. The circuit board integrates laser source and photodetectors (PDs). Specifically, a 1,550 nm distributed feedback laser source is chosen due to its small size, light weight, low power consumption, and high electrooptical conversion efficiency (Lyu et al. 2019; Tian et al. 2017). PDs detect changes in the interference signal and convert the associated changing optical intensity signal into an electrical signal (Chen et al. 2020). The three-channel signal can be sampled through the National Instruments USB-6001 data acquisition card.
Fig. 2. Smart cushion system: (a) optical fiber-based interferometer for detecting signals; (b) circuit board for converting signals; and (c) data acquisition card for obtaining signals.
The phase changes of light beams in the fiber caused by external factors are used to monitor various parameters (Lyu et al. 2022). When the subject sits on the sensing area of a cushion, the body recoils from heartbeat and respiration, introducing phase difference in the interferometer, and the corresponding intensity variation can be used for signal extraction. Then, the required heartbeat and respiration signals can be obtained based on the MATLAB demodulation program. However, although the optical fiber interferometer offers significant advantages in terms of mobility and sensitivity, signal attenuation produced by its structure will also degrade signal quality, making it unsuitable for long-term stable vital signs measurements (Lyu et al. 2022). Hence, a 3×3 coupler demodulation scheme is used to make the interferometer-based optical fiber sensors perform stably, which can effectively solve the signal fading problem (Kamenev et al. 2018). The signals are received by three PDs with a phase difference of 120°. The output light intensity of the interferometer can be expressed as Eq. (1)
Im=D+I0cos[φ(t)−(m−1)23π]
(1)
where D and I0 = constants, indicating the average value of output light intensity and the peak intensity of interference fringes, respectively. m=1,2,3, indicating the number of the output light path. φ(t) = phase difference signal of the sensor.

Experimental Scenario Design

An excavation task was designed for the data collection based on a simulation system instead of the real world since the experiment aims to test the feasibility of the developed smart cushion system, and fatigue-operating may lead to catastrophic accidents on construction sites, which are unacceptable. Filed experiments will be conducted in the future. The simulated scenario is a typical open construction site under daylight conditions, in which all light conditions and noise changes are generated from the simulation system. As shown in Fig. 3, to simulate the real-world operations, participants were required to sit on the smart cushion to finish the designed task in the simulation system by controlling the joystick.
Fig. 3. Designed experiment scenario with the excavation task.

Experimental Participants

Sixteen students were recruited from the Hong Kong Polytechnic University with a mean age of 26 to validate the proposed method’s feasibility. Students from the construction engineering programs were recruited as experimental participants instead of experienced CEOs. This is because this research aims to introduce the smart cushion solution to bridge the knowledge gap regarding noninvasive and noncontact mental fatigue monitoring and develop corresponding algorithms for fatigue recognition. The on-site evaluation was listed as a future task. Moreover, personal traits (e.g., experience, age) should be consistent for involved participants rather than being another independent variable (Han et al. 2020). Workers’ experiences also affect their hazard detection or safety perception performance. Therefore, students from similar backgrounds and ages would be more appropriate, which has been proved by previous studies in the construction industry (Han et al. 2020; Xing et al. 2020). In addition, this study is experimental research in which the sample size cannot be as large as for survey-style studies (Hasanzadeh et al. 2017). According to previous studies (Li et al. 2019c, 2020; Xing et al. 2019, 2020), the sample is robust given the number of subjects and trials.
All participants have basic knowledge of construction engineering and little work experience operating heavy construction equipment. Moreover, both are in good physical health and are not suffering from any self-reported mental illnesses (e.g., without neural disease history and medicine uptake that might affect brain functions). They must have enough sleep (more than seven hours) and quit alcohol or caffeine beverages 24 hours before the experiment. Written informed consent was obtained before the experiment.

Experimental Procedure

The experiment was conducted in an indoor laboratory at the Hong Kong Polytechnic University. Participants were divided into control and experimental groups to experience these two types of trials (shown in Fig. 4). Compared with the control group, a fatigue-inducing task was added to the experimental group before the simulated excavation task. This setting allows us to effectively observe the influence of the state and degree of mental fatigue on the progress of the experiment. Specifically, the experimental group contains 11 participants, and the control group contains 5 participants. The division ratio refers to Xing et al. (2019). Required data were collected during the excavation simulation stage.
Fig. 4. Timeline of the experiment.
Each trial contains four steps:
Step 1: At first, participants reported their basic information. We informed qualified participants of the objectives, processes, involved equipment, and tasks. Then, all subjects experienced a pretraining before the main experiment to learn how to use joysticks to complete the excavation task. After practicing, each participant was given 10 mins to rest to maintain an alert and wakeful mood before trials. The subjective assessment using stanford sleepiness scale (SSS) and NASA task load index (NASA-TLX) was adopted to assess participants’ status before the trials to ensure they met the criteria.
Step 2: After 10 minutes of rest, the participant completed the subjective assessment through SSS and NASA-TLX. Participants with SSS 1-2 are marked as the alert state, while the SSS of fatigue states should be greater than 2 (Li et al. 2021b). Participants need to keep alert before the excavation simulation in Trial A, while participants in Trial B should be in a fatigued state. Therefore, participants in Trial B were required to complete the hazard identification task (15 mins) before the excavation simulation, aiming to induce the fatigue state. The task design was adopted by Xing et al. (2020) and Li et al. (2019c). Specifically, an image database was developed for the recognition-required hazard identification task, in which 20% of the data set are target images comprising various types of hazards (e.g., falling from a height, tripping, struck by objects). Images were presented on a mobile device one by one at regular intervals (4 s). Participants must concentrate on the image and then try to find potential hazards from the image. The hazard identification task would continue for 15 minutes to induce a mental fatigue state. Different from the image data used by Xing et al. (2020), all hazardous images used in this study were captured from heights like tower cranes and involved human workers, construction equipment, and materials. These images contain various hazards, which could lead to high cognitive workloads. Examples are presented in Fig. 5.
Fig. 5. Examples of hazardous images presented to subjects.
Step 3: Participants in Trial A and Trial B were required to focus on the simulation scenario and complete the excavation task (60 min) in a safe way through joysticks. Such a task duration has been widely used in previous studies (Li et al. 2019c; Xing et al. 2020). It should be noted that while subjects were not needed to finish the job, they were expected to focus on it. The number of excavator operating cycles per minute was used as an efficiency indicator to measure the excavation task performance of the subjects. Specifically, subjects should manually respond to visual stimuli, such as occasional appearance of workers and other equipment. For example, they should judge whether the worker is in a hazardous area (within the radius of rotation of the excavator) by observing the distance between the worker and the excavator reflected in the rearview mirrors. During the excavation simulation stage, the smart cushion device collected the heartbeat and respiration signals for validating the fatigue state.
Step 4: Finally, the subject came to a stop and completed the SSS and NASA-TLX assessment again. Notably, participants do not know the specific duration period of a step to avoid the end-spurt effect reactivation (Morales et al. 2017).

Deep Learning-Based Mental Fatigue Recognition

GMM-Based Data Processing and Clustering

The subjects’ inevitable body movement interfered with the source heartbeat and respiration signals. Hence, Gaussian filtering was used to remove motion-related noise (Harishvijey and Benadict Raja 2022). Considering the frequency difference between the heartbeat and respiration signals, a low-pass filter with a bandwidth of 1 Hz was used to recover the respiration signals. Then, the heartbeat signals were obtained using a low-pass filter with a bandwidth of 30 Hz (Tan et al. 2019). The obtained heartbeat and respiration signals are adequate indicators of mental fatigue during subject excavation simulations (Fairclough et al. 2005; Yao et al. 2008; Argyle et al. 2021).
Considering the time-consuming and labor-intensive manual labeling of time-series data, this study applied the GMM clustering method to identify the mental fatigue level of the subjects and automatically label the time-series data. GMM is an extension of a single Gaussian probability density function, which accurately quantifies the variable distribution with multiple Gaussian probability density functions. GMM is extensively used to sort data into clusters since it can conduct soft clustering on data samples while providing the posterior probability for each cluster (Ameli et al. 2017; Pan et al. 2020). GMM outperforms the traditional k-means clustering method in accuracy (Wang et al. 2019). If all data points X={x1,x2,…,xn} are independently sampled based on the same probability density function, then the GMM can be expressed as
P(xi|Θ)=∑k=1KαkG(xi|μk,Σk)
(2)
where G(xi|μk,Σk) = high-dimensional Gaussian function, which is the k-th component in the mixture model. The expression is as follows:
G(xi|μk,Σk)=1(2π)n|Σ|e−12(xi−μk)T∑k−1(xi−μk)
(3)
where αk = mixture coefficient, indicating the prior probability that data point xi is generated from the i-th Gaussian member
∑k=1Kαk=1,0≤αk≤1,∀  k=1,2,…,K
(4)
Θk={αk,μk,Σk} is the parameter to be estimated in GMM, which can be obtained according to the observed data points. The expectation-maximization (EM) algorithm was used to estimate the GMM parameters. The values of the model parameters were first estimated from known observation data. Values for missing data are then calculated based on the estimated parameter values. Finally, the missing and known observation data were used to reestimate the model parameter values. The model parameters were iterated repeatedly until convergence (Wang and Jiang 2021).
In this study, a total of 28 features were extracted for modeling analysis, including 10 time-domain HRV features, 5 frequency domain HRV features, 6 time-domain RR features, and 7 frequency domain RR features (Wang et al. 2023). Notably, the collected data were normalized to eliminate the influence of individual differences on the results. Data segmentation was conducted using a sliding window approach to obtain labeled segments from continuous heartbeat and respiration data streams (Preece et al. 2009). Considering the HR and RR of a normal adult are 60–120 and 12–20 times per minute, respectively, a window size of 5 s was appropriate for extracting a complete signal. Additionally, a 4.5 s overlap of the continuous window data segment length was employed to prevent the loss of relevant data and enhance processing precision.

BiLSTM-Based Mental Fatigue Recognition

The BiLSTM model was used to recognize mental fatigue from the raw data since it can learn forward and backward on input sequences and extract global characteristics of HRV and RR for better recognition performance (Ansari et al. 2022b). BiLSTM represents each training sequence as two independent recurrent networks connected to the same output layer with shared weights. The model structure is illustrated in Fig. 6. A forward computation is performed from time 1 to time t to record the output of the forward hidden layer at each time, while the backward output can be obtained through reverse input from time t to time 1
ht=f(ω1xt+ω2ht−1)
(5)
ht′=f(ω3xt+ω5ht+1′)
(6)
where ht and ht′ = forward and backward output of the BiLSTM module, respectively. Hence, at time step t, the feature vector generated by the BiLSTM module through the forward and backward extraction can be expressed as
ot=g(ω4ht+ω6ht′)
(7)
Fig. 6. BiLSTM network architecture.
Furthermore, the forget gate (ft), input gate (it), and output gate (ot) collaborate to determine the forwards and backward output of the BiLSTM module. Specifically, ft specifies how much of the prior cell state is retained to the current moment, it specifies how much of the current input xt is saved to the cell state, and ot specifies the contribution of the current cell state to the output of the BiLSTM module. Therefore, the storage, deletion, and updating of HRV and RR information can be realized.
In this study, the BiLSTM model consisted of six layers with an activation function of LeakyReLU. Notably, we introduce BatchNormalization to eliminate gradient explosion. The cross-entropy loss (log loss) function was chosen as the cost function to evaluate the classification model accuracy (Alawneh et al. 2021). A lower loss value indicates a lesser deviation from the actual value. The optimization function is responsible for updating the weights and biases in the network. The Adam optimization function was used for model training to ensure accurate and fast update of network parameters (Kingma and Ba 2014). When the loss function performs well solely on the training data, it can lead to overfitting. To mitigate this issue, we implemented a widely recognized stochastic regularization method known as the dropout technique (Srivastava et al. 2014). This method helps prevent excessive co-adaptation among neurons within the network, thereby enhancing the model’s generalization ability.
Regarding mental fatigue classification model evaluation, metrics such as weighted precision, weighted recall, and weighted accuracy were used to assess the performance of the BiLSTM model. This weighted-average method considers the imbalance between different classes. More specifically, precision quantifies the proportion of positively predicted instances that are accurately classified as true positives. Recall assesses the model’s ability to correctly identify a substantial portion of actual positive instances. Accuracy provides a balanced evaluation of the model’s performance across all classes. These evaluation metrics are expressed as follows (Tiwari et al. 2023):
PrecisionW=∑i=1cωiTPiTPi+FPi
(8)
RecallW=∑i=1cωiTPiTPi+FNi
(9)
AccuracyW=∑i=1cωiTPi+TNiTPi+FPi+TNi+FNi
(10)
where c = total number of mental fatigue levels; ωi = weight of the i-th mental fatigue level; TPi = number of accurately classified samples for class i; TNi = number of accurately classified samples for other classes except for class i; FPi = number of samples misclassified as class i that belong to other classes; and FNi = number of samples misclassified as other classes that belong to class i.

Results

Subjects Excavation Simulation Task Feedback

All participants in the control group and the experimental group completed the excavation simulation experiment, and we found some interesting conclusions by combining the subjects’ task performance and subjective fatigue feedback after excavation, as demonstrated in Fig. 7. Notably, participants’ subjective feedback was normalized to a range of 0–1. Fig. 7 demonstrated the feedback results of 5 subjects in the control group and 11 subjects in the experimental group. Specifically, the mean values of NASA-TLX and SSS subjectively reported by the subjects in the control group after the excavation simulation experiment were 0.348 and 0.392, respectively. For the experimental group, the values were 0.617 and 0.673, respectively. It can be discovered that the mental fatigue level of subjects in the experimental group was much higher than that of subjects in the control group, indicating that the mental fatigue-inducing phase had a significant effect on the experimental group. Furthermore, the average excavation task performance of the subjects in the control group was 0.806, while in the experimental group was 0.714, a decrease of 11.4%.
Fig. 7. Comparison of feedback from subjects in the excavation simulation experiment.

Multilevel Identification of Mental Fatigue

A total of 115,200 samples of data from 16 participants were used for GMM cluster analysis. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) were adopted to determine the optimal number of clusters that best characterize the Gaussian distribution law of features (Henson et al. 2007). The lower AIC and BIC values demonstrated the effectiveness of the corresponding cluster distribution for the mental fatigue state of the subjects. According to Fig. 8, it was evident that five clusters with the lowest AIC and BIC values may adequately discover the distribution laws of the HRV and RR features.
Fig. 8. AIC and BIC score of GMM clusters.
To elucidate participants’ mental fatigue states acquired through GMM clustering, five categories were further defined and described and combined with the performance of the experimental process (Ansari et al. 2022a, b), as demonstrated in Table 2.
Table 2. Mental fatigue levels and corresponding descriptions
StatesLevelDescription
Attention1Focused on the excavation task with minimal distracting movements.
Activation2Slight movements such as itching and rubbing.
Transition3Full alertness and gradual transition to a drowsy state.
Drowsy4Substantial movements such as yawning and head nodding off.
Fatigue5Head shaking with maximum body movements.

Mental Fatigue Classification

Based on the GMM, the HRV and RR data were labeled into five categories, indicating different mental fatigue levels. Then, the BiLSTM model was used to classify the participants’ mental fatigue levels. The data set was divided sequentially according to the ratio of 7∶1∶2, which was used for model training, validation, and testing, respectively. The fivefold crossvalidation method was used to evaluate the classification performance of the BiLSTM model. The results demonstrated that our fine-tuning process yielded optimal accuracy for the data sets by configuring the epoch, dropout, batch size, and learning rate parameters at values of 50, 0.5, 64, and 0.001, respectively.
Fig. 9 illustrates the accuracy and loss iteration curves of the BiLSTM model on the test set. Within each epoch, the model performed forward propagation on the training data, subsequently computing the loss and performing backpropagation. Throughout this process, the parameters of the model were continuously updated to gradually adapt to the characteristics of the training data. It is evident that the BiLSTM model demonstrated heightened accuracy and reduced loss. In other words, the iteration curves converged with increased accuracy and diminished loss values. Notably, both the accuracy and loss value curves converged near epoch 50, indicating that the BiLSTM model was trained effectively without overfitting the HRV and RR data.
Fig. 9. Accuracy and loss of BiLSTM model over iteration curves.
To investigate the potential of measuring participants’ mental fatigue levels using extracted HRV and RR features, a confusion matrix was used to visualize the performance of the BiLSTM model, which effectively identifies the confusion between different mental fatigue levels. This study applied a normalized confusion matrix to intuitively explain the misclassified mental fatigue level, as shown in Fig. 10. The rows and columns of the matrix represent the classified mental fatigue levels by the BiLSTM model and the actual mental fatigue levels based on GMM clustering, respectively. The values of the diagonal elements indicate the proportion of correctly classified levels. As shown in Fig. 10, the BiLSTM model effectively classified samples from five levels of mental fatigue with the correct rate exceeding 80%. It obtained an even higher correct classification rate for mental fatigue levels I (96.9%) and V (99.9%). Notably, the BiLSTM model only achieved a correct classification rate of 81.9% for level III. This is because there is a certain degree of ambiguity and uncertainty in the HRV and RR feature manifestations during the translation from the activation state to the fatigued state. Such uncertainty will affect the classification accuracy of the BiLSTM model.
Fig. 10. Accuracy of the BiLSTM model on different fatigue labels.
To further demonstrate the effectiveness of the BiLSTM model, the LSTM was introduced for comparison using the same data set for training. Notably, the LSTM model also consisted of six layers with an activation function of LeakyReLU, using fivefold crossvalidation to measure classification accuracy. Table 3 shows the classification performance of the two RNN-based DL models. As presented in Table 3, BiLSTM and LSTM achieved accuracies of 94.0% and 91.9%, respectively. The findings underscored the superior performance of the BiLSTM model in utilizing HRV and RR data for the classification of mental fatigue levels among CEOs. Furthermore, the BiLSTM model achieved precision and recall scores both exceeding 90%, specifically 93.4% and 93.3%, respectively. In contrast, the LSTM model exhibited relatively lower performance in terms of precision and recall, scoring 85.2% and 88.4%, respectively. This demonstrates that the BiLSTM model is more capable of effectively capturing samples of varying levels of mental fatigue. On the other hand, evaluating the two DL models from the perspective of training time, the durations for the BiLSTM and LSTM networks were almost 30 min and 17 min, respectively. The results indicate that the BiLSTM model requires more training time compared to the LSTM model, owing to its higher complexity.
Table 3. Classification performance for BiLSTM and LSTM models
ModelsPrecision (%)Recall (%)Accuracy (%)Training time
BiLSTM93.493.394.029 min and 47 s
LSTM85.288.491.916 min and 36 s
In summary, the BiLSTM model, which accomplishes bidirectional capturing of HRV and RR information and possesses enhanced memory capabilities, performs well in the classification of mental fatigue. The findings of this study offer an effective, noninvasive method for identifying and classifying the mental fatigue states among CEOs during construction tasks.

Discussion

Construction equipment operation is a cognitively demanding task in which operators must comprehend constantly evolving surroundings and make proper operations. The prolonged working time and intensive cognitive workloads make operators mentally fatigued in practice. Mental fatigue can cause human error-related accidents, lower productivity, and psychological illnesses. Thus, it is imperative to continuously monitor operators’ mental fatigue conveniently and noninvasively. This study proposed a noninvasive and noncontact optical fiber sensor-based smart cushion method for CEOs’ mental fatigue monitoring. Experimental results demonstrate that the developed smart cushion consisting of optical fiber sensors can reliably collect the heartbeat and respiration data, and the proposed BiLSTM model can accurately identify fatigued status from the raw data.

Theoretical Contributions and Management Implications

This study contributes to the current body of knowledge regarding the nonintrusive and noncontact monitoring of operators’ mental fatigue. Many recent studies utilized wearable devices (e.g., smartwatches, EEG, or eye-tracking devices) to detect mental fatigue (Li et al. 2019c, 2020; Lee et al. 2020; Park et al. 2023); however, these methods are either too interruptive or too intrusive since they require operators to wear special devices, such as EEG sensors. Hence, their applications are highly limited in construction scenarios. Although vision-based facial feature analysis is nonintrusive, it is unreliable due to the challenging lighting conditions (Mehmood et al. 2022). Construction at night or in cloudy weather without sufficient lighting is normal in this industry. For example, concrete pouring work is usually conducted at night. Privacy concerns should also be considered in facial feature methods. How to monitor operators’ mental status continuously and accurately without causing discomfort and aversion in dynamic construction sites should be further investigated. The proposed method bridges the above knowledge gap by proposing a smart cushion system that can be easily used in construction scenarios. Moreover, previous studies usually view mental fatigue monitoring as a simple binary problem (Jebelli et al. 2018b; Umer et al. 2022; Xing et al. 2019). Instead, fatigue is a gradual accumulation process, and merely classifying it as fatigue or nonfatigue does not accurately characterize the fatigue changes that occur over the construction process. This study first utilized the GMM model to identify different levels of mental fatigue and then used the BiLSTM model to recognize fatigued status. The proposed methods can detect multiple fatigue levels and give researchers and practitioners an adequate understanding of mental fatigue.
The proposed method also has managerial implications. First, the proposed method is expected to improve on-site safety performance and facilitate the fatigue management of CEOs. The developed smart cushion system has clear advantages. It no longer requires sensors attached to the skin of operators (required for EEG, eye-tracking, and wristbands), it is not sensitive to dynamic lighting conditions, and does not generate privacy concerns when compared to vision-based methods since vision-based methods need to collect human facial features (eye aspect ratio, eye distance, and mouth aspect ratio) using RGB cameras, and it does not necessitate charging and replacing batteries for sensors. Smart cushions can be placed on the operator’s seat or seatback and do not need additional power. Hence, this equipment can be widely used in construction practices. The system can provide construction managers with a quantitative and reliable assessment tool to measure workers’ mental workloads and then support the interventions for mental fatigue (e.g., worker shifts and breaks). If wirelessly connected to a smartphone, the smart cushion can conveniently provide early warnings to operators and/or their managers to alert the operators at stake to take breaks/rests to avoid mental fatigue-related ill consequences. Moreover, the proposed method could be used to study and prevent the risks resulting from prolonged sitting, which is a grave occupational health and safety problem plaguing many industries.

Limitations and Future Works

This study has the following limitations. First, the experiment was conducted in a laboratory environment. Although we have implemented measures from the designed environment, participants, and monitoring methods to ensure the ecological validity of this research, there may be some differences between the simulated environment and the real-world environment (e.g., weather conditions). For example, unlike the laboratory environment, construction sites have high levels of noise that negatively influence human mental fatigue status. Similarly, the simulated task could not replicate construction equipment operations, and noise data caused by body movements would rise during real-world operations. Moreover, due to the limited funding, the experimental study was conducted on only 16 students rather than skilled operators. The mental stress levels of students without working experience may differ from those of real equipment operators. However, notwithstanding the limitations, as the main aim of our research is to develop a new nonintrusive and noncontact method for monitoring operators’ mental fatigue, the experiment can demonstrate the feasibility of the proposed method. We also acknowledge that the proposed smart cushion system should be evaluated and optimized in different operation scenarios on construction sites in the future. Second, only heartbeat and respiration signals were collected to measure participants’ mental fatigue states. Other wearable devices (e.g., EEG) can be employed to collect additional real-time neurophysiological signals. Integrating multimodal data can provide a more comprehensive and reliable reflection of the relationship between operator physiological characteristics and mental fatigue. We will further explore different fatigue monitoring methods and attempt to integrate data from multiple sources to achieve more accurate measurement results. Third, the scope of this research was limited to monitoring mental fatigue. Active interventions for operators’ mental fatigue should be investigated in the future to support the mental fatigue management of CEOs.

Conclusion

CEOs are prone to experience mental fatigue due to the highly cognitive demanding tasks and prolonged working time. This study developed and tested a noninvasive, noncontact smart cushion technology for the mental fatigue monitoring of CEOs. Heart and breathing rates were collected by the smart cushion system consisting of embedded optical fiber, laser technology, and associated electronics. GMM was adopted to determine multiple levels of mental fatigue, while the BiLSTM model was utilized to recognize fatigue from the raw data. An experiment with 16 participants was conducted using a simulated excavation task. Experimental results show that the BiLSTM can achieve high classification accuracy (94.0%). Compared to existing works, this study contributes to the body of knowledge regarding CEOs’ mental fatigue monitoring by providing an easy-to-use, privacy-preserving, nonintrusive, and noncontact solution. Due to such advantages, the proposed technology can provide construction managers with a tool to better understand and manage CEOs’ mental fatigue, avoid fatigue-related ill consequences, and facilitate on-site safety management. However, future research is required to test the performance of the proposed method in real-world environments and engage enough skilled operators. Moreover, future works could develop a convenient-to-use and easy-to-implement method to help operators relax from fatigued status. Olfactory cues could be an ideal solution for future studies.

Data Availability Statement

Raw data (e.g., heartbeat and respiration signals) and codes that support the findings of this study are available from the corresponding author upon reasonable request.

Acknowledgments

The authors acknowledge the following funding grants: General Research Fund (GRF) Grant 15201621 titled “Monitoring and managing fatigue of construction plant and equipment operators exposed to prolonged sitting”; GRF Grant 15210720 titled “The development and validation of a noninvasive tool to monitor mental and physical stress in construction workers”; and GRF Grant 15210923 titled “Noninvasive noncontact mental workload and stress monitoring of construction equipment operators.”

References

Alawneh, L., T. Alsarhan, M. Al-Zinati, M. Al-Ayyoub, Y. Jararweh, and H. Lu. 2021. “Enhancing human activity recognition using deep learning and time series augmented data.” J. Ambient Intell. Hum. Comput. 12 (12): 10565–10580. https://doi.org/10.1007/s12652-020-02865-4.
Google Scholar
Ameli, S., F. Naghdy, D. Stirling, G. Naghdy, and M. Aghmesheh. 2017. “Objective clinical gait analysis using inertial sensors and six minute walking test.” Pattern Recognit. 63 (Apr): 246–257. https://doi.org/10.1016/j.patcog.2016.08.002.
Google Scholar
Ansari, S., H. Du, F. Naghdy, and D. Stirling. 2022a. “Automatic driver cognitive fatigue detection based on upper body posture variations.” Expert Syst. Appl. 203 (Jun): 117568. https://doi.org/10.1016/j.eswa.2022.117568.
Google Scholar
Ansari, S., F. Naghdy, H. Du, and Y. N. Pahnwar. 2022b. “Driver mental fatigue detection based on head posture using new modified reLU-BiLSTM deep neural network.” IEEE Trans Intell. Transp. Syst. 23 (8): 10957–10969. https://doi.org/10.1109/TITS.2021.3098309.
Google Scholar
Antwi-Afari, M. F., Y. Qarout, R. Herzallah, S. Anwer, W. Umer, Y. Zhang, and P. Manu. 2022. “Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data.” Autom. Constr. 136 (Sep): 104181. https://doi.org/10.1016/j.autcon.2022.104181.
Google Scholar
Argyle, E., A. Marinescu, M. Wilson, G. Lawson, and S. Sharples. 2021. “Physiological indicators of task demand, fatigue, and cognition in future digital manufacturing environments.” Int. J. Hum. Comput. Interact. 145 (Jan): 102522. https://doi.org/10.1016/j.ijhcs.2020.102522.
Google Scholar
Aryal, A., A. Ghahramani, and B. Becerik-Gerber. 2017. “Monitoring fatigue in construction workers using physiological measurements.” Autom. Constr. 82 (Sep): 154–165. https://doi.org/10.1016/j.autcon.2017.03.003.
Google Scholar
BLS (Bureau of Labor Statistics). 2022. “Census of fatal occupational injuries summary, 2022.” Accessed July 20, 2023. https://www.bls.gov/news.release/cfoi.nr0.htm.
Google Scholar
Chen, L. L., Y. Zhao, P. F. Ye, J. Zhang, and J. Z. Zou. 2017. “Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers.” Expert Syst. Appl. 85 (Sep): 279–291. https://doi.org/10.1016/j.eswa.2017.01.040.
Google Scholar
Chen, S. Y., F. Z. Tan, W. M. Lyu, and C. Y. Yu. 2020. “Ballistocardiography monitoring system based on optical fiber interferometer aided with heartbeat segmentation algorithm.” Biomed. Opt. Express 11 (10): 5458–5469. https://doi.org/10.1364/BOE.403086.
Google Scholar
Craig, A., Y. Tran, N. Wijesuriya, and H. Nguyen. 2012. “Regional brain wave activity changes associated with fatigue.” Psychophysiology 49 (4): 574–582. https://doi.org/10.1111/j.1469-8986.2011.01329.x.
Google Scholar
Dawson, D., A. K. Searle, and J. L. Paterson. 2014. “Look before you (s)leep: Evaluating the use of fatigue detection technologies within a fatigue risk management system for the road transport industry.” Sleep Med. Rev. 18 (2): 141–152. https://doi.org/10.1016/j.smrv.2013.03.003.
Google Scholar
Fairclough, S. H., L. Venables, and A. Tattersall. 2005. “The influence of task demand and learning on the psychophysiological response.” Int. J. Psychophysiol. 56 (2): 171–184. https://doi.org/10.1016/j.ijpsycho.2004.11.003.
Google Scholar
Fang, Y., and Y. K. Cho. 2017. “Effectiveness analysis from a cognitive perspective for a real-time safety assistance system for mobile crane lifting operations.” J. Constr. Eng. Manage. 143 (4): 05016025. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001258.
Google Scholar
Giannakakis, G., M. Pediaditis, D. Manousos, E. Kazantzaki, F. Chiarugi, P. G. Simos, K. Marias, and M. Tsiknakis. 2017. “Stress and anxiety detection using facial cues from videos.” Biomed Signal Process Control 31 (Jun): 89–101. https://doi.org/10.1016/j.bspc.2016.06.020.
Google Scholar
Han, Y., Z. Yin, J. Zhang, R. Jin, and T. Yang. 2020. “Eye-tracking experimental study investigating the influence factors of construction safety hazard recognition.” J. Constr. Eng. Manage. 146 (8): 04020091. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001884.
Google Scholar
Harishvijey, A., and J. Benadict Raja. 2022. “Automated technique for EEG signal processing to detect seizure with optimized variable Gaussian filter and fuzzy RBFELM classifier.” Biomed. Signal Process Control 74 (Sep): 103450. https://doi.org/10.1016/j.bspc.2021.103450.
Google Scholar
Hasanzadeh, S., B. Esmaeili, and M. D. Dodd. 2017. “Measuring the impacts of safety knowledge on construction workers’ attentional allocation and hazard detection using remote eye-tracking technology.” J. Manage. Eng. 33 (5): 04017024. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000526.
Google Scholar
Henson, J. M., S. P. Reise, and K. H. Kim. 2007. “Detecting mixtures from structural model differences using latent variable mixture modeling: A comparison of relative model fit statistics.” Struct. Equation Model. 14 (2): 202–226. https://doi.org/10.1080/10705510709336744.
Google Scholar
HSE (Health and Safety Executive). 2022. “Construction statistics in Great Britain, 2022/23.” Accessed December 15, 2023. https://www.hse.gov.uk/statistics/fatals.htm.
Google Scholar
Hu, X., and G. Lodewijks. 2020. “Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue.” J. Saf. Res. 72 (Apr): 173–187. https://doi.org/10.1016/j.jsr.2019.12.015.
Google Scholar
Jebelli, H., S. Hwang, and S. Lee. 2018a. “EEG-based workers’ stress recognition at construction sites.” Autom. Constr. 93 (Jun): 315–324. https://doi.org/10.1016/j.autcon.2018.05.027.
Google Scholar
Jebelli, H., S. Hwang, and S. Lee. 2018b. “EEG signal-processing framework to obtain high-quality brain waves from an off-the-shelf wearable EEG device.” J. Comput. Civ. Eng. 32 (1): 04017070. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000719.
Google Scholar
Kamenev, O. T., Y. S. Petrov, and A. A. Podlesnykh. 2018. “Use of 3×3 Coupler in the Fiber Optic Strainmeter Based on Mach-Zehnder Interferometer.” PKnE Energy 3 (3): 388–392. https://doi.org/10.18502/ken.v3i3.2052.
Google Scholar
Kim, K., and Y. K. Cho. 2020. “Effective inertial sensor quantity and locations on a body for deep learning-based worker’s motion recognition.” Autom. Constr. 113 (May): 103126. https://doi.org/10.1016/j.autcon.2020.103126.
Google Scholar
Kingma, D. P., and J. Ba. 2014. “Adam: A method for stochastic optimization.” Preprint, submitted July 25, 2019. http://arxiv.org/abs/1412.6980.
Google Scholar
Lee, G., B. Choi, C. R. Ahn, and S. Lee. 2020. “Wearable biosensor and hotspot analysis–based framework to detect stress hotspots for advancing elderly’s mobility.” J. Manage. Eng. 36 (3): 04020010. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000753.
Google Scholar
Lee, W., K.-Y. Lin, P. W. Johnson, and E. Y. W. Seto. 2022. “Selection of wearable sensor measurements for monitoring and managing entry-level construction worker fatigue: A logistic regression approach.” Eng. Constr. Archit. Manage. 29 (8): 2905–2923. https://doi.org/10.1108/ECAM-02-2021-0106.
Google Scholar
Li, F., C.-H. Chen, G. Xu, L. P. Khoo, and Y. Liu. 2019a. “Proactive mental fatigue detection of traffic control operators using bagged trees and gaze-bin analysis.” Adv. Eng. Inform 42 (May): 100987. https://doi.org/10.1016/j.aei.2019.100987.
Google Scholar
Li, H., D. Wang, J. Chen, X. Luo, J. Li, and X. Xing. 2019b. “Pre-service fatigue screening for construction workers through wearable EEG-based signal spectral analysis.” Autom. Constr. 106 (Mar): 102851. https://doi.org/10.1016/j.autcon.2019.102851.
Google Scholar
Li, J., H. Li, W. Umer, H. Wang, X. Xing, S. Zhao, and J. Hou. 2020. “Identification and classification of construction equipment operators’ mental fatigue using wearable eye-tracking technology.” Autom. Constr. 109 (Jun): 103000. https://doi.org/10.1016/j.autcon.2019.103000.
Google Scholar
Li, J., H. Li, F. Wang, A. S. Cheng, X. Yang, and H. Wang. 2021a. “Proactive analysis of construction equipment operators’ hazard perception error based on cognitive modeling and a dynamic Bayesian network.” Reliab. Eng. Syst. Saf. 205 (Aug): 107203. https://doi.org/10.1016/j.ress.2020.107203.
Google Scholar
Li, J., H. Li, H. Wang, W. Umer, H. Fu, and X. Xing. 2019c. “Evaluating the impact of mental fatigue on construction equipment operators’ ability to detect hazards using wearable eye-tracking technology.” Autom. Constr. 105 (Sep): 102835. https://doi.org/10.1016/j.autcon.2019.102835.
Google Scholar
Li, R., Y. V. Chen, and L. Zhang. 2021b. “A method for fatigue detection based on Driver’s steering wheel grip.” Int. J. Ind. Ergon. 82 (Feb): 103083. https://doi.org/10.1016/j.ergon.2021.103083.
Google Scholar
Lin, C. T., M. Nascimben, J. T. King, and Y. K. Wang. 2018. “Task-related EEG and HRV entropy factors under different real-world fatigue scenarios.” Neurocomputing 311 (Sep): 24–31. https://doi.org/10.1016/j.neucom.2018.05.043.
Google Scholar
Lyu, W. M., S. Y. Chen, F. Z. Tan, and C. Y. Yu. 2022. “Vital signs monitoring based on interferometric fiber optic sensors.” Photonics 9 (2): 50. https://doi.org/10.3390/photonics9020050.
Google Scholar
Lyu, W. M., F. Z. Tan, S. Y. Chen, and C. Y. Yu. 2019. “Myocardial contractility assessment using fiber optic sensors.” In Proc., Asia Communications and Photonics Conference (ACP), 1–3. New York: IEEE.
Google Scholar
Martin, K., R. Meeusen, K. G. Thompson, R. Keegan, and B. Rattray. 2018. “Mental fatigue impairs endurance performance: A physiological explanation.” Sports Med. 48 (9): 2041–2051. https://doi.org/10.1007/s40279-018-0946-9.
Google Scholar
Matuz, A., D. van der Linden, G. Darnai, and A. Csatho. 2022. “Generalisable machine learning models trained on heart rate variability data to predict mental fatigue.” Sci. Rep. 12 (1): 200023. https://doi.org/10.1038/s41598-022-24415-y.
Google Scholar
Mehmood, I., H. Li, W. Umer, A. Arsalan, M. Saad Shakeel, and S. Anwer. 2022. “Validity of facial features’ geometric measurements for real-time assessment of mental fatigue in construction equipment operators.” Adv. Eng. Inf. 54 (Sep): 101777. https://doi.org/10.1016/j.aei.2022.101777.
Google Scholar
Morales, J. M., C. Díaz-Piedra, H. Rieiro, J. Roca-González, S. Romero, A. Catena, L. J. Fuentes, and L. L. Di Stasi. 2017. “Monitoring driver fatigue using a single-channel electroencephalographic device: A validation study by gaze-based, driving performance, and subjective data.” Accid. Anal. Prev. 109 (Mar): 62–69. https://doi.org/10.1016/j.aap.2017.09.025.
Google Scholar
Nnaji, C., and A. A. Karakhan. 2020. “Technologies for safety and health management in construction: Current use, implementation benefits and limitations, and adoption barriers.” J. Build. Eng. 29 (Apr): 101212. https://doi.org/10.1016/j.jobe.2020.101212.
Google Scholar
Pan, Y., L. M. Zhang, and M. J. Skibniewski. 2020. “Clustering of designers based on building information modeling event logs.” Comput. Aided Civ. Inf. 35 (7): 701–718. https://doi.org/10.1111/mice.12551.
Google Scholar
Park, S., S. Seong, Y. Ahn, and H. Kim. 2023. “Real-time fatigue evaluation using ecological momentary assessment and smartwatch data: An observational field study on construction workers.” J. Manage. Eng. 39 (3): 04023008. https://doi.org/10.1061/JMENEA.MEENG-4953.
Google Scholar
Preece, S. J., J. Y. Goulermas, L. P. J. Kenney, D. Howard, K. Meijer, and R. Crompton. 2009. “Activity identification using body-mounted sensors-a review of classification techniques.” Physiol. Meas. 30 (4): R1–R33. https://doi.org/10.1088/0967-3334/30/4/R01.
Google Scholar
Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. “Dropout: A simple way to prevent neural networks from overfitting.” J. Mach. Learn. Res 15 (1): 1929–1958.
Google Scholar
Tam, V. W., and I. W. Fung. 2011. “Tower crane safety in the construction industry: A Hong Kong study.” Saf. Sci. 49 (2): 208–215. https://doi.org/10.1016/j.ssci.2010.08.001.
Google Scholar
Tan, F. Z., S. Y. Chen, W. M. Lyu, Z. Y. Liu, C. Y. Yu, C. Lu, and W. Y. Tam. 2019. “Non-invasive human vital signs monitoring based on twin-core optical fiber sensors.” Biomed. Opt. Express 10 (11): 5940–5951. https://doi.org/10.1364/BOE.10.005940.
Google Scholar
Techera, U., M. Hallowell, R. Littlejohn, and S. Rajendran. 2018. “Measuring and predicting fatigue in construction: Empirical field study.” J. Constr. Eng. Manage. 144 (8): 04018062. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001513.
Google Scholar
Tian, Y., J. T. Wei, M. S. Jiang, L. Zhang, F. Y. Zhang, Q. M. Sui, and W. Sun. 2017. “Drive design and performance test of a tunable DFB laser.” In Proc., Chinese Automation Congress (CAC), 4024–4027. New York: IEEE. https://doi.org/10.1109/CAC.2017.8243484.
Google Scholar
Tiwari, D., B. Nagpal, B. S. Bhati, A. Mishra, and M. Kumar. 2023. “A systematic review of social network sentiment analysis with comparative study of ensemble-based techniques.” Artif. Intell. Rev. 56 (11): 13407–13461. https://doi.org/10.1007/s10462-023-10472-w.
Google Scholar
Tran, Y., A. Craig, R. Craig, R. Chai, and H. Nguyen. 2020. “The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses.” Psychophysiology 57 (5): e13554. https://doi.org/10.1111/psyp.13554.
Google Scholar
Tyagi, R., K. Q. Shen, S. Y. Shao, and X. P. Li. 2009. “A novel auditory working-memory vigilance task for mental fatigue assessment.” Saf. Sci. 47 (7): 967–972. https://doi.org/10.1016/j.ssci.2008.10.018.
Google Scholar
Umer, W., Y. Yu, M. F. Antwi-Afari, L. Jue, M. K. Siddiqui, and H. Li. 2022. “Heart rate variability based physical exertion monitoring for manual material handling tasks.” Int. J. Ind. Ergon. 89 (Jun): 103301. https://doi.org/10.1016/j.ergon.2022.103301.
Google Scholar
Wang, J. H., and J. M. Jiang. 2021. “Unsupervised deep clustering via adaptive GMM modeling and optimization.” Neurocomputing 433 (Sep): 199–211. https://doi.org/10.1016/j.neucom.2020.12.082.
Google Scholar
Wang, L., H. Li, Y. Yao, D. Han, C. Yu, W. Lyu, and H. Wu. 2023. “Smart cushion-based non-invasive mental fatigue assessment of construction equipment operators: A feasible study.” Adv. Eng. Inform 58 (Apr): 102134. https://doi.org/10.1016/j.aei.2023.102134.
Google Scholar
Wang, X., and C. Xu. 2016. “Driver drowsiness detection based on non-intrusive metrics considering individual specifics.” Accid. Anal. Prev. 95 (Jun): 350–357. https://doi.org/10.1016/j.aap.2015.09.002.
Google Scholar
Wang, Z. Q., C. Da Cunha, M. Ritou, and B. Furet. 2019. “Comparison of K-means and GMM methods for contextual clustering in HSM.” Procedia Manuf. 28 (Mar): 154–159. https://doi.org/10.1016/j.promfg.2018.12.025.
Google Scholar
Xing, X., H. Li, J. Li, B. Zhong, H. Luo, and M. Skitmore. 2019. “A multicomponent and neurophysiological intervention for the emotional and mental states of high-altitude construction workers.” Autom. Constr. 105 (Sep): 102836. https://doi.org/10.1016/j.autcon.2019.102836.
Google Scholar
Xing, X., B. Zhong, H. Luo, T. Rose, J. Li, and M. F. Antwi-Afari. 2020. “Effects of physical fatigue on the induction of mental fatigue of construction workers: A pilot study based on a neurophysiological approach.” Autom. Constr. 120 (Jun): 103381. https://doi.org/10.1016/j.autcon.2020.103381.
Google Scholar
Yang, K., C. R. Ahn, and H. Kim. 2020. “Deep learning-based classification of work-related physical load levels in construction.” Adv. Eng. Inf. 45 (Aug): 101104. https://doi.org/10.1016/j.aei.2020.101104.
Google Scholar
Yao, Y., Y. Chang, X. Xie, X. Cao, X. Sun, and Y. Wu. 2008. “Heart rate and respiration responses to real traffic pattern flight.” Appl. Psychophysiol. Biofeedback 33 (4): 203–209, https://doi.org/10.1007/s10484-008-9066-x.
Google Scholar

Information & Authors

Information

Published In

Go to Journal of Management in Engineering
Journal of Management in Engineering
Volume 40 • Issue 5 • September 2024

Copyright

© 2024 American Society of Civil Engineers.

History

Received: Sep 11, 2023
Accepted: Apr 12, 2024
Published online: Jul 10, 2024
Published in print: Sep 1, 2024
Discussion open until: Dec 10, 2024

Permissions

Request permissions for this article.

ASCE Technical Topics:

  1. Business management
  2. Construction engineering
  3. Construction equipment
  4. Construction management
  5. Construction methods
  6. Construction sites
  7. Data collection
  8. Engineering fundamentals
  9. Engineering mechanics
  10. Equipment and machinery
  11. Fatigue (material)
  12. Human and behavioral factors
  13. Material mechanics
  14. Material properties
  15. Materials engineering
  16. Methodology (by type)
  17. Occupational safety
  18. Practice and Profession
  19. Public administration
  20. Public health and safety
  21. Research methods (by type)
  22. Safety

Authors

Affiliations

Lei Wang [email protected]
Ph.D. Student, Dept. of Building and Real Estate, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. Email: [email protected]
View all articles by this author
Heng Li https://orcid.org/0000-0002-3187-9041 [email protected]
Chair Professor, Dept. of Building and Real Estate, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. ORCID: https://orcid.org/0000-0002-3187-9041. Email: [email protected]
View all articles by this author
Haitao Wu https://orcid.org/0000-0002-4804-3806 [email protected]
Associate Professor, School of Finance, Zhongnan Univ. of Economics and Law, Wuhan 430073, China (corresponding author). ORCID: https://orcid.org/0000-0002-4804-3806. Email: [email protected]
View all articles by this author
Yizhi Yao [email protected]
Ph.D. Student, Dept. of Building and Real Estate, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. Email: [email protected]
View all articles by this author
Changyuan Yu [email protected]
Professor, Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. Email: [email protected]
View all articles by this author
Waleed Umer https://orcid.org/0000-0003-2419-4172 [email protected]
Assistant Professor, Dept. of Architecture and Built Environment, Northumbria Univ., Newcastle upon Tyne NE18ST, UK. ORCID: https://orcid.org/0000-0003-2419-4172. Email: [email protected]
View all articles by this author
Dongliang Han [email protected]
Ph.D. Student, Dept. of Building and Real Estate, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. Email: [email protected]
View all articles by this author
Jie Ma https://orcid.org/0000-0002-0827-3842 [email protected]
Ph.D. Candidate, Dept. of Building and Real Estate, The Hong Kong Polytechnic Univ., Hung Hom, Kowloon, Hong Kong. ORCID: https://orcid.org/0000-0002-0827-3842. Email: [email protected]
View all articles by this author

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share