Acoustic Emotion Recognition for Improved Safety in the Construction Industry
Publication: Construction Research Congress 2024
ABSTRACT
In the construction industry, clear communication is vital for optimal teamwork and performance. Yet, diverse languages, accents, and continuous worksite noises often impede speech clarity, leading to misunderstandings and consequential errors. Addressing this challenge, the presented research introduces a speech emotion recognition algorithm tailored for construction. Unlike prior algorithms emphasizing language content, this approach centers on speech features, bridging a significant gap in construction-specific applications. The algorithm aims to identify four key emotions: anger, happiness, sadness, and neutrality, using six distinct acoustic features: pitch, intensity, frequency formants, jitter, shimmer, and zero crossing rate. Leveraging these features collectively enhances system accuracy. Designed in MATLAB, the decision-tree-based method calculates confidentiality intervals for each feature and is specifically developed for speaker-dependent emotion recognition. Remarkably, the algorithm secured an 86% accuracy rate in detecting anger, outperforming existing models. By recognizing and addressing the emotional states of workers, this method holds the promise to greatly enhance safety on construction sites, averting potential hazards.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Arnold, M. B. 1960. Emotion and personality.
Averill, J. R. 1975. A semantic atlas of emotional concepts (American Psycholog. Ass., Journal Suppl. Abstract Service).
Bower, G. H. 1981. 'Mood and memory', American psychologist, 36: 129.
Cavazza, M., F. Charles, and S. J. Mead. 2002. 'Character-based interactive storytelling', IEEE Intelligent systems, 17: 17–24.
Cowie, R., and R. R. Cornelius. 2003. 'Describing the emotional states that are expressed in speech', Speech communication, 40: 5–32.
Cowie, R., E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor. 2001. 'Emotion recognition in human-computer interaction', IEEE Signal Processing Magazine, 18: 32–80.
Damasio, A. R. 2000. 'A second chance for emotion', Cognitive neuroscience of emotion, 2: 12–23.
Fónagy, I. 1981. 'Emotions, voice and music', Research aspects on singing, 33: 51–79.
Gobl, C., and A. N. Chasaide. 2003. 'The role of voice quality in communicating emotion, mood and attitude', Speech communication, 40: 189–212.
Huang, R., and C. Ma. 2006. “Toward a speaker-independent real-time affect detection system.” In 18th International Conference on Pattern Recognition (ICPR'06), 1204–07. IEEE.
James, L. 2009. Road rage and aggressive driving: Steering clear of highway warfare (Prometheus Books).
Kessous, L., G. Castellano, and G. Caridakis. 2010. 'Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis', Journal on Multimodal User Interfaces, 3: 33–48.
Knauer, H. S., S. Pedersen, C. N. Reherman, J. L. Rochat, E. S. Thalheimer, M. C. Lau, G. G. Fleming, M. Ferroni, and C. Corbisier. 2006. FHWA highway construction noise handbook. In.: United States. Federal Highway Administration.
Kramer, E. 1963. 'Judgment of personal characteristics and emotions from nonverbal properties of speech', Psychological Bulletin, 60: 408.
Lee, C. M., and S. S. Narayanan. 2005. 'Toward detecting emotions in spoken dialogs', IEEE transactions on speech and audio processing, 13: 293–303.
Liberman, M. 2002. 'Emotional prosody speech and transcripts', http://www. ldc. upenn. edu/Catalog/CatalogEntry. jsp? catalogId= LDC2002S28.
Lo, V. E.-W., and P. A. Green. 2013. 'Development and evaluation of automotive speech interfaces: useful information from the human factors and the related literature', International Journal of Vehicular Technology, 2013.
Mansoor, A., S. Liu, G. M. Ali, A. Bouferguene, M. Al-Hussein, and I. Hassan. 2022. 'Keyword identification framework for speech communication on construction sites', Modular and Offsite Construction (MOC) Summit Proceedings: 106–13.
Mowrer, O. 1960. 'Learning theory and behavior'.
Murray, I. R., and J. L. Arnott. 1993. 'Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion', The Journal of the Acoustical Society of America, 93: 1097–108.
Pollack, I., H. Rubenstein, and A. Horowitz. 1960. 'Communication of verbal modes of expression', Language and Speech, 3: 121–30.
Scherer, K. R. 1984. 'On the nature and function of emotion: A component process approach', Approaches to emotion, 2293: 31.
Scherer, K. R. 2003. 'Vocal communication of emotion: A review of research paradigms', Speech communication, 40: 227–56.
Scherer, K. R., R. Banse, and H. G. Wallbott. 2001. 'Emotion inferences from vocal expression correlate across languages and cultures', Journal of Cross-cultural psychology, 32: 76–92.
Sethu, V., J. Epps, and E. Ambikairajah. 2014. 'Speech based emotion recognition.' in, Speech and Audio Processing for Coding, Enhancement and Recognition (Springer).
Tarnowski, P., M. Kołodziej, A. Majkowski, and R. J. Rak. 2017. 'Emotion recognition using facial expressions', Procedia Computer Science, 108: 1175–84.
Tomkins, S. 1962. Affect imagery consciousness: Volume I: The positive affects (Springer publishing company).
Udhan, T., and S. Bernadin. 2018. 'Speaker-dependent low-level acoustic feature extraction for emotion recognition', The Journal of the Acoustical Society of America, 143: 1747–47.
Yang, Y.-H., and H. H. Chen. 2012. 'Machine recognition of music emotion: A review', ACM Transactions on Intelligent Systems and Technology (TIST), 3: 1–30.
Information & Authors
Information
Published In
History
Published online: Mar 18, 2024
ASCE Technical Topics:
- Algorithms
- Business management
- Construction engineering
- Construction industry
- Construction management
- Construction methods
- Engineering fundamentals
- Human and behavioral factors
- Mathematics
- Model accuracy
- Models (by type)
- Occupational safety
- Practice and Profession
- Public administration
- Public health and safety
- Safety
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.