Chapter
Mar 18, 2024

Acoustic Emotion Recognition for Improved Safety in the Construction Industry

Publication: Construction Research Congress 2024

ABSTRACT

In the construction industry, clear communication is vital for optimal teamwork and performance. Yet, diverse languages, accents, and continuous worksite noises often impede speech clarity, leading to misunderstandings and consequential errors. Addressing this challenge, the presented research introduces a speech emotion recognition algorithm tailored for construction. Unlike prior algorithms emphasizing language content, this approach centers on speech features, bridging a significant gap in construction-specific applications. The algorithm aims to identify four key emotions: anger, happiness, sadness, and neutrality, using six distinct acoustic features: pitch, intensity, frequency formants, jitter, shimmer, and zero crossing rate. Leveraging these features collectively enhances system accuracy. Designed in MATLAB, the decision-tree-based method calculates confidentiality intervals for each feature and is specifically developed for speaker-dependent emotion recognition. Remarkably, the algorithm secured an 86% accuracy rate in detecting anger, outperforming existing models. By recognizing and addressing the emotional states of workers, this method holds the promise to greatly enhance safety on construction sites, averting potential hazards.

Get full access to this article

View all available purchase options and get full access to this chapter.

REFERENCES

Arnold, M. B. 1960. Emotion and personality.
Averill, J. R. 1975. A semantic atlas of emotional concepts (American Psycholog. Ass., Journal Suppl. Abstract Service).
Bower, G. H. 1981. 'Mood and memory', American psychologist, 36: 129.
Cavazza, M., F. Charles, and S. J. Mead. 2002. 'Character-based interactive storytelling', IEEE Intelligent systems, 17: 17–24.
Cowie, R., and R. R. Cornelius. 2003. 'Describing the emotional states that are expressed in speech', Speech communication, 40: 5–32.
Cowie, R., E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor. 2001. 'Emotion recognition in human-computer interaction', IEEE Signal Processing Magazine, 18: 32–80.
Damasio, A. R. 2000. 'A second chance for emotion', Cognitive neuroscience of emotion, 2: 12–23.
Fónagy, I. 1981. 'Emotions, voice and music', Research aspects on singing, 33: 51–79.
Gobl, C., and A. N. Chasaide. 2003. 'The role of voice quality in communicating emotion, mood and attitude', Speech communication, 40: 189–212.
Huang, R., and C. Ma. 2006. “Toward a speaker-independent real-time affect detection system.” In 18th International Conference on Pattern Recognition (ICPR'06), 1204–07. IEEE.
James, L. 2009. Road rage and aggressive driving: Steering clear of highway warfare (Prometheus Books).
Kessous, L., G. Castellano, and G. Caridakis. 2010. 'Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis', Journal on Multimodal User Interfaces, 3: 33–48.
Knauer, H. S., S. Pedersen, C. N. Reherman, J. L. Rochat, E. S. Thalheimer, M. C. Lau, G. G. Fleming, M. Ferroni, and C. Corbisier. 2006. FHWA highway construction noise handbook. In.: United States. Federal Highway Administration.
Kramer, E. 1963. 'Judgment of personal characteristics and emotions from nonverbal properties of speech', Psychological Bulletin, 60: 408.
Lee, C. M., and S. S. Narayanan. 2005. 'Toward detecting emotions in spoken dialogs', IEEE transactions on speech and audio processing, 13: 293–303.
Liberman, M. 2002. 'Emotional prosody speech and transcripts', http://www. ldc. upenn. edu/Catalog/CatalogEntry. jsp? catalogId= LDC2002S28.
Lo, V. E.-W., and P. A. Green. 2013. 'Development and evaluation of automotive speech interfaces: useful information from the human factors and the related literature', International Journal of Vehicular Technology, 2013.
Mansoor, A., S. Liu, G. M. Ali, A. Bouferguene, M. Al-Hussein, and I. Hassan. 2022. 'Keyword identification framework for speech communication on construction sites', Modular and Offsite Construction (MOC) Summit Proceedings: 106–13.
Mowrer, O. 1960. 'Learning theory and behavior'.
Murray, I. R., and J. L. Arnott. 1993. 'Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion', The Journal of the Acoustical Society of America, 93: 1097–108.
Pollack, I., H. Rubenstein, and A. Horowitz. 1960. 'Communication of verbal modes of expression', Language and Speech, 3: 121–30.
Scherer, K. R. 1984. 'On the nature and function of emotion: A component process approach', Approaches to emotion, 2293: 31.
Scherer, K. R. 2003. 'Vocal communication of emotion: A review of research paradigms', Speech communication, 40: 227–56.
Scherer, K. R., R. Banse, and H. G. Wallbott. 2001. 'Emotion inferences from vocal expression correlate across languages and cultures', Journal of Cross-cultural psychology, 32: 76–92.
Sethu, V., J. Epps, and E. Ambikairajah. 2014. 'Speech based emotion recognition.' in, Speech and Audio Processing for Coding, Enhancement and Recognition (Springer).
Tarnowski, P., M. Kołodziej, A. Majkowski, and R. J. Rak. 2017. 'Emotion recognition using facial expressions', Procedia Computer Science, 108: 1175–84.
Tomkins, S. 1962. Affect imagery consciousness: Volume I: The positive affects (Springer publishing company).
Udhan, T., and S. Bernadin. 2018. 'Speaker-dependent low-level acoustic feature extraction for emotion recognition', The Journal of the Acoustical Society of America, 143: 1747–47.
Yang, Y.-H., and H. H. Chen. 2012. 'Machine recognition of music emotion: A review', ACM Transactions on Intelligent Systems and Technology (TIST), 3: 1–30.

Information & Authors

Information

Published In

Go to Construction Research Congress 2024
Construction Research Congress 2024
Pages: 477 - 486

History

Published online: Mar 18, 2024

Permissions

Request permissions for this article.

ASCE Technical Topics:

Authors

Affiliations

Tejal Mulay, Ph.D. [email protected]
1Assistant Professor, Dept. of Computer Science and Engineering Technology, Valdosta State Univ. Email: [email protected]
Mohamed Khalafalla, Ph.D. [email protected]
2Assistant Professor, Division of Engineering Technology, Florida A&M Univ., Tallahassee, FL. ORCID: https://orcid.org/0000-0001-9621-4940. Email: [email protected]
Shonda Bernadin, Ph.D. [email protected]
3Associate Professor, FAMU-FSU College of Engineering Technology, Tallahassee, FL. Email: [email protected]

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Paper
$35.00
Add to cart
Buy E-book
$190.00
Add to cart

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Paper
$35.00
Add to cart
Buy E-book
$190.00
Add to cart

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share