International Journal on Science and Technology

E-ISSN: 2229-7677     Impact Factor: 9.88

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 17 Issue 2 April-June 2026 Submit your research before last 3 days of June to publish your research paper in the issue of April-June.

Face Emotion and Speech Recognition in Worker Stress Analysis

Author(s) Prof. SOWNDARYA S, Ms. PONLAKSHMI R, Ms. THARANI PRIYA M, Ms. KEERTHILAYA D, Ms. PRIYANKA SS
Country India
Abstract Human-computer interaction can benefit from better detection of emotion in the user. By allowing computers to detect and respond appropriately to users’ emotions, user experience will be improved. A new project has been developed which uses both facial recognition technology and audio recordings to detect user emotion. The first part of the system uses facial emotion detection technology. A Haar cascade classifier is used to identify a face in an image or video feed before passing this data onto a CNN-based model (Mini-XCEPTION) that determines what kind of facial expression has been made (Anger, Disgust, Fear, Happiness, Sadness, Surprise, Neutral). The next part of the system uses audio detection technology to identify the speaker's emotional state by analyzing audio recordings or live audio streams recorded through a microphone. The system uses audio samples to compile relevant acoustic features (MFCC's, Pitch, Energy) that are analyzed by machine/deep learning algorithms to identify the speaker's emotion. Both facial and audio detection systems output data about your emotion, which is then processed together through a multimodal fusion mechanism to improve performance and accuracy when detecting user emotion. The entire system was created with Python and utilizes Stream lit to provide a user-friendly interface for displaying real-time visualizations, analyzing confidence levels while detecting emotion and tracking emotion overtime.
Keywords Facial Emotion Recognition, Speech Emotion Recognition, Deep Learning
Field Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In Volume 17, Issue 2, April-June 2026
Published On 2026-04-03

Share this