Deep Learning Model Integrating Computational Linguistics: Research on Depression Recognition via Facial Expressions and Speech Signals

Authors

  • Hongzhi Pan
  • Yungang Wei

DOI:

https://doi.org/10.54097/2ngv0c66

Keywords:

Computational Linguistics, Depression Recognition, Deep Learning, Facial Expressions, Speech Signals, Multimodal Fusion

Abstract

To address the challenges of subjective diagnosis and delayed identification caused by imbalanced doctor-patient ratios in depression treatment, this study proposes a deep learning-based multimodal recognition system integrating facial expressions and speech signals. By analyzing characteristic audiovisual patterns in depression patients, we developed a dual-modal feature extraction and fusion framework: 3D Convolutional Neural Networks (3D-CNN) extract spatiotemporal facial expression features, while Causal Convolutional Networks (C-CNN) capture long-term temporal patterns in speech signals. The attention mechanism enables adaptive fusion across modalities. Experimental results on DAIC-WOZ and AVEC2013 datasets demonstrate 83.7% accuracy in classifying major depressive disorder, with an average regression error of 3.52 on the PHQ scale-outperforming single-modal and traditional fusion methods. This research provides a technical foundation for objective and automated depression screening.

Downloads

Download data is not yet available.

References

[1] Davidson F K G. DEFINITION AND EPIDEMIOLOGY OF TREATMENT-RESISTANT DEPRESSION[J]. Psychiatric Clinics of North America, 1996. DOI:10.1016/S0193-953X(05)70283-5.

[2] Smith M, Dietrich B J, Bai E W, et al. Vocal pattern detection of depression among older adults[J]. International Journal of Mental Health Nursing, 2019. DOI:10.1111/inm.12678.

[3] Lacroix A, Calvet B, Laplace B, et al. Predictors of clinical response after rTMS treatment of patients suffering from drug-resistant depression[J]. Translational Psychiatry, 2021, 11: 1-7. DOI:10.1038/s41398-021-01555-9.

[4] Di Y, Wang J, Li W, et al. Using i-vectors from voice features to identify major depressive disorder[J]. Journal of Affective Disorders, 2021, 288(10). DOI:10.1016/j.jad.2021.04.004.

[5] Simon T L, Smith K J. The issues in autologous transfusion[J]. 2019. DOI:10.1016/0046-8177(89)90196-2.

[6] Sverdlov O, Curcic J, Hannesdottir K, et al. A Study of Novel Exploratory Tools, Digital Technologies, and Central Nervous System Biomarkers to Characterize Unipolar Depression[J]. Frontiers in Psychiatry, 2021, 12: 640741. DOI:10.3389/fpsyt.2021.640741.

[7] Nuez N A, Joseph B, Pahwa M, et al. Augmentation strategies for treatment resistant major depression: A systematic review and network meta-analysis[J]. Journal of Affective Disorders, 2022, 302: 385-400. DOI:10.1016/j.jad.2021.12.134.

[8] Zhu T, Li K, Herrero P, et al. Deep Learning for Diabetes: A Systematic Review[J]. IEEE Journal of Biomedical and Health Informatics (J-BHI), 2021, 25(7). DOI:10.1109/JBHI.2020.3040225.

[9] Zhang L, Duvvuri R, Chandra K K L, et al. Automated voice biomarkers for depression symptoms using an online cross-sectional data collection initiative[J]. Depression & Anxiety (1091-4269), 2020, 37(7). DOI:10.1002/da.23020.

[10] Hansen L, Zhang Y, Wolf D, et al. A Generalizable Speech Emotion Recognition Model Reveals Depression and Remission[J]. Acta Psychiatrica Scandinavica, 2021. DOI:10.1111/acps.13388.

Downloads

Published

23-12-2025

Issue

Section

Articles

How to Cite

Pan, H., & Wei, Y. (2025). Deep Learning Model Integrating Computational Linguistics: Research on Depression Recognition via Facial Expressions and Speech Signals. International Journal of Education and Humanities, 21(3), 53-57. https://doi.org/10.54097/2ngv0c66