AN INTELLIGENT SYSTEM FOR EMOTIONAL AND PSYCHOLOGICAL DISORDER DETECTION VIA FACIAL FEATURE ANALYSIS
DOI:
https://doi.org/10.48047/dfmbyb74Keywords:
Depression, Emotions, Classification, Deep Learning, DisorderAbstract
Depression is a common mental disorder that can have a big effect on people's daily lives. The detection of depressive disorder and other facial emotion recognition disorders plays an important role in identifying psychological disorders through facial features. The manual diagnosis process is time-consuming and hard for doctors. Therefore, many researchers have proposed computerized techniques for detecting these disorders. However, images can be analyzed for diagnosis. Deep convolutional neural networks (CNNs) have demonstrated impressive performance in different computer vision tasks. Many researchers suggest that machine learning techniques can be used to analyze emotional images, such as own syndrome images, to assist in the detection of disorders. Additionally, machine learning techniques can be used to analyze time-series data, such as videos, to detect changes in depression and facial emotion features over time. Facial emotional images may be of poor quality due to factors such as poor lighting or camera movements, which can make it difficult to accurately detect depressive disorder and facial emotion indefinitely. In this research, we have used computer vision-based techniques to detect are classify depressive disorders and facial emotion recognition. We used facial emotion datasets such as FER-2013, CK+, and JAFEE for training deep learning AlexNet model, but the results were not satisfactory, and then the AlexNet is used with classifiers by combined layer (fc6, fc7, fc8). The proposed method obtained 92.60% accuracy on the FER-2013 dataset. Our proposed method is better than existing methods because it identifies the facial features from any type of image provided.
Downloads
References
AGARWAL, M., MITTAL, M., GAUTAM, M., and NAND, D. (2022) Real-Time Facial Emotion Classification Using Deep Convolution Neural Network. In: 2022 2nd International Conference on Intelligent Technologies (CONIT 2022), pp. 1–6. doi: 10.1109/CONIT55038.2022.9847774.
ANSARI, L., JI, S., CHEN, Q., and CAMBRIA, E. (2023) Ensemble Hybrid Learning Methods for Automated Depression Detection. IEEE Transactions on Computational Social Systems, 10 (1), pp. 211–219. doi: 10.1109/TCSS.2022.3154442.
ANYANWU, C., and HAYS, C. (2022) A Convolutional Neural Network for Emotion Assessment and Recognition. In: Proceedings of the 2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS 2022), pp. 759–763. doi: 10.1109/MASS56207.2022.00119.
BORGALLI, R.A., and SURVE, S. (2022) Deep Learning Framework for Facial Emotion Recognition Using CNN Architectures. In: Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022), pp. 1777–1784. doi: 10.1109/ICEARS53579.2022.9751735.
AZIZI, F.N., KURNIAWARDHANI, A., and PAPUTUNGAN, I.V. (2022) Facial Expression Image Based Emotion Detection Using Convolutional Neural Network. In: 2022 IEEE 20th Student Conference on Research and Development (SCOReD 2022), pp. 157–162. doi: 10.1109/SCOReD57082.2022.9974104.
BEGAJ, S., TOPAL, A.O., and ALI, M. (2020) Emotion Recognition Based on Facial Expressions Using Convolutional Neural Network (CNN). In: Proceedings of the 2020 International Conference on Computing, Networking, Telecommunications and Engineering Sciences Applications (CoNTESA 2020), pp. 58–63. doi: 10.1109/CoNTESA50436.2020.9302866.
BENAVIDES, A., RUBIO, G., BARRANTES, G., VARGAS, J.J., LARA, A., and QUESADA, L. (2021) Emotions Classifier Based on Facial Expressions. In: Proceedings of the 5th Jornadas Costarricenses de Investigación en Computación e Informática (JoCICI 2021). doi: 10.1109/JoCICI54528.2021.9794338.
Umm-e-Asma, F.S., Shah, M.A., Abbas, K.J., Ramzan, H., Asif, I., Nija, D.E., Younas, E., Bilal, A. (2024) Exploring the Relationship Between Psychological Stressors and Myocardial Infarctions In Humans Using Statistical Techniques. African Journal Biomedical Research. 27-1
BHOGAN, S., SAWANT, K., GONDALEKAR, N., CARVALHO, R., KALANGUTKAR, V., and MATHEW, A. (2023) Facial Emotion Detection Using Machine Learning and Deep Learning Algorithms. In: Proceedings of the 2nd International Conference on Edge Computing and Applications (ICECAA 2023), pp. 1134–1139. doi: 10.1109/ICECAA58104.2023.10212261.
BISOGNI, C., CASTIGLIONE, A., HOSSAIN, S., NARDUCCI, F., and UMER, S. (2022) Impact of Deep Learning Approaches on Facial Expression Recognition in Healthcare Industries. IEEE Transactions on Industrial Informatics, 18 (8), pp. 5619–5627. doi: 10.1109/TII.2022.3141400.
BUKHARI, N., HUSSAIN, S., AYOUB, M., YU, Y., and KHAN, A. (2022) Deep Learning Based Framework for Emotion Recognition Using Facial Expression. Pakistan Journal of Engineering and Technology, 5 (3), pp. 51–57. doi: 10.51846/vol5iss3pp51-57.
CAHYANI, D.E., WIBAWA, A.P., PRASETYA, D.D., GUMILAR, L., AKHBAR, F., and TRIYULINAR, E.R. (2022) Emotion Detection in Text Using Convolutional Neural Network. In: Proceedings of the 2022 International Conference on Electrical and Information Technology (IEIT 2022), pp. 372–376. doi: 10.1109/IEIT56384.2022.9967913.
Bilal, A., Fatima, U. (2021) Diagnosis and Treatment of Hepatitis-C in Bhalwal: Case Report. Palliative Medicine and Care International Journal. 4-1:1-3.
CHEN, T., HONG, R., GUO, Y., HAO, S., and HU, B. (2022) MS$^{2}$-GNN: Exploring GNN Based Multimodal Fusion Network for Depression Detection. IEEE Transactions on Cybernetics, 1–11. doi: 10.1109/TCYB.2022.3197127.
DAS, A. and NEELIMA, N. (2023) Facial Expression Recognition System with Local Binary Features of Neural Network. In: 2023 International Conference on Data Science and Network Security, ICDSNS 2023, pp. 1–5. doi: 10.1109/ICDSNS58469.2023.10244983.
Bilal, A., Ahmad, S., Tanvir, F., Tariq, M., Ramzan, K., Saleem, M., Saleem, H, G.. (2022). Predictive modeling of N-acetyl transferase 2 single nucleotide polymorphisms and breast cancer risk using in-silco approaches. The Journal of Microbiology and Molecular Genetics.31;3-2:105-21.
DOSHI, U., BAROT, V. and GAVHANE, S. (2020) Emotion Detection and Sentiment Analysis of Static Images. In: 2020 International Conference on Convergence to Digital World - Quo Vadis, ICCDW 2020. doi: 10.1109/ICCDW45521.2020.9318713.
[15] EZERCELI, Ö. and ESKIL, M.T. (2022) Convolutional Neural Network (CNN) Algorithm Based Facial Emotion Recognition (FER) System for FER-2013 Dataset. In: International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2022. doi: 10.1109/ICECCME55909.2022.9988371.
GIRI, S., SINGH, G., KUMAR, B., SINGH, M., VASHISHT, D., SHARMA, S. and JAIN, P. (2022) Emotion Detection with Facial Feature Recognition Using CNN & OpenCV. In: 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering, ICACITE 2022, pp. 230–232. doi: 10.1109/ICACITE53722.2022.9823786.
GRANGER, E. (2019) Wheidima Carneiro de Melo, Abdenour Hadid, Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland, École de Technologie Supérieure, Montreal, Canada. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 4544–4548.
GRANGER, E. and BORDALLO LOPEZ, M. (2020) Encoding Temporal Information for Automatic Depression Recognition. In: Proceedings of the Conference, pp. 1080–1084.
Jawad, M., Bilal, A., Khan, S., Rizwan, M., Arshad, M.. (2023) Prevalence and awareness survey of tuberculosis in the suspected population of Bajaur Agency in Fata, Pakistan: Prevalence and awareness survey of tuberculosis. Pakistan Journal of Health Sciences. 30:56-61.
Ahmad, R.Z., Khan, M.S., Bilal, A., Ali, U., Sattar, R.Z,. (2023) Effect of Locus of Control and Depression Among Young Adults in Multan (Pakistan). Journal of Asian Development Studies.;12-4:684-92.
HOSUR, V. and DESAI, A. (2022) Facial Emotion Detection Using Convolutional Neural Networks. In: MysuruCon 2022 - 2022 IEEE 2nd Mysore Sub Section International Conference, pp. 1–4. doi: 10.1109/MysuruCon55714.2022.9972510.
HUSSEIN, E.S., QIDWAI, U. and AL-MEER, M. (2020) Emotional Stability Detection Using Convolutional Neural Networks. In: 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies, ICIoT 2020, pp. 136–140. doi: 10.1109/ICIoT48696.2020.9089440.
Bilal, A., Nazar, I., Abbas, A., Shafiq, H.R., Rafique, A., Saddiqa, A., Ali, K., Ullah, M.K. (2024) Existence, Evolution, History and Impact of Ebola Virus Disease on Humans: A Mini Review. International Journal of Environmental Chemistry. 10-1:1-5p.
JAISWAL, A., RAJU, A.K. and DEB, S. (2020) Facial Emotion Detection Using Deep Learning. In: 2020 International Conference for Emerging Technology, INCET 2020, 1:1–4. doi: 10.1109/INCET49848.2020.9154121.
JIANG, Z., HARATI, S., CROWELL, A., MAYBERG, H.S., NEMATI, S. and CLIFFORD, G.D. (2021) Classifying Major Depressive Disorder and Response to Deep Brain Stimulation over Time by Analyzing Facial Expressions. IEEE Transactions on Biomedical Engineering, 68 (2), pp. 664–672. doi: 10.1109/TBME.2020.3010472.
JOSHI, MANJU LATA, and NEHAL KANOONGO. 2022. Depression Detection Using Emotional Artificial Intelligence and Machine Learning: A Closer Review. Materials Today: Proceedings, 58(February), 217–226. https://doi.org/10.1016/j.matpr.2022.01.467
KARATAY, BUSRA, DENIZ BESTEPE, KASHFIA SAILUNAZ, TANSEL OZYER, and REDA ALHAJJ. 2022. A Multi-Modal Emotion Recognition System Based on CNN-Transformer Deep Learning Technique. Proceedings of the 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA 2022), 145–150. https://doi.org/10.1109/CDMA54072.2022.00029
KHAN, AMJAD REHMAN. 2022. Facial Emotion Recognition Using Conventional Machine Learning and Deep Learning Methods: Current Achievements, Analysis and Remaining Challenges. Information (Switzerland), 13(6). https://doi.org/10.3390/info13060268
KONDAVEETI, HARI KISHAN, and MOGILI VISHAL GOUD. 2020. Emotion Detection Using Deep Facial Features. Proceedings of IEEE International Conference on Advent Trends in Multidisciplinary Research and Innovation (ICATMRI 2020). https://doi.org/10.1109/ICATMRI51801.2020.9398439
KUMAR, ASHWINI, and CHINTAN THACKER. 2023. Using CNN and OpenCV, Emotion Recognition with Facial Feature Approach. 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), 2179–2182. https://doi.org/10.1109/icacite57410.2023.10183086
KUMAR, MANISH, and SWATI SRIVASTAVA. 2021. Emotion Detection through Facial Expression Using Deep Learning. 2021 5th International Conference on Information Systems and Computer Networks (ISCON 2021), 1–4. https://doi.org/10.1109/ISCON52037.2021.9702451
LI, XIAOJIE, RENLI YUE, WEIKUAN JIA, HONG WANG, and YUANJIE ZHENG. 2021. Recognizing Students’ Emotions Based on Facial Expression Analysis. Proceedings of the 11th International Conference on Information Technology in Medicine and Education (ITME 2021), 96–100. https://doi.org/10.1109/ITME53901.2021.00030
LOKESHNAIK, S. K., A. PUNITHA, P. VIJAYAKARTHIK, AJMEERA KIRAN, A. NARSIMHA DHANGAR, B. JATHIN REDDY, and M. SUDHEEKSHA. 2023. Real Time Facial Emotion Recognition Using Deep Learning and CNN. 2023 International Conference on Computer Communication and Informatics (ICCCI 2023), 1–5. https://doi.org/10.1109/ICCCI56745.2023.10128259
MAURYA, AAKASH, and VANDANA SHARMA. 2022. Facial Emotion Recognition Using Keras and CNN. 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE 2022), 2539–2543. https://doi.org/10.1109/ICACITE53722.2022.9823480
MODI, SHREY, and MOHAMMED HUSAIN BOHARA. 2021. Facial Emotion Recognition Using Convolution Neural Network. Proceedings of the 5th International Conference on Intelligent Computing and Control Systems (ICICCS 2021), 1339–1344. https://doi.org/10.1109/ICICCS51141.2021.9432156
MOHAN, MAYA, A. K. ABHINAV, AKARSH ASHOK, A. V. AKHIL, and P. R. ACHINTH. 2021. Depression Detection Using Facial Expression and Sentiment Analysis. 2021 Asian Conference on Innovation in Technology (ASIANCON 2021), 1–6. https://doi.org/10.1109/ASIANCON51346.2021.9544819
NI, RONGRONG, BIAO YANG, XU ZHOU, ANGELO CANGELOSI, and XIAOFENG LIU. 2023. Facial Expression Recognition Through Cross-Modality Attention Fusion. IEEE Transactions on Cognitive and Developmental Systems, 15(1), 175–185. https://doi.org/10.1109/TCDS.2022.3150019
POKURI, BHARATH KUMAR CHOWDARY, SATYA SREE ANNAMNEEDI, ANUDEEP CHOWDARY KAMANI, SAI THARUN THALLURI, and SARAYU KURRA. 2022. Mental Illness Detection with Facial Movements Using Neural Networks. Proceedings of the 2022 International Conference on Recent Trends in Microelectronics, Automation, Computing and Communications Systems (ICMACC 2022), 82–87. https://doi.org/10.1109/ICMACC54824.2022.10093486
PRATHWINI, ROSHAN FERNANDES, and ANISHA P. RODRIGUES. 2022. Emotion Detection in Multimedia Data Using Convolution Neural Network. International Conference on Artificial Intelligence and Data Engineering (AIDE 2022), 157–161. https://doi.org/10.1109/AIDE57180.2022.10060384
REDDY, BOLLA GOPI KRISHNA, PITCHUKA YASHWANTHSAAI, AKURATHI RAVI RAJA, AVINASH JAGARLAMUDI, NIMMAGADDA LEELADHAR, and TELLA TRILOK KUMAR. 2021. Emotion Recognition Based on Convolutional Neural Network (CNN). 2021 International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA 2021), 1–5. https://doi.org/10.1109/ICAECA52838.2021.9675688
RIZWAN, SYEDA AMNA, YAZEED YASIN GHADI, AHMAD JALAL, and KIBUM KIM. 2022. Automated Facial Expression Recognition and Age Estimation Using Deep Learning. Computers, Materials and Continua, 71(2), 5235–5252. https://doi.org/10.32604/cmc.2022.023328
SHAH, DEVANSHU, KHUSHI CHAVAN, SANKET SHAH, and PRATIK KANANI. 2021. Real-Time Facial Emotion Recognition. 2021 2nd Global Conference for Advancement in Technology (GCAT 2021), 1–4. https://doi.org/10.1109/GCAT52182.2021.9587707
SHANGGUAN, ZIXUAN, ZHENYU LIU, GANG LI, QIONGQIONG CHEN, ZHIJIE DING, and BIN HU. 2023. Dual-Stream Multiple Instance Learning for Depression Detection With Facial Expression Videos. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 554–563. https://doi.org/10.1109/TNSRE.2022.3204757
SHIRSATH, SUPRIYA, VAISHALI VIKHE, and PRATAP VIKHE. 2022. Xception CNN-Ensemble Learning Based Facial Emotion Recognition. 2022 6th International Conference on Computing, Communication, Control and Automation (ICCUBEA 2022), 1–4. https://doi.org/10.1109/ICCUBEA54992.2022.10011111
SINGH, JAITEG, and GAURAV GOYAL. 2021. Decoding Depressive Disorder Using Computer Vision. Multimedia Tools and Applications, 80(6), 8189–8212. https://doi.org/10.1007/s11042-020-10128-9
SINGH, SHUBHAM KUMAR, REVANT KUMAR THAKUR, SATISH KUMAR, and ROHIT ANAND. 2022. Deep Learning and Machine Learning Based Facial Emotion Detection Using CNN. Proceedings of the 2022 9th International Conference on Computing for Sustainable Global Development (INDIACom 2022), 530–535. https://doi.org/10.23919/INDIACom54597.2022.9763165
YU, YAO, SHUAI DING, XIAOJIAN LI, ZI XU, YAPING WANG, LINA QU, YINGHUI LI, and SHANLIN YANG. 2023. Cloud-Edge Collaborative Depression Detection Using Negative Emotion Recognition and Cross-Scale Facial Feature Analysis. IEEE Transactions on Industrial Informatics, 19(3), 3088–3098. https://doi.org/10.1109/TII.2022.3163512
ZHANG, J., MEI, X., LIU, H., YUAN, S., and QIAN, T. (2019) Detecting Negative Emotional Stress Based on Facial Expression in Real Time. 2019 IEEE 4th International Conference on Signal and Image Processing, ICSIP 2019, pp. 430–434. doi: 10.1109/SIPROCESS.2019.8868735.
ZHANG, Y., LI, X., RONG, L., and TIWARI, P. (2021) Multi-Task Learning for Jointly Detecting Depression and Emotion. Proceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021, pp. 3142–3149. doi: 10.1109/BIBM52615.2021.9669546.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
