Abstract
Affective computing has grown into a field that spans emotion theory, multimodal data, computational modeling, human–AI interaction, social applications, and ethical evaluation, yet most existing surveys address only one or two of these layers at a time and fail to capture how the layers depend on one another. Rather than offering another comprehensive survey, this position paper reorganizes the field around a six-layer pipeline and reads the literature as a chain of dependencies in which upstream design choices propagate into downstream evaluation, deployment, and governance. Our central diagnosis is that four silo-bridge patterns recur across the field: disconnections at the theory/data (Pattern 1), modeling/interaction (Pattern 2), technology/ethics (Pattern 3), and data/social-application (Pattern 4) boundaries. These are not incidental implementation problems but recurring failure modes rooted in the pipeline structure itself, forming a cascade in which upstream patterns structurally enable downstream ones. We argue that this dependency chain is not merely a technical liability but a socio-philosophical one: as upstream inconsistencies propagate downstream, they threaten the individual’s authority to interpret their own emotions (emotional agency), pull affective expression toward machine-legible norms (algorithmic conformity), and relocate the power to define and act on emotion from persons to the institutions that operate these systems (institutional power). As a prescriptive program, we present five integrated design criteria: theory-explicit affective representation (DC1), affective reasoning with bounded intervention (DC2), longitudinal interaction-in-the-loop evaluation (DC3), deployment-specific affective accountability (DC4), and user-retained interpretive authority (DC5). The five criteria are designed as an interlocking response to the cascade rather than as independent recommendations. We situate affective AI not as a mere recognition or generation technology but as a sociotechnical system that can intervene in human emotional interpretation, interaction, and social decision-making, and call upon future research to move toward systems that are more coherent, more responsible, and more respectful of human emotional agency.
Similar content being viewed by others
Data availability
No datasets were generated or analyzed during the current study.
Materials availability
Not applicable.
Code availability
Not applicable.
References
Ahir A, Gohokar V (2019) Driver inattention monitoring system: a review. In: Proceedings of the 2019 international conference on innovative trends and advances in engineering and technology (ICITAET), pp 188–194. https://doi.org/10.1109/ICITAET47105.2019.9170249
Andalibi N, Ingber AS (2025) Public perceptions about emotion ai use across contexts in the united states. In: Proceedings of the 2025 CHI conference on human factors in computing systems, pp 1–16. https://doi.org/10.1145/3706598.3713501
Averill FR (1980) A constructivist view of emotion. In: Theories of emotion, pp 305–339. https://doi.org/10.1016/B978-0-12-558701-3.50018-1
Barker D, Tippireddy MKR, Farhan A, Ahmed B (2025) Ethical considerations in emotion recognition research. Psychol Int 7(2):43. https://doi.org/10.3390/psycholint7020043
Barrett LF (2006) Are emotions natural kinds? Perspect Psychol Sci 1(1):28–58. https://doi.org/10.1111/j.1745-6916.2006.00003.x
Barrett LF (2017) The theory of constructed emotion: an active inference account of interoception and categorization. Soc Cognit Affect Neurosci 12(1):1–23. https://doi.org/10.1093/scan/nsw154
Barrett LF, Adolphs R, Marsella S, Martinez AM, Pollak SD (2019) Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Psychol Sci Public Interest 20(1):1–68. https://doi.org/10.1177/1529100619832930
Bender EM, Friedman B (2018) Data statements for natural language processing: toward mitigating system bias and enabling better science. Trans Assoc Comput Linguist 6:587–604. https://doi.org/10.1162/tacl_a_00041
Benjamin R (2019) Race after technology: abolitionist tools for the new jim code. Polity
Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput Hum Interact 12(2):293–327. https://doi.org/10.1145/1067860.1067867
Bohus D, Horvitz E (2009) Models for multiparty engagement in open-world dialog. In: Proceedings of SIGDIAL 2009, pp 225–234
Bostan LAM, Klinger R (2018) An analysis of annotated corpora for emotion classification in text. In: Proceedings of the 27th international conference on computational linguistics, pp 2104–2119
Breazeal C (2003) Emotion and sociable humanoid robots. Int J Hum Comput Stud 59(1–2):119–155. https://doi.org/10.1016/S1071-5819(03)00018-1
Breazeal C (2004) Designing sociable robots. MIT Press. https://doi.org/10.7551/mitpress/2376.001.0001
Broekens J, Hilpert B, Verberne S, Baraka K, Gebhard P, Plaat A (2023) Fine-grained affective processing capabilities emerging from large language models. In: Proceedings of the international conference on affective computing and intelligent interaction, pp 1–8. https://doi.org/10.1109/ACII59096.2023.10388177
Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Proceedings of the 1st conference on fairness, accountability and transparency. Proceedings of machine learning research, vol 81, pp 77–91
Burkhardt F, Paeschke A, Rolfes M, Sendlmeier WF, Weiss B (2005) A database of German emotional speech. In: Proceedings of the annual conference of the international speech communication association, vol 5. ISCA, Lisbon, pp 1517–1520. https://doi.org/10.21437/Interspeech.2005-446
Busso C, Bulut M, Lee C-C, Kazemzadeh A, Mower E, Kim S, Chang JN, Lee S, Narayanan SS (2008) IEMOCAP: interactive emotional dyadic motion capture database. Lang Resour Eval 42:335–359. https://doi.org/10.1007/S10579-008-9076-6
Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput. https://doi.org/10.1109/T-AFFC.2010.1
Cambria E (2016) Affective computing and sentiment analysis. IEEE Intell Syst 31(2):102–107. https://doi.org/10.1109/MIS.2016.31
Cao H, Cooper DG, Keutmann MK, Gur RC, Nenkova A, Verma R (2014) CREMA-D: crowd-sourced emotional multimodal actors dataset. IEEE Trans Affect Comput 5(4):377–390. https://doi.org/10.1109/TAFFC.2014.2336244
Cassell J, Sullivan J, Prevost S, Churchill EF (2000) Embodied conversational agents. MIT Press. https://doi.org/10.7551/mitpress/2697.001.0001
Chalmers DJ (1995) Facing up to the problem of consciousness. J Conscious Stud 2(3):200–219. https://doi.org/10.1093/acprof:oso/9780195311105.003.0001
Chandra NA, Murtfeldt R, Qiu L, Karmakar A, Lee H, Tanumihardja E, Farhat K, Caffee B, Paik S, Lee C, Choi J, Kim A, Etzioni O (2025) Deepfake-Eval-2024: a multi-modal in-the-wild benchmark of deepfakes circulated in 2024. https://doi.org/10.48550/arXiv.2503.02857 arXiv: 2503.02857
Chavan V, Cenaj A, Shen S, Bar A, Binwani S, Del Becaro T, Funk M, Greschner L, Hung R, Klein S, Kleiner R, Krause S, Olbrych S, Parmar V, Sarafraz J, Soroko D, Withanage Don D, Zhou C, Vu HTD, Semnani P, Weinhardt D, Andre E, Kr ger J, Fresquet X (2025) Feeling machines: ethics, culture, and the rise of emotional ai. https://doi.org/10.48550/arXiv.2506.12437 arXiv:2506.12437
Cheng Z, Cheng Z-Q, He J-Y, Sun J, Wang K, Lin Y, Lian Z, Peng X, Hauptmann AG (2024) Emotion-LLaMA: multimodal emotion recognition and reasoning with instruction tuning. In: Proceedings of the 38th international conference on neural information processing systems, pp 110805–110853. https://doi.org/10.5555/3737916.3741434
Damasio A (1994) Descartes’ error: emotion, reason, and the human brain. Putnam
Davidson RJ (2004) What does the prefrontal cortex“do’’in affect: perspectives on frontal EEG asymmetry research. Biol Psychol 67(1–2):219–234. https://doi.org/10.1016/j.biopsycho.2004.03.008
Davis MH (1983) Measuring individual differences in empathy: evidence for a multidimensional approach. J Pers Soc Psychol 44(1):113–126. https://doi.org/10.1037/0022-3514.44.1.113
Davison AK, Lansley C, Costen N, Tan K, Yap MH (2018) Samm: a spontaneous micro-facial movement dataset. IEEE Trans Affect Comput 9(1):116–129. https://doi.org/10.1109/TAFFC.2016.2573832
De Choudhury M, Gamon M, Counts S, Horvitz E (2013) Predicting depression via social media. In: Proceedings of the international AAAI conference on web and social media, vol 7, pp 128–137. https://doi.org/10.1609/icwsm.v7i1.14432
Decety J, Jackson PL (2004) The functional architecture of human empathy. Behav Cogn Neurosci Rev 3(2):71–100. https://doi.org/10.1177/1534582304267
Défossez A, Mazaré L, Orsini M, Royer A, Pérez P, Jégou H, Grave E, Zeghidour N (2024) Moshi: a speech-text foundation model for real-time dialogue. https://doi.org/10.48550/arXiv.2410.00037 arXiv:2410.00037
Demszky D, Movshovitz-Attias D, Ko J, Cowen A, Nemade G, Ravi S (2020) GoEmotions: a dataset of fine-grained emotions. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 4040–4054. https://doi.org/10.18653/v1/2020.acl-main.372
DeVault D, Artstein R, Benn G, Dey T, Fast E, Gainer A, Georgila K, Gratch J, Hartholt A, Lhommet M, Lucas G, Marsella S, Morbini F, Nazarian A, Scherer S, Stratou G, Suri A, Traum D, Wood R, Xu Y, Rizzo A, Morency L-P (2014) Simsensei kiosk: a virtual human interviewer for healthcare decision support. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, pp 1061–1068. https://doi.org/10.5555/2615731.2617415
Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 4171–4186. https://doi.org/10.18653/v1/N19-1423
Dinan E, Roller S, Shuster K, Fan A, Auli M, Weston J (2019) Wizard of wikipedia: knowledge-powered conversational agents. In: Proceedings of the international conference on learning representations. https://doi.org/10.48550/arXiv.1811.01241
D’Mello S (2013) A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J Educ Psychol 105(4):1082–1099. https://doi.org/10.1037/a0032674
D’Mello SK, Graesser A (2010) Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model User-Adap Interact 20:147–187. https://doi.org/10.1007/s11257-010-9074-4
Duffy BR (2003) Anthropomorphism and the social robot. Robot Auton Syst 42(3–4):177–190. https://doi.org/10.1016/S0921-8890(02)00374-3
Ekman P (1992) An argument for basic emotions. Cognit Emot 6(3–4):169–200. https://doi.org/10.1080/02699939208411068
Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press. https://doi.org/10.1037/t27734-000
Elfenbein HA, Ambady N (2002) On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychol Bull 128(2):203–235. https://doi.org/10.1037/0033-2909.128.2.203
Emmelkamp PMG, Meyerbrker K (2021) Virtual reality therapy in mental health. Annu Rev Clin Psychol 17:495–519. https://doi.org/10.1146/annurev-clinpsy-081219-115923
Fang CM, Liu AR, Danry V, Lee E, Chan SWT, Pataranutaporn P, Maes P, Phang J, Lampe M, Ahmad L, Agarwal S (2024) How AI and human behaviors shape psychosocial effects of chatbot use: a longitudinal randomized controlled study. https://doi.org/10.48550/arXiv.2503.17473 arXiv: 2503.17473
Fischer T, Biemann C (2024) Exploring large language models for qualitative data analysis. In: Proceedings of the 4th international conference on natural language processing for digital humanities, pp 423–437. https://aclanthology.org/2024.nlp4dh-1.41/
Fitzpatrick KK, Darcy A, Vierhile M (2017) Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4(2):19. https://doi.org/10.2196/mental.7785
Foucault M (1977) Discipline and punish: the birth of the prison. Vintage Books
Gabriel S, Puri I, Xu X, Malgaroli M, Ghassemi M (2024) Can AI relate: testing large language model response for mental health support. In: Findings of the association for computational linguistics: EMNLP 2024, pp 2206–2221. https://doi.org/10.18653/v1/2024.findings-emnlp.120
Gebru T, Morgenstern J, Vecchione B, Vaughan JW, Wallach H, Daum H III, Crawford K (2021) Datasheets for datasets. Commun ACM 64(12):86–92
Geirhos R, Jacobsen J-H, Michaelis C, Zemel R, Brendel W, Bethge M, Wichmann FA (2020) Shortcut learning in deep neural networks. Nat Mach Intell 2:665–673. https://doi.org/10.1038/s42256-020-00257-z
Gergen KJ (1985) The social constructionist movement in modern psychology. Am Psychol 40(3):266–275. https://doi.org/10.1037/0003-066X.40.3.266
Ghosal D, Majumder N, Poria S, Chhaya N, Gelbukh A (2019) DialogueGCN: a graph convolutional neural network for emotion recognition in conversation. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 154–164. https://doi.org/10.18653/v1/D19-1015
Gilardi F, Alizadeh M, Kubli M (2023) ChatGPT outperforms crowd workers for text-annotation tasks. Proc Natl Acad Sci. https://doi.org/10.1073/pnas.2305016120
Goodfellow IJ, Erhan D, Carrier PL, Courville A, Mirza M, Hamner B, Cukierski W, Tang Y, Thaler D, Lee D-H, Zhou Y, Ramaiah C, Feng F, Li R, Wang X, Athanasakis D, Shawe-Taylor J, Milakov M, Park J, Ionescu R, Popescu M, Grozea C, Bergstra J, Xie J, Romaszko L, Xu B, Chuang Z, Bengio Y (2013) Challenges in representation learning: a report on three machine learning contests. In: Proceedings of the international conference on neural information processing, pp 117–124. https://doi.org/10.1007/978-3-642-42051-1_16
Gross JJ (1998) The emerging field of emotion regulation: an integrative review. Rev Gen Psychol 2(3):271–299. https://doi.org/10.1037/1089-2680.2.3.271
Gross JJ (2015) Emotion regulation: current status and future prospects. Psychol Inq 26(1):1–26. https://doi.org/10.1080/1047840X.2014.940781
Gross JJ, Levenson RW (1995) Emotion elicitation using films. Cognit Emot 9(1):87–108. https://doi.org/10.1080/02699939508408966
Guo Z, Lai A, Thygesen JH, Farrington J, Keen T, Li K (2024) Large language models for mental health applications: systematic review. JMIR Mental Health 11:57400. https://doi.org/10.2196/57400
Hagendorff T (2020) The ethics of AI ethics: an evaluation of guidelines. Mind Mach 30(1):99–120. https://doi.org/10.1007/s11023-020-09517-8
Halkiopoulos C, Gkintoni E, Aroutzidis A, Antonopoulou H (2025) Advances in neuroimaging and deep learning for emotion detection: a systematic review of cognitive neuroscience and algorithmic innovations. Diagnostics 15(4):456. https://doi.org/10.3390/diagnostics15040456
He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284. https://doi.org/10.1109/TKDE.2008.239
Hegde K, Jayalath H (2025) Emotions in the loop: a survey of affective computing for emotional support. https://doi.org/10.48550/arXiv.2505.01542 arXiv:2505.01542
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hochschild AR (1979) The managed heart: commercialization of human feeling. University of California Press
Huang C-ZA, Vaswani A, Uszkoreit J, Shazeer N, Simon I, Hawthorne C, Dai AM, Hoffman MD, Dinculescu M, Eck D (2018) Music transformer. https://doi.org/10.48550/arXiv.1809.04281 arXiv:1809.04281
Huang X, Hong X, Mao Q, Zheng W, Dhall A (2024) A survey on deep learning for group-level emotion recognition. IEEE Trans Comput Soc Syst 13(2):2475–2500. https://doi.org/10.1109/TCSS.2025.3638859
Hutto C, Gilbert E (2014) VADER: a parsimonious rule-based model for sentiment analysis of social media text. In: Proceedings of the international AAAI conference on web and social media, vol 8, pp 216–225
Imel ZE, Caperton DD, Tanana M, Atkins DC (2017) Technology-enhanced human interaction in psychotherapy. J Couns Psychol 64(4):385–393. https://doi.org/10.1037/cou0000213
Indrasiri PL, Kashyap B, Kolambahewage C, Nakisa B, Ijaz K, Pathirana PN (2024) VR based emotion recognition using deep multimodal fusion with biosignals across multiple anatomical domains. https://doi.org/10.48550/arXiv.2412.02283 arXiv:2412.02283
Inkster B, Sarda S, Subramanian V (2018) Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4(2):19. https://doi.org/10.2196/12106
Inoshita K (2024) Sentiment analysis of Japanese twitter users regarding the Ukraine-Russia War and its implications for security policy. In: 2024 11th international conference on information technology, computer, and electrical engineering, pp 338–343. https://doi.org/10.1109/ICITACEE62763.2024.10762783
Inoshita K, Harada R (2026) PersonaGen: persona-based synthetic data generation using multi-stage conditioning with large language models for emotion recognition. Int J Act Behav Comput 1:1–18. https://doi.org/10.60401/ijabc.133
Inoshita K, Mizuno S (2026) World model inspired sarcasm reasoning with large language model agents. https://doi.org/10.48550/arXiv.2512.24329
Inoshita K, Tomisu H, Zhou X, Kawai A, Yada K (2026) KDDA: a knowledge-driven domain and diversity alignment framework for emotion data generation with large language models. Int J Act Behav Comput 1:1–24
Inoshita K, Zhou X, Kawai A, Yada K (2026b) LLMs capture emotion labels, not emotion uncertainty: distributional analysis and calibration of human-LLM judgment gaps. https://doi.org/10.48550/arXiv.2604.27345
Irfan B, Kuoppamäki S, Skantze G (2024) Recommendations for designing conversational companion robots with older adults through foundation models. Front Robot AI. https://doi.org/10.3389/frobt.2024.1363713
Jiang W, Windl M, Tag B, Sarsenbayeva Z, Mayer S (2024) An immersive and interactive vr dataset to elicit emotions. IEEE Trans Visual Comput Graph 30(11):7343–7353. https://doi.org/10.1109/TVCG.2024.3456202
Jin Y, Liu J, Li P, Wang B, Yan Y, Zhang H, Ni C, Wang J, Li Y, Bu Y, Wang Y (2025) The applications of large language models in mental health: scoping review. J Med Internet Res 27:69284. https://doi.org/10.2196/69284
Jobin A, Ienca M, Vayena E (2019) The global landscape of AI ethics guidelines. Nat Mach Intell 1:389–399. https://doi.org/10.1038/s42256-019-0088-2
Johnson KT, Narain J, Thomas Q, Maes P, Picard RW (2023) Recanvo: a database of real-world communicative and affective nonverbal vocalizations. Sci Data. https://doi.org/10.1038/s41597-023-02405-7
Ju Z, Wang Y, Shen K, Tan X, Xin D, Yang D, Liu Y, Leng Y, Song K, Tang S, Wu Z, Qin T, Li X, Ye W, Zhang S, Bian J, He L, Li J, Zhao S (2024) NaturalSpeech 3: zero-shot speech synthesis with factorized codec and diffusion models. In: Proceedings of the 41st international conference on machine learning, pp 22605–22623. https://doi.org/10.5555/3692070.3692979
Kaplan AD, Kessler TT, Brill JC, Hancock PA (2021) Trust in artificial intelligence: meta-analytic findings. J Hum Factors Ergon Soc. https://doi.org/10.1177/00187208211013988
Khalil HA, Hammad SA, Munim HEAE, Maged SA (2023) Low-cost driver monitoring system using deep learning. IEEE Access 13:14151–14164. https://doi.org/10.1109/ACCESS.2025.3530296
Khan UA, Xu Q, Liu Y, Lagstedt A, Alamäki A, Kauttonen J (2024) Exploring contactless techniques in multimodal emotion recognition: insights into diverse applications, challenges, solutions, and prospects. Multimed Syst. https://doi.org/10.1007/s00530-024-01302-2
Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing, pp 1746–1751. https://doi.org/10.3115/v1/D14-1181
Kim RS (2026) Formal and computational foundations for implementing affective sovereignty in emotion AI systems. Discov Artif Intell 6(235):501–507. https://doi.org/10.1007/s44163-026-01000-0
Koelstra S, Mühl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3(1):18–31. https://doi.org/10.1109/T-AFFC.2011.15
Kumar MJD, Rao MS, Narendra KC (2025) Multimodal emotion recognition: a comprehensive survey of datasets, methods, and applications. IEEE Access 13:201067–201097. https://doi.org/10.1109/ACCESS.2025.3636186
Lang PJ, Bradley MM, Cuthbert BN (2005) International affective picture system (iaps): affective ratings of pictures and instruction manual. Technical Report Technical Report A-6. University of Florida, Gainesville
Lazarus RS (1991) Emotion and adaptation. Oxford University Press
LeDoux JE (1998) The emotional brain: the mysterious underpinnings of emotional life. Weidenfeld & Nicolson
Lei S, Dong G, Wang X, Wang K, Qiao R, Wang S (2024a) InstructERC: reforming emotion recognition in conversation with multi-task retrieval-augmented large language models. arXiv:2309.11911. https://doi.org/10.48550/arXiv.2309.11911
Lei S, Zhou Y, Tang B, Lam MWY, Liu F, Liu H, Wu J, Kang S, Wu Z, Meng H (2024b) SongCreator: lyrics-based Universal Song Generation. In: Proceedings of the 38th international conference on neural information processing systems, pp 80107–80140. https://doi.org/10.52202/079017-2546
Leite I, Castellano G, Pereira A, Martinho C, Paiva A (2014) Empathic robots for long-term interaction. Int J Soc Robot 6:329–341. https://doi.org/10.1007/s12369-014-0227-1
Li Y, Su H, Shen X, Li W, Cao Z, Niu S (2017) DailyDialog: a manually labelled multi-turn dialogue dataset. In: Proceedings of the 8th international joint conference on natural language processing, pp 986–995
Lian Z, Sun H, Chen L, Sun H, Sun L, Ren Y, Cheng Z, Liu B, Liu R, Peng X, Yi J, Tao J (2025) AffectGPT: a new dataset, model, and benchmark for emotion understanding with multimodal large language models. In: Proceedings of the 42nd international conference on machine learning, pp 36993–37014
Lin Z, Madotto A, Shin J, Xu P, Fung P (2019) MoEL: mixture of empathetic listeners. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 121–132. https://doi.org/10.18653/v1/D19-1012
Lin H, Czarnek G, Lewis B, White JP, Berinsky AJ, Costello T, Pennycook G, Rand DG (2025) Persuading voters using human-artificial intelligence dialogues. Nature 648:394–401. https://doi.org/10.1038/s41586-025-09771-9
Liu Z, Shen Y, Lakshminarasimhan VB, Liang PP, Zadeh AB, Morency L-P (2018) Efficient low-rank multimodal fusion with modality-specific factors. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp 2247–2256. https://doi.org/10.18653/v1/P18-1209
Liu S, Zheng C, Demasi O, Sabour S, Li Y, Yu Z, Jiang Y, Huang M (2021) Towards emotional support dialog systems. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, pp 3469–3483. https://doi.org/10.18653/v1/2021.acl-long.269
Liu Y, Dai W, Feng C, Wang W, Yin G, Zeng J, Shan S (2022) MAFW: a large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild. In: Proceedings of the 30th ACM international conference on multimedia. Association for Computing Machinery, pp 24–32. https://doi.org/10.1145/3503161.3548190
Livingstone SR, Russo FA (2018) The Ryerson audio-visual database of emotional speech and song (ravdess): a dynamic, multimodal set of facial and vocal expressions in north American English. PLoS ONE 13(5):0196391. https://doi.org/10.1371/journal.pone.0196391
Lotfian R, Busso C (2019) Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Trans Affect Comput 10(4):471–483. https://doi.org/10.1109/TAFFC.2017.2736999
Lubold N, Walker E, Pon-Barry H, Ogan A (2018) Automated pitch convergence improves learning in a social, teachable robot for middle school mathematics. In: Proceedings of the 19th international conference on artificial intelligence in education. Lecture notes in computer science, vol 10947, pp 282–296. https://doi.org/10.1007/978-3-319-93843-1_21
Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn-Kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE computer society conference on computer vision and pattern recognition workshops, pp 94–101. https://doi.org/10.1109/CVPRW.2010.5543262
Lutz CA (1988) Unnatural emotions: everyday sentiments on a micronesian atoll and their challenge to western theory. University of Chicago Press
Ma Z, Zheng Z, Ye J, Li J, Gao Z, Zhang S, Chen X (2024) emotion2vec: self-supervised pre-training for speech emotion representation. In: Findings of the association for computational linguistics: ACL 2024, pp 15747–15760. https://doi.org/10.18653/v1/2024.findings-acl.931
Ma F, Yuan Y, Xie Y, Ren H, Liu I, He Y, Ren F, Yu FR, Ni S (2025) Generative technology for human emotion recognition: a scoping review. Inf Fusion 115:102753. https://doi.org/10.1016/j.inffus.2024.102753
Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pp 142–150
Madiega T (2024) Artificial intelligence act. Briefing PE 698.792. European Parliamentary Research Service
Maeda T, Quan-Haase A (2024) When human-AI interactions become parasocial: agency and anthropomorphism in affective design. In: Proceedings of the 2024 ACM conference on fairness, accountability, and transparency. https://doi.org/10.1145/3630106.3658956
Majumder N, Poria S, Hazarika D, Mihalcea R, Gelbukh A, Cambria E (2019) DialogueRNN: an attentive RNN for emotion detection in conversations. In: Proceedings of the thirty-third AAAI conference on artificial intelligence, pp 6818–6825. https://doi.org/10.1609/aaai.v33i01.33016818
Mancuso V, Borghesi F, Chirico A, Bruni F, Sarcinella ED, Pedroli E, Cipresso P (2024) Iavrs international affective virtual reality system: psychometric assessment of 360 images by using psychophysiological data. Sensors 24(13):4204. https://doi.org/10.3390/s24134204
Maples B, Cerit M, Vishwanath A, Pea R (2024) Loneliness and suicide mitigation for students using GPT3-enabled chatbots. NPJ Ment Health Res. https://doi.org/10.1038/s44184-023-00047-6
McColl D, Hong A, Hatakeyama N, Nejat G, Benhabib B (2016) A survey of autonomous human affect detection methods for social robots engaged in natural HRI. J Intell Robot Syst 82:101–133. https://doi.org/10.1007/s10846-015-0259-2
McKeown G, Valstar M, Cowie R, Pantic M, Schroder M (2012) The SEMAINE database: annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Trans Affect Comput 3(1):5–17. https://doi.org/10.1109/T-AFFC.2011.25
McStay A (2018) Emotional AI: the rise of empathic media. SAGE Publications. https://doi.org/10.4135/9781526451293
Mehrabian A, Russell JA (1974) An approach to environmental psychology. MIT Press
Miner AS, Milstein A, Schueller S, Hegde R, Mangurian C, Linos E (2016) Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med 176(5):619–625
Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T (2019) Model cards for model reporting. In: Proceedings of the conference on fairness, accountability, and transparency, pp 220–229. https://doi.org/10.1145/3287560.3287596
Mittelstadt B (2019) Principles alone cannot guarantee ethical AI. Nat Mach Intell 1:501–507. https://doi.org/10.1038/s42256-019-0114-4
Mohammad SM (2022) Ethics sheet for automatic emotion recognition and sentiment analysis. Comput Linguist 48(2):239–278. https://doi.org/10.1162/coli_a_00433
Mohammad SM, Turney PD (2012) Crowdsourcing a word-emotion association lexicon. Comput Intell. https://doi.org/10.1111/j.1467-8640.2012.00460.x
Mohamed S, Png M-T, Isaac W (2020) Decolonial AI: decolonial theory as sociotechnical foresight in artificial intelligence. Philos Technol 33:659–684. https://doi.org/10.1007/s13347-020-00405-8
Mollahosseini A, Hasani B, Mahoor MH (2019) AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans Affect Comput 10(1):18–31. https://doi.org/10.1109/TAFFC.2017.2740923
Moon A-S, Kim H, Park Y-C, Lee J (2026) A survey on multimodal emotion recognition: methods, datasets, and future directions. Comput Mater Continua 1:1. https://doi.org/10.32604/cmc.2026.076411
Moors A, Ellsworth PC, Scherer KR, Frijda NH (2013) Appraisal theories of emotion: state of the art and future development. Emot Rev 5(2):119–124. https://doi.org/10.1177/1754073912468165
Mori H, Nishino H (2025) End-to-end conversational speech synthesis with controllable emotions in the dimensions of pleasantness and arousal. Acoust Sci Technol 46(1):70–77. https://doi.org/10.1250/ast.e24.13
Munezero MD, Montero CS, Sutinen E, Pajunen J (2014) Are they different? Affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Trans Affect Comput 5(2):101–111. https://doi.org/10.1109/TAFFC.2014.2317187
Nabulsi J (2025) Affective sovereignty: a decolonising politics of emotion in palestine. Rev Int Stud. https://doi.org/10.1017/S0260210525100880
Niedenthal PM (2007) Embodying emotion. Science 316(5827):1002–1005. https://doi.org/10.1126/science.1136930
Norcross JC (2011) Psychotherapy relationships that work: evidence-based responsiveness. In: Psychotherapy relationships that work: evidence-based responsiveness, 2nd edn. Oxford University Press
Northcutt CG, Athalye A, Mueller J (2021) Pervasive label errors in test sets destabilize machine learning benchmarks. In: Proceedings of the 35th international conference on neural information processing systems, track on datasets and benchmarks
Ocumpaugh J, Baker RS, Gowda SM, Heffernan NT, Heffernan C (2014) Population validity for educational data mining models: a case study in affect detection. Br J Edu Technol 45:487–501. https://doi.org/10.1111/bjet.12156
Omarov B, Narynov S, Zhumanov Z (2022) Artificial intelligence-enabled chatbots in mental health: a systematic review. Comput Mater Continua 74(3):5105–5122. https://doi.org/10.32604/cmc.2023.034655
Ong DC (2021) An ethical framework for guiding the development of affectively-aware artificial intelligence. In: Proceedings of the 9th international conference on affective computing and intelligent interaction, pp 1–8. https://doi.org/10.1109/ACII52823.2021.9597441
Oorloff T, Koppisetti S, Bonettini N, Solanki D, Colman B, Yacoob Y, Shahriyari A, Bharaj G (2024) AVFF: audio-visual feature fusion for video deepfake detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2024, pp 27102–27112
Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University Press. https://doi.org/10.1017/CBO9780511571299
Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retr 2(1–2):1–135. https://doi.org/10.1561/1500000011
Park CY, Cha N, Kang S, Kim A, Khandoker AH, Hadjileontiadis L, Oh A, Jeong Y, Lee U (2020) K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations. Sci Data. https://doi.org/10.1038/s41597-020-00630-y
Parra-Gallego LF, Arias-Vergara T, Orozco-Arroyave JR (2025) Multimodal evaluation of customer satisfaction from voicemails using speech and language representations. Dig Signal Process 156(B):104820. https://doi.org/10.1016/j.dsp.2024.104820
Pei G, Haiying L, Lu Y, Wang Y, Hua S, Thihao L (2024) Affective computing: recent advances, challenges, and future trends. Intell Comput. https://doi.org/10.34133/icomputing.0076
Pelachaud C (2009) Studies on gesture expressivity for a virtual agent. Speech Commun 51(7):630–639. https://doi.org/10.1016/j.specom.2008.04.009
Picard RW (1997) Affective computing. MIT Press. https://doi.org/10.7551/mitpress/1140.001.0001
Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191. https://doi.org/10.1109/34.954607
Plutchik R (1980) A general psychoevolutionary theory of emotion. In: Plutchik R, Kellerman H (eds) Theories of emotion, vol 1. Academic Press, pp 3–33. https://doi.org/10.1016/B978-0-12-558701-3.50007-7
Poria S, Cambria E, Bajpai R, Hussain A (2017) A review of affective computing: from unimodal analysis to multimodal fusion. Inf Fusion 37:98–125. https://doi.org/10.1016/j.inffus.2017.02.003
Poria S, Majumder N, Mihalcea R, Hovy E (2019) Emotion recognition in conversation: research challenges, datasets, and recent advances. IEEE Access 7:100943–100953. https://doi.org/10.1109/ACCESS.2019.2929050
Poria S, Hazarika D, Majumder N, Naik G, Cambria E, Mihalcea R (2019b) MELD: a multimodal multi-party dataset for emotion recognition in conversations. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 527–536. https://doi.org/10.18653/v1/P19-1050
Poria S, Majumder N, Hazarika D, Ghosal D, Bhardwaj R, Jian SYB, Hong P, Ghosh R, Roy A, Chhaya N, Gelbukh A, Mihalcea R (2021) Recognizing emotion cause in conversations. Cogn Comput 13:1317–1332. https://doi.org/10.1007/s12559-021-09925-7
Qu C, Che X, Yang Y, Zhang Z, Chang E, Zhang J, Zhu H, Yang L (2025) Enhancing emotion recognition in virtual reality: a multimodal dataset and a temporal emotion detector. Front Psychol. https://doi.org/10.3389/fpsyg.2025.1709943
Rahwan I, Cebrian M, Obradovich N, Bongard J, Bonnefon J-F, Breazeal C, Crandall JW, Christakis NA, Couzin ID, Jackson MO, Jennings NR, Kamar E, Kloumann IM, Larochelle H, Lazer D, McElreath R, Mislove A, Parkes DC, Pentland A, Roberts ME, Shariff A, Tenenbaum JB, Wellman M (2019) Machine behaviour. Nature 568:477–486. https://doi.org/10.1038/s41586-019-1138-y
Rashkin H, Smith EM, Li M, Boureau Y-L (2019) Towards empathetic open-domain conversation models: a new benchmark and dataset. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 5370–5381. https://doi.org/10.18653/v1/P19-1534
Rhue L (2018) Racial influence on automated perceptions of emotions. SSRN working paper. https://doi.org/10.2139/ssrn.3281765
Riek LD (2012) Wizard of Oz studies in HRI: a systematic review and new reporting guidelines. J Hum Robot Interact 1(1):119–136
Ringeval F, Sonderegger A, Sauer J, Lalanne D (2013) Introducing the recola multimodal corpus of remote collaborative and affective interactions. In: Proceedings of the 10th IEEE international conference and workshops on automatic face and gesture recognition, pp 1–8. https://doi.org/10.1109/FG.2013.6553805
Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Xu J, Ott M, Smith EM, Boureau Y-L, Weston J (2021) Recipes for building an open-domain chatbot. In: Proceedings of the 16th conference of the European chapter of the association for computational linguistics, pp 300–325. https://doi.org/10.18653/v1/2021.eacl-main.24
Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 39(6):1161–1178. https://doi.org/10.1037/h0077714
Sabour S, Liu S, Zhang Z, Liu J, Zhou J, Sunaryo A, Lee T, Mihalcea R, Huang M (2024) EmoBench: evaluating the emotional intelligence of large language models. In: Proceedings of the 62nd annual meeting of the association for computational linguistics, pp 5986–6004. https://doi.org/10.18653/v1/2024.acl-long.326
Sambasivan N, Kapania S, Highfill H, Akrong D, Paritosh P, Aroyo LM (2021) “Everyone wants to do the model work, not the data work”: data cascades in high-stakes AI. In: Proceedings of the 2021 CHI conference on human factors in computing systems, pp 1–15. https://doi.org/10.1145/3411764.3445518
Scassellati B, Admoni H, Matarić M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294. https://doi.org/10.1146/annurev-bioeng-071811-150036
Scherer KR (2001) Appraisal considered as a process of multilevel sequential checking. In: Appraisal processes in emotion: theory, methods, research. Oxford University Press, pp 92–120
Scherer KR (2005) What are emotions? And how can they be measured? Soc Sci Inf 44(4):695–729. https://doi.org/10.1177/053901840505821
Scherer KR, Moors A (2019) The emotion process: event appraisal and component differentiation. Annu Rev Psychol 70:719–745. https://doi.org/10.1146/annurev-psych-122216-011854
Schlicher M, Li Y, Murthy SMK, Sun Q, Schuller BW (2025) Emotionally adaptive support: a narrative review of affective computing for mental health. Front Dig Health. https://doi.org/10.3389/fdgth.2025.1657031
Schmidt P, Reiss A, Duerichen R, Marberger C, Van Laerhoven K (2018) Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In: Proceedings of the 20th ACM international conference on multimodal interaction, pp 400–408. https://doi.org/10.1145/3242969.3242985
Searle JR (1980) Minds, brains, and programs. Behav Brain Sci 3(3):417–424. https://doi.org/10.1017/S0140525X00005756
Sharma A, Miner A, Atkins D, Althoff T (2020) A computational approach to understanding empathy expressed in text-based mental health support. In: Proceedings of the 2020 conference on empirical methods in natural language processing, pp 5263–5276. https://doi.org/10.18653/v1/2020.emnlp-main.425
Sharma A, Lin IW, Miner AS, Atkins DC, Althoff T (2021) Towards facilitating empathic conversations in online mental health support: a reinforcement learning approach. In: Proceedings of the web conference 2021. https://doi.org/10.1145/3442381.3450097
Shi Y, Yu K, Dong Y, Chen F (2026) Large language models in education: a systematic review of empirical applications, benefits, and challenges. Comput Educ Artif Intell 10:100529. https://doi.org/10.1016/j.caeai.2025.100529
Shingjergji K, Iren D, Urlings C, Klemke R (2026) Affective computing in online higher education: a systematic literature review. Comput Educ Artif Intell 10:100499. https://doi.org/10.1016/j.caeai.2025.100499
Shum H-Y, He X-D, Li D (2018) From Eliza to Xiaoice: challenges and opportunities with social chatbots. Front Inf Technol Electron Eng 19(1):10–26. https://doi.org/10.1631/FITEE.1700826
Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in an immersive virtual environment. Philos Trans R Soc B 364(1535):3549–3557. https://doi.org/10.1098/rstb.2009.0138
Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642
Sofroniew N, Kauvar I, Saunders W, Chen R, Henighan T, Hydrie S, Citro C, Pearce A, Tarng J, Gurnee W, Batson J, Zimmerman S, Rivoire K, Fish K, Olah C, Lindsey J (2026) Emotion concepts and their function in a large language model. Anthoropic
Somarathna R, Bednarz T, Mohammadi G (2023) Virtual reality for emotion elicitation—a review. IEEE Trans Affect Comput 14(4):2626–2645. https://doi.org/10.1109/TAFFC.2022.3181053
Spitale M, Axelsson M, Gunes H (2024) Appropriateness of LLM-equipped robotic well-being coach language in the workplace: a qualitative pilot study. https://doi.org/10.48550/arXiv.2401.14935 arXiv:2401.14935
Stark L (2018) Algorithmic psychometrics and the scalable subject. Soc Stud Sci. https://doi.org/10.1177/03063127187720
Stark L, Hoey J (2021) The ethics of emotion in artificial intelligence systems. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 782–793. https://doi.org/10.1145/3442188.3445939
Stieglitz S, Mirbabaie M, Ross B, Neuberger C (2018) Social media analytics: challenges in topic discovery, data collection, and data preparation. Int J Inf Manag 39:156–168. https://doi.org/10.1016/j.ijinfomgt.2017.12.002
Tang C, Yu W, Sun G, Chen X, Tan T, Li W, Lu L, Ma Z, Zhang C (2024) SALMONN: towards generic hearing abilities for large language models. In: Proceedings of the international conference on learning representations. https://doi.org/10.48550/arXiv.2310.13289
Tian Y, Huang T, Liu M, Jiang D, Spangher A, Chen M, May J, Peng N (2024a) Are large language models capable of generating human-level narratives? In: Proceedings of the 2024 conference on empirical methods in natural language processing, pp 17659–17681. https://doi.org/10.18653/v1/2024.emnlp-main.978
Tian L, Wang Q, Zhang B, Bo L (2024b) EMO: emote portrait alive generating expressive portrait videos with Audio2Video diffusion model under weak conditions. In: Computer vision—ECCV 2024, pp 244–260. https://doi.org/10.1007/978-3-031-73010-8_15
Troiano E, Oberländer L, Klinger R (2023) Dimensional modeling of emotions in text with appraisal theories: corpus creation, annotation reliability, and prediction. Comput Linguist 49(1):1–72. https://doi.org/10.1162/coli_a_00461
Tsai Y-HH, Bai S, Liang PP, Kolter JZ, Morency L-P, Salakhutdinov R (2019) Multimodal transformer for unaligned multimodal language sequences. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 6558–6569. https://doi.org/10.18653/v1/P19-1656
Turing AM (1950) Computing machinery and intelligence. Mind 49:433–460
Tzirakis P, Trigeorgis G, Nicolaou MA, Schuller BW, Zafeiriou S (2017) End-to-end multimodal emotion recognition using deep neural networks. IEEE J Sel Top Signal Proces 11(8):1301–1309. https://doi.org/10.1109/JSTSP.2017.2764438
Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB (2019) Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry. https://doi.org/10.1177/0706743719828977
Veale M, Borgesius ZF (2021) Demystifying the draft EU Artificial Intelligence Act. Comput Law Rev Int 22(4):97–112. https://doi.org/10.9785/cri-2021-220402
Wang Y, Guo J, Bai J, Yu R, He T, Tan X, Sun X, Bian J (2025) InstructAvatar: text-guided emotion and motion control for avatar generation. In: Proceedings of the AAAI conference on artificial intelligence, vol 39, pp 8132–8140. https://doi.org/10.1609/aaai.v39i8.32877
Wankhade M, Kulkarni C, Rao ACS (2025) A survey on aspect base sentiment analysis methods and challenges. Appl Soft Comput 167(A):112249. https://doi.org/10.1016/j.asoc.2024.112249
Winner L (1980) Do artifacts have politics? Daedalus 109(1):121–136
Witte BD, Reynaert V, Kieken D, Jabbour J, Demarey C, Dumoulin A, Possik J (2026) Immersive virtual reality learning and cognitive load: a multiple-day field study. Comput Hum Behav 176:108853. https://doi.org/10.1016/j.chb.2025.108853
Xia R, Ding Z (2019) Emotion-cause pair extraction: a new task to emotion analysis in texts. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 1003–1012. https://doi.org/10.18653/v1/P19-1096
Xu S, Chen G, Guo Y-X, Yang J, Li C, Zang Z, Zhang Y, Tong X, Guo B (2024) VASA-1: lifelike audio-driven talking faces generated in real time. In: Proceedings of the 38th international conference on neural information processing systems, pp 660–684. https://doi.org/10.5555/3737916.3737937
Yan W-J, Li X, Wang S-J, Zhao G, Liu Y-J, Chen Y-H, Fu X (2014) Casme II: an improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1):86041. https://doi.org/10.1371/journal.pone.0086041
Yan HY, Morrow G, Yang K-C, Wihbey J (2025) The origin of public concerns over AI supercharging misinformation in the 2024 U.S. presidential election. In: Harvard Kennedy School misinformation review. https://doi.org/10.37016/mr-2020-171
Yannakakis GN, Spronck P, Loiacono D, André E (2013) Player modeling. In: Artificial and computational intelligence in games, vol 6, pp 45–59. https://doi.org/10.4230/DFU.Vol6.12191.45
Zadeh A, Zellers R, Pincus E, Morency L-P (2016) Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages (CMU-MOSI). IEEE Intell Syst 31:82–88. https://doi.org/10.1109/MIS.2016.94
Zadeh A, Chen M, Poria S, Cambria E, Morency L-P (2017) Tensor fusion network for multimodal sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1103–1114. https://doi.org/10.18653/v1/D17-1115
Zadeh AB, Liang PP, Poria S, Cambria E, Morency L-P (2018) Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph. In: Proceedings of the 56th annual meeting of the association for computational linguistics. pp 2236–2246. https://doi.org/10.18653/v1/P18-1208
Zhang Y, Wang M, Wu Y, Tiwari P, Li Q, Wang B, Qin J (2024) DialogueLLM: context and emotion knowledge-tuned large language models for emotion recognition in conversations. Neural Netw 192:107901. https://doi.org/10.1016/j.neunet.2025.107901
Zhang W, Deng Y, Liu B, Pan S, Bing L (2024b) Sentiment analysis in the era of large language models: a reality check. In: Findings of the association for computational linguistics: NAACL 2024, pp 3881–3906. https://doi.org/10.18653/v1/2024.findings-naacl.246
Zhang Y, Zhao D, Hancock JT, Kraut R, Yang D (2025) The rise of AI companions: how human-chatbot relationships influence well-being. https://doi.org/10.48550/arXiv.2506.12605
Zhang Y, Yang X, Xu X, Gao Z, Huang Y, Mu S, Feng S, Wang D, Zhang Y, Song K, Yu G (2026) Affective computing in the era of large language models: a survey from the NLP perspective. Knowl-Based Syst 337:115411. https://doi.org/10.1016/j.knosys.2026.115411
Zhao S, Wang S, Soleymani M, Joshi D, Ji Q (2019) Affective computing for large-scale heterogeneous multimedia data: a survey. ACM Trans Multimed Comput Commun Appl 15(93):1–32. https://doi.org/10.1145/3363560
Zhao J, Zhang T, Hu J, Liu Y, Jin Q, Wang X, Li H (2022) M3ED: multi-modal multi-scene multi-label emotional dialogue database. In: Proceedings of the 60th annual meeting of the association for computational linguistics. Association for Computational Linguistics, pp 5699–5710. https://doi.org/10.18653/v1/2022.acl-long.391
Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175
Zheng C, Sabour S, Wen J, Zhang Z, Huang M (2023) AugESC: dialogue augmentation with large language models for emotional support conversation. In: Findings of the association for computational linguistics: ACL 2023. Association for Computational Linguistics, pp 1552–1568. https://doi.org/10.18653/v1/2023.findings-acl.99
Zhou L, Gao J, Li D, Shum H-Y (2018) The design and implementation of XiaoIce, an empathetic social chatbot. Comput Linguist 46(1):53–93. https://doi.org/10.1162/coli_a_00368
Ziems C, Held W, Shaikh O, Chen J, Zhang Z, Yang D (2024) Can large language models transform computational social science? Comput Linguist 50:237–291. https://doi.org/10.1162/coli_a_00502
Zuboff S (2019) The age of surveillance capitalism: the fight for a human future at the new frontier of power. PublicAffairs
Acknowledgements
The author gratefully acknowledges the support of JST SPRING, the Nippon Foundation HUMAI Program, the GMO Internet Foundation, and the Telecommunications Advancement Foundation, which provided the funding and research environment that enabled this work.
Funding
This work was supported in part by JST SPRING under Grant Number JPMJSP2150, the Nippon Foundation HUMAI Program, the GMO Internet Foundation, and the Telecommunications Advancement Foundation.
Author information
Authors and Affiliations
Contributions
The author confirms sole responsibility for the following: study conception and design, methodology, analysis and interpretation of the results, and manuscript preparation.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Use of AI tools
The author used AI-based language tools to support translation and language editing during manuscript preparation. All AI-assisted outputs were reviewed, revised, and verified by the author, who takes full responsibility for the final content of the manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A: Coding rubric for the prior-surveys comparison table
Appendix A: Coding rubric for the prior-surveys comparison table
This appendix documents the rubric used to score each prior survey listed in Table 1 along the two diagnostic axes used in this paper: cross-layer dependence treated as central and prescriptive stance. The purpose of the rubric is not to rank prior surveys but to make the differentiation claim auditable: a future reader who disagrees with the placement of any row should be able to identify which scoring anchor they would invoke instead and why.
1.1 A.1 Scoring axes
Axis 1: Cross-layer dependence treated as central. This axis asks whether the survey treats inter-layer dependence—for instance theory–data, modeling–interaction, technology–ethics, or data–application—as a central analytic object rather than as a residual concern. The three-level scale is:
-
Yes: the survey thematically (at the section or chapter level) frames inter-layer disconnections as the object of analysis, and the structure of the survey is organized around such disconnections rather than around a single layer or a single modality.
-
Partial: the survey discusses two or more layers in interaction (e.g., couples ethics to method, or data to deployment), but the inter-layer disconnection itself is not the principal organizing object; cross-layer concern surfaces locally rather than systemically.
-
No: the survey is organized within a single layer (e.g., data, modeling, modality), reviews multiple layers in parallel without analyzing their dependence, or adopts a bibliometric stance in which layer-level analytic structure is not used.
Axis 2: Prescriptive stance. This axis asks whether the survey advances a prescriptive program that goes beyond taxonomy or empirical taking-stock and is intended to constrain subsequent research practice. The two-level scale used in Table 1 is:
-
Yes: the survey advances a normative program, such as a foundational vision for the field, a prescriptive ethics agenda, or a set of design recommendations intended to constrain how subsequent work should proceed. The recommendation may be confined to one layer (e.g., ethics) or may be field-wide; what matters is that the survey takes a position rather than merely describing the literature.
-
No: the survey is descriptive, taxonomic, or bibliometric in stance. Local recommendations may appear (most surveys close with “future directions”), but the survey does not commit to a programmatic position whose acceptance would reshape research practice.
A finer-grained four-level scale (Strong/Moderate/Weak/None) was considered during rubric development; it was collapsed to the binary Yes/No scale used in Table 1 because the disambiguation required to separate Strong from Moderate could not be applied uniformly across the eleven surveys without introducing reviewer-dependent variance. The collapse is conservative: surveys scored Yes satisfy at least the Moderate threshold, and surveys scored No do not reach the Moderate threshold. The richer scale is documented here for transparency but is not used in the table itself.
1.2 A.2 Inclusion criteria for the comparison set
The comparison set in Table 1 consists of eleven prior surveys selected under the following inclusion rules.
-
1.
Surveys, not primary papers. Only works whose self-stated genre is review, survey, manifesto, or ethics sheet are included. Influential primary papers cited elsewhere in this paper (e.g., Ekman 1992, Russell 1980, Barrett 2017) are not eligible for the differentiation table because their genre is theoretical or empirical contribution rather than survey.
-
2.
Direct relevance to affective computing as a field. Surveys whose object is affective computing, emotion recognition, affect detection, emotion in dialog, or the ethics of emotion AI are included. Surveys of adjacent fields (general HCI, general dialog systems, general AI ethics) are excluded unless their scope explicitly subsumes affective computing.
-
3.
Temporal coverage. The set spans 1997–2025 and is intentionally chosen to include the foundational manifesto (Picard 1997), mature interdisciplinary reviews from the 2010s (Calvo and Mello 2010; Mccoll et al. 2016; Poria et al. 2017, 2019a; Zhao et al. 2019), the inflection point of ethics-explicit work (Mohammad 2022), and the 2024–2025 LLM- and foundation-model-era surveys (Pei et al. 2024; Kumar et al. 2025; Moon et al. 2026; Zhang et al. 2026). The intent is to cover the inflection points of the field rather than to be exhaustive.
-
4.
Representative scope. Where multiple surveys cover essentially the same scope in the same year, the comparison set retains the one most cited at the time of writing or, when citation counts are similar, the one whose framing is closest to the cross-layer concern of this paper. The two Dey et al. 2025 surveys are both retained because they target distinct slices (general multimodal MER vs. LLM-/foundation-model-based MER 2021–2025) and the differentiation between them is itself diagnostic of the field’s current trajectory.
-
5.
Exclusions. Workshop-only proceedings, vendor white papers, and surveys whose published version was not available at the time of writing are excluded. This rubric does not claim that no other survey could have been included; it claims that the eleven included surveys are representative of the comparison categories the differentiation table is designed to span.
1.3 A.3 Disambiguation procedures for borderline cases
Three disambiguation procedures were applied during scoring; they are documented here so that the borderline judgments are auditable.
D1: Foundational vision vs. cross-layer diagnosis. A survey whose mission is to define a new field (e.g., Picard 1997) is necessarily multi-topic: data, models, applications, and ethics are all touched. The rubric does not score such breadth as Yes on Axis 1. Yes on Axis 1 requires that the inter-layer dependence (one layer’s choice constraining another layer’s evaluation) be the analytic object. Foundational manifestos that span layers but do not analyze their dependence as such are scored No on Axis 1 even when they take a strong programmatic position on Axis 2.
D2: Ethics-and-method coupling. A survey whose argument explicitly couples ethical evaluation to methodological design (e.g., Mohammad 2022, whose 50 considerations span task design, data, method, evaluation, and implications) is scored Partial on Axis 1. The coupling is real and intentional, but it does not reach the full cross-layer-as-central threshold: theory–data and modeling–interaction couplings are not the principal analytic objects. We note that the method layer (L3) and the ethics layer (L6) are not adjacent strata in the six-layer pipeline of Sect. 1; an ethics-method coupling is therefore substantively an L3–L6 inter-layer coupling, which is precisely why it justifies a Partial score rather than No, even though it falls short of the systemic cross-layer-as-central threshold required for Yes. Partial is also used when a survey integrates two adjacent layers without taking the overall layered structure as its frame.
D3: Bibliometric and taxonomic surveys. A survey whose primary method is bibliometric (citation analysis, co-occurrence mapping; e.g., Pei et al. (2024)) is scored No on Axis 1 even when its descriptive coverage formally spans every layer. The rationale is that bibliometric methods describe the distribution of attention across topics rather than analyze inter-layer dependence; without an analytic claim about dependence, parallel coverage is not cross-layer dependence in the sense this paper requires.
1.4 A.4 Per-survey scoring rationales
The following entries provide, for each row of Table 1, the scoring rationale on both axes. The scores reproduced below match Table 1 cell-for-cell; any divergence should be treated as an error. Where the rationale relies on disambiguation procedures D1–D3, the procedure is named.
1. Picard (1997). Axis 1: No. Foundational manifesto for the field; layers (data, models, applications, ethics) are surveyed but inter-layer dependence is not the analytic object (D1). Axis 2: Yes (foundational vision, not cross-layer diagnosis). The work issues a programmatic position by defining what affective computing should be. It is scored Yes on Axis 2 for that reason. The qualifier in Table 1 explicitly distinguishes this stance from cross-layer diagnosis.
2. Calvo and Mello (2010). Axis 1: No (layers reviewed in parallel). The survey covers face, voice, body, physiology, EEG, text, and multimodal channels in parallel, alongside emotion theory; coverage is broad, but the dependence among these strata is not made into the analytic object. Axis 2: No. The work is an interdisciplinary review with descriptive intent; it does not advance a prescriptive program in the sense Axis 2 requires.
3. Mccoll et al. (2016). Axis 1: No. The scope is autonomous human affect detection methods within HRI scenarios; the focus is methodological taxonomy within a single application context rather than inter-layer dependence. Axis 2: No. The survey is descriptive, oriented toward HRI practitioners selecting detection methods; no field-wide prescriptive position is advanced.
4. Poria et al. (2017). Axis 1: No (modality-centric). The survey is organized around unimodal (visual, audio, text) analysis and multimodal fusion; modality is the organizing principle, and inter-layer dependence is not foregrounded. Axis 2: No. Comprehensive review aimed at modeling-layer practitioners; the closing remarks discuss future directions but do not amount to a prescriptive program over the field.
5. Poria et al. (2019a). Axis 1: No (task-internal). The scope is the ERC task: research challenges, datasets, and recent neural approaches, such as CMN, ICON, and DialogueRNN. The analysis stays within one task family rather than treating cross-layer dependence as its central analytic object. Axis 2: No. Systematic review of one task; descriptive in stance.
6. Zhao et al. (2019). Axis 1: No. Focus on heterogeneous multimedia (images, music, videos, multimodal) at the modeling and data layers; cross-layer dependence is not the analytic object. Axis 2: No. Modality-and-content-centered survey without a prescriptive program.
7. Mohammad (2022). Axis 1: Partial (ethics–method coupling). The 50 considerations span task design, data, method, evaluation, and implications; this is a deliberate coupling of ethical reasoning to methodological choice (D2). The coupling does not extend to the full theory–data and modeling–interaction couplings that would warrant Yes, but it reaches further than No. Axis 2: Yes (prescriptive ethics agenda). The ethics sheet format itself is a programmatic intervention: it tells the field how ethical reasoning should be incorporated into emotion-recognition and sentiment-analysis research. This is a clear prescriptive program, even though its scope is bounded by the ethics layer.
8. Pei et al. (2024). Axis 1: No (bibliometric, not analytic). A bibliometric review of 33,448 papers from 1997–2023; the method is descriptive distribution analysis (D3). Even where every layer appears in the dataset, parallel coverage is not cross-layer dependence in the sense Axis 1 requires. Axis 2: No. The method does not yield a programmatic position; the contribution is a map of the literature, not a stance on it.
9. Kumar et al. (2025). Axis 1: No. Comprehensive survey of multimodal emotion recognition; modality and architectural lineage are the organizing axes. Axis 2: No. The survey is descriptive of recent technical advances; future-direction remarks do not constitute a prescriptive program.
10. Moon et al. (2026). Axis 1: No. Survey of multimodal emotion recognition 2021–2025 with focus on LLM- and foundation-model-based architectures; concentrates on the modeling layer with attention to architectural taxonomy of LLM-based MER. Axis 2: No. Technical review; descriptive rather than prescriptive in stance.
11. Zhang et al. (2026). Axis 1: No. LLM-era affective computing survey from an NLP perspective covering affect understanding and affect generation tasks, instruction tuning, prompt engineering, and benchmarks; the analysis stays at the modeling and benchmark layers. Axis 2: No. Technical survey oriented toward NLP practitioners; future-direction remarks are local rather than programmatic over the field.
1.5 A.5 Limitations of the rubric
Three limitations of the rubric are recorded here in the interest of fair self-assessment.
First, the scoring was performed by the authors of this paper without an independent second coder; inter-rater reliability (IRR) is therefore not measured. This is a real limitation, and the position-paper genre is not, in itself, an excuse for it. The decision to defer second-coder scoring rests on two grounds: (i) the binary collapse described in Sect. A.1 is intended precisely to reduce the surface on which IRR variance acts, and (ii) the per-survey rationales above expose every disambiguation step so that an independent reader can reproduce the scoring with public information. A future revision of this paper, or a follow-up, could readily fold in second-coder scoring; the rubric is designed to support that.
Second, the inclusion criteria of Sect. A.2 are themselves selection choices, and any selection choice is in principle contestable. The rubric’s defense is not that the eleven surveys are uniquely correct but that the comparison categories they span (foundational manifesto, interdisciplinary review, modality-centric review, task-centric review, ethics sheet, bibliometric review, LLM/foundation-model survey) are the categories against which a position-paper claim of novelty must be checked.
Third, the rubric does not score any survey on the further axis of cascade depth, that is, whether the survey, when it does treat inter-layer dependence, treats the dependence as a single coupling or as a chained cascade of couplings. Adding such an axis would be informative, but it would also collapse onto the position of this paper itself (since this paper is the explicit cascade claim), and the rubric’s purpose here is comparison, not self-promotion. The absence of this axis is therefore a deliberate scope decision rather than an oversight.
The rubric, the per-survey rationales, and the table together support the position-paper claim of Sect. 8 and the prescriptive program of Sect. 9 at the level of evidence: any reader who disagrees with the placement of a row in Table 1 is invited to re-score that row using this rubric and to re-examine, against the new score, whether the cascade diagnosis and the DC1–DC5 program continue to follow.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Inoshita, K. Bridging the silos in affective AI: a critical perspective from data to society. AI & Soc (2026). https://doi.org/10.1007/s00146-026-03324-y
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s00146-026-03324-y