Mirror Neurons cannot be Fooled by Artificial Voices – a study with Implications for Education using Magnetic Resonance Imaging (MRI) and Convolutional Neural Network (CNN)

c4755d77-3c38-4c19-928a-ed5e4abec02d20250730092211027naun:naunmdt@crossref.orgMDT DepositInternational Journal of Education and Information Technologies2074-131610.46300/9109http://www.naun.org/cms.action?id=3037131202513120251910.46300/9109.2025.19https://npublications.com/journals/educationinformation/2025.phpMirror Neurons cannot be Fooled by Artificial Voices – a study with Implications for Education using Magnetic Resonance Imaging (MRI) and Convolutional Neural Network (CNN)Costin AndreiBratanPolitehnica University of Bucharest, RomaniaAndreeaMarinescuUniversity of Medicine and Pharmacy „Carol Davila”, Bucharest, RomaniaElenaTerecoasaUniversity of Medicine and Pharmacy „Carol Davila”, Bucharest, RomaniaAna VoichitaTebeanuPolitehnica University of Bucharest, RomaniaBogdanMorosanuPolitehnica University of Bucharest, RomaniaEduardFrantiResearch Institute for Artificial Intelligence “Mihai Drăgănescu”, Bucharest, RomaniaMonicaDascaluPolitehnica University of Bucharest, RomaniaAlexandraAndreiPolitehnica University of Bucharest, RomaniaClaudiaTocila-MataselMedima Health, Otopeni, RomaniaBogdanIonescuPolitehnica University of Bucharest, RomaniaGheorgheIanaMedima Health, Otopeni, RomaniaAna MariaOproiuUniversity of Medicine and Pharmacy „Carol Davila”, Bucharest, RomaniaGabrielaIorgulescuUniversity of Medicine and Pharmacy „Carol Davila”, Bucharest, RomaniaMirror neurons have a crucial role in detecting and reproducing the actions of others as if the observer himself were performing the specific action. In this paper, four different audio voice files are used to determine, in two methods, the idea that the mirror neurons can be activated only by a real human original voice with a strong emotional load. The first method is the magnetic resonance imaging (MRI), which gives information about the brain activity regarding the specific areas where the mirror neurons are located when the four different audio files are listened to by a group of ten volunteers. The second method implies a deep learning approach, using two convolutional neural network (CNNs) architectures, one used to recognize the timbre of the audio speaker and the second one to determine the level of remnant (residual) emotion in the audio files listened to by them. The four audio files used are an audio text recorded by a Romanian actress with a specific emotion, two different actresses’ voice recordings with the same text and emotion, and with very similar voice features to the main actress, and the last one is an artificially generated voice using AI algorithm. The results show a promising response from both perspectives- the hypothesis that mirror neurons can’t be fooled by an artificial voice is confirmed, and that the intensity of emotion is higher in the original voice than the two imitating voices.7302025730202512012712https://npublications.com/journals/educationinformation/2025/a262008-013(2025).pdf10.46300/9109.2025.19.12https://npublications.com/journals/educationinformation/2025/a262008-013(2025).pdf10.1111/1467-8721.00013Bachorowski, J.-A. (1999). Vocal Expression and Perception of Emotion. Current Directions in Psychological Science, 8(2), 53-57. https://doi.org/10.1111/1467-8721.00013 (Original work published 1999). Abelin, Å., Allwood, J. (2000) Cross linguistic interpretation of emotional prosody. Proceedings ITRW on Speech and Emotion, 110-113. 10.1371/journal.pone.0030278Chen, X., Yang, J., Gan, S., & Yang, Y. (2012). The contribution of sound intensity in vocal emotion perception: behavioral and electrophysiological evidence. PloS one, 7(1), e30278. https://doi.org/10.1371/journal.pone.0030278. 10.1371/journal.pone.0136675Bänziger, T., Hosoya, G., & Scherer, K. R. (2015). Path Models of Vocal Emotion Communication. PloS one, 10(9), e0136675. https://doi.org/10.1371/journal.pone.0136675. 10.3389/fpsyg.2015.01750Mitchell, R. L., & Xu, Y. (2015). What is the Value of Embedding Artificial Emotional Prosody in HumanComputer Interactions? Implications for Theory and Design in Psychological Science. Frontiers in psychology, 6, 1750. https://doi.org/10.3389/fpsyg.2015.01750. Huang, K. L., Duan, S. F., & Lyu, X. (2021). Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model. Frontiers in psychology, 12, 664925. https://doi.org/10.3389/fpsyg.2021.664925. 10.1038/35090060Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature reviews. Neuroscience, 2(9), 661–670. https://doi.org/10.1038/35090060. 10.1093/cercor/bhn181Gazzola, V., & Keysers, C. (2009). The observation and execution of actions share motor and somatosensory voxels in all tested subjects: singlesubject analyses of unsmoothed fMRI data. Cerebral cortex (New York, N.Y.: 1991), 19(6), 1239–1255. https://doi.org/10.1093/cercor/bhn181. 10.1016/j.tics.2011.09.003Gallese, V., & Sinigaglia, C. (2011). What is so special about embodied simulation? Trends in cognitive sciences, 15(11), 512–519. https://doi.org/10.1016/j.tics.2011.09.003. 10.1093/cercor/bhl161Bufalari, I., Aprile, T., Avenanti, A., Di Russo, F., & Aglioti, S. M. (2007). Empathy for pain and touch in the human somatosensory cortex. Cerebral cortex (New York, N.Y.: 1991), 17(11), 2553–2561. https://doi.org/10.1093/cercor/bhl161. 10.1016/j.neuroimage.2008.08.014Nummenmaa, L., Hirvonen, J., Parkkola, R., & Hietanen, J. K. (2008). Is emotional contagion special? An fMRI study on neural systems for affective and cognitive empathy. NeuroImage, 43(3), 571–580. https://doi.org/10.1016/j.neuroimage.2008.08.014. 10.1016/j.tics.2004.01.008Belin, P., Fecteau, S., & Bédard, C. (2004). Thinking the voice: neural correlates of voice perception. Trends in cognitive sciences, 8(3), 129– 135. https://doi.org/10.1016/j.tics.2004.01.008. 10.1016/j.neuroimage.2005.09.059Ethofer, T., Anders, S., Erb, M., Herbert, C., Wiethoff, S., Kissler, J., … Wildgruber, D. (2006). Cerebral pathways in processing of affective prosody: A dynamic causal modeling study. NeuroImage, 30(2), 580-587. https://doi.org/10.1016/j.neuroimage.2005.09.059. 10.1080/17549500902943043Wildgruber, D., Ethofer, T., Grandjean, D., & Kreifelts, B. (2009). A cerebral network model of speech prosody comprehension. International Journal of Speech-Language Pathology, 11(4), 277–281. https://doi.org/10.1080/17549500902943043. 10.1016/j.cortex.2012.08.003Frühholz, S., & Grandjean, D. (2013). Amygdala subregions differentially respond and rapidly adapt to threatening voices. Cortex: a journal devoted to the study of the nervous system and behavior, 49(5), 1394–1403. https://doi.org/10.1016/j.cortex.2012.08.003. 10.1016/j.pneurobio.2014.09.003Frühholz, S., Trost, W., & Grandjean, D. (2014). The role of the medial temporal limbic system in processing emotions in voice and music. Progress in neurobiology, 123, 1–17. https://doi.org/10.1016/j.pneurobio.2014.09.003. 10.1038/nn1392Grandjean, D., Sander, D., Pourtois, G., Schwartz, S., Seghier, M. L., Scherer, K. R., & Vuilleumier, P. (2005). The voices of wrath: brain responses to angry prosody in meaningless speech. Nature neuroscience, 8(2), 145–146. https://doi.org/10.1038/nn1392. 10.1016/j.tics.2005.11.009Schirmer, A., & Kotz, S. A. (2006). Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. Trends in cognitive sciences, 10(1), 24–30. https://doi.org/10.1016/j.tics.2005.11.009. 10.1177/0305735620968260Liu, Y., Tang, Q., Zhao, X., Lu, H., Yuan, G., Wang, G., & Liu, G. (2020). Neural activation of different music styles during emotion-evoking. Psychology of Music, 49(6), 1546-1560. https://doi.org/10.1177/0305735620968260 (Original work published 2021). 10.1038/nrn.2017.72de Waal, F. B. M., & Preston, S. D. (2017). Mammalian empathy: behavioural manifestations and neural basis. Nature reviews. Neuroscience, 18(8), 498–509. https://doi.org/10.1038/nrn.2017.72. 10.1111/j.1749-818x.2010.00267.xKotz, S.A., Paulmann, S. (2011), Emotion, Language, and the Brain. Language and Linguistics Compass, 5: 108-125. https://doi.org/10.1111/j.1749- 818X.2010.00267.x. 10.1016/j.wocn.2009.07.005Pell, M. D., Paulmann, S., Dara, C., Alasseri, A., & Kotz, S. A. (2009). Factors in the recognition of vocally expressed emotions: A comparison of four languages. Journal of Phonetics, 37 (4), 417-435. https://doi.org/10.1016/j.wocn.2009.07.005. 10.1037/a0017088Bänziger, T., Grandjean, D., & Scherer, K. R. (2009). Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). Emotion (Washington, D.C.), 9(5), 691–704. https://doi.org/10.1037/a0017088. 10.1016/j.plrev.2011.10.002Brück, C., Kreifelts, B., & Wildgruber, D. (2011). Emotional voices in context: a neurobiological model of multimodal affective information processing. Physics of life reviews, 8(4), 383–403. https://doi.org/10.1016/j.plrev.2011.10.002. 10.1016/j.pneurobio.2014.09.003Frühholz, S., Trost, W., & Grandjean, D. (2014). The role of the medial temporal limbic system in processing emotions in voice and music. Progress in neurobiology, 123, 1–17. https://doi.org/10.1016/j.pneurobio.2014.09.003. 10.1038/s41598-025-92702-5Zhang, W., Li, J., Ji, L. et al. fNIRS experimental study on the impact of AI-synthesized familiar voices on brain neural responses. Scientific Reports 15, 16872 (2025). https://doi.org/10.1038/s41598-025- 92702-5. 10.1146/annurev-psych-013123-123421Harris L. T. (2024). The Neuroscience of Human and Artificial Intelligence Presence. Annual review of psychology, 75, 433–466. https://doi.org/10.1146/annurev-psych-013123- 123421. 10.1523/jneurosci.3814-11.2012Lee, Y. S., Turkeltaub, P., Granger, R., & Raizada, R. D. (2012). Categorical speech processing in Broca's area: an fMRI study using multivariate pattern-based analysis. The Journal of neuroscience: the official journal of the Society for Neuroscience, 32(11), 3942– 3948. https://doi.org/10.1523/JNEUROSCI.3814- 11.2012. 10.3389/fpsyg.2015.00390Kätsyri, J., Förger, K., Mäkäräinen, M., & Takala, T. (2015). A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness. Frontiers in psychology, 6, 390. https://doi.org/10.3389/fpsyg.2015.00390. 10.1371/journal.pone.0028701Alba-Ferrara, L., Hausmann, M., Mitchell, R. L., & Weis, S. (2011). The neural correlates of emotional prosody comprehension: disentangling simple from complex emotion. PloS one, 6(12), e28701. https://doi.org/10.1371/journal.pone.0028701. Wallenwein, L.A., Schmidt, S.N., Hass, J., & Mier, D. (2024). Cross-modal decoding of emotional expressions in fMRI—Cross-session and cross-sample replication. Imaging Neuroscience, 2, 1-15. https://doi.org/10.1162/imag_a_00289. 10.3389/fnhum.2012.00214Likowski K.U., Mühlberger A, Gerdes A.B.M., Wieser M.J., Pauli P., Weyers P. (2012) Facial mimicry and the mirror neuron system: simultaneous acquisition of facial electromyography and functional magnetic resonance imaging. Frontiers in Human Neuroscience 6:214. https://doi.org/10.3389/fnhum.2012.00214. 10.1080/17470910701391943Oberman, L. M., Winkielman, P., & Ramachandran, V. S. (2007). Face to face: blocking facial mimicry can selectively impair recognition of emotional expressions. Social neuroscience, 2(3-4), 167–178. https://doi.org/10.1080/17470910701391943. 10.1162/jocn.2009.21099Ethofer, T., Kreifelts, B., Wiethoff, S., Wolf, J., Grodd, W., Vuilleumier, P., & Wildgruber, D. (2009). Differential influences of emotion, task, and novelty on brain regions underlying the processing of speech melody. Journal of cognitive neuroscience, 21(7), 1255–1268. https://doi.org/10.1162/jocn.2009.21099. 10.1371/journal.pbio.0030079Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C., & Rizzolatti, G. (2005). Grasping the intentions of others with one's own mirror neuron system. PLoS biology, 3(3), e79. https://doi.org/10.1371/journal.pbio.0030079. Mara, D. (2017). The function of mirror neurons in the learning process. 8 th International Conference on Manufacturing Science and Education – MSE 2017 “Trends in New Industrial Revolution”, 12012, https://doi.org/10.1051/matecconf/201712112012. 10.1109/mass62177.2024.00066Choksi, K., Chen, H., Joshi, K., Jade, S., Nirjon, S., & Lin, S. (2024). SensEmo: Enabling affective learning through real-time emotion recognition with smartwatches. arXiv. https://doi.org/10.48550/arXiv.2407.09911. Sun, F., Li, L., Meng, S., Teng, X., Payne, T., & Craig, P. (2025). Integrating emotional intelligence, memory architecture, and gestures to achieve empathetic humanoid robot interaction in an educational setting. arXiv. https://doi.org/10.48550/arXiv.2505.19803. 10.1177/1046878111432108Bachen, C. M., Hernández-Ramos, P. F., & Raphael, C. (2012). Simulating REAL LIVES: Promoting global empathy and interest in learning through simulation games. Simulation & Gaming, 43(4), 437– 460. https://doi.org/10.1177/1046878111432108. 10.1007/978-3-030-29876-0_2Trieu, M., Foster, A.E., Yaseen, Z.S., Beaubian, C., Calati, R. (2019). Neurobiology of Empathy. In: Foster, A.E., Yaseen, Z.S. (eds) Teaching Empathy in Healthcare. Springer, Cham. https://doi.org/10.1007/978-3-030-29876-0_2. 10.4103/0976-9668.101878Acharya, S., & Shukla, S. (2012). Mirror neurons: Enigma of the metaphysical modular brain. Journal of natural science, biology, and medicine, 3(2), 118– 124. https://doi.org/10.4103/0976-9668.101878.