Artificial violence: violence against women and the dark side of artificial intelligence
DOI:
https://doi.org/10.11606/issn.2316-9125.v30i1p181-197Keywords:
artificial intelligence , cyberviolence, Replika, AI literacy, gender education, violence against womenAbstract
Violence against women is increasingly becoming a global phenomenon, involving millions of people worldwide, reinforcing the condition of social inequality that has historically relegated women to a subordinate position, within a framework of inequality and discrimination. The development of digital technologies (Web 2.0, social media, digital platforms) and, more recently, artificial intelligence, is contributing to the emergence of new, sometimes more insidious than traditional ones, forms of violence, raising questions about how to combat them. Specifically, the article proposes a theoretical reflection on the new forms of digital violence enabled by technology, with a focus on the case of the chatbot Replika. The qualitative and quantitative expansion of violence poses new media-education challenges, particularly regarding the need to design and propose new forms of prevention suited to this new landscape and to strengthen specific AI Literacy.
Downloads
References
AUFDERHEIDE, Patricia. Media Literacy: A Report of the National Leadership Conference on Media Literacy. Aspen: Aspen Institute, 1993.
BACKE, Emma; LILLESTON, Pamela; MCCLEARY-SILLS, Jennifer. Networked individuals, gendered violence: A literature review of cyberviolence. Violence and gender, v. 5, n. 3, p. 135-146, 2018. https://doi.org/10.1089/vio.2017.0056
BELLUATI, Marinella. Femminicidio: Una lettura tra realtà e rappresentazione. Roma: Carocci, 2021.
BELLUATI, Marinella; TIROCCHI, Simona. Sociologia della comunicazione e degli ambienti mediali. Milano: Pearson, 2023.
BEAUVOIR, Simone de. The second sex. New York: Bantam Books, 1952.
BOCCIA ARTIERI, Giovanni et al. Fenomenologia dei social network. Presenza, relazioni e consumi mediali degli italiani online. Milano: Guerini, 2017.
BOURDIEU, Pierre. Masculine domination. Cambridge: Polity, 2001.
CANNITO, Maddalena et al. Fare maschilità online: definire e indagare la manosphere. AG-ABOUT GENDER, v. 10, n. 19, p. 1-41, 2021. https://doi.org/10.15167/2279-5057/AG2021.10.19.1326
CHO, Hyunyi et al. Social media literacy: A conceptual framework. New media & society, v. 26, n. 2, p. 941-960, 2024. https://doi.org/10.1177/14614448211068530.
COLE, Samantha. ‘My AI is sexually harassing me’: Replika users say the chatbot has gotten way too horny. Vice, 12 jan. 2023. Disponível em: https://www.vice.com/en/article/my-ai-is-sexually-harassing-me-replika-chatbot-nudes/. Acesso em: 15 jan. 2025.
CORBISIERO, Fabio; NOCENZI, Mariella. Manuale di educazione al genere e alla sessualità. Torino: Utet, 2022.
CRENSHAW, Kimberle. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, n. 1, 1989. Disponível em: https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1052&context=uclf. Acesso em: 15 jan. 2025.
DEPOUNTI, Iliana; SAUKKO, Paula; NATALE, Simone. Ideal technologies, ideal women: AI and gender imaginaries in Redditors’ discussions on the Replika bot girlfriend. Media, Culture & Society, v. 45, n. 4, p. 720-736, 2023. https://doi.org/10.1177/01634437221119021
DODA, Irene. Il chatbot Replika ha iniziato a molestare sessualmente gli utenti. Nato come un “amico virtuale”, a volte ha oltrepassato i limiti. Wired, 13 jan. 2023. Disponível em: https://www.wired.it/article/chatbot-replika-molestie-sessuali/. Acesso em: 15 jan. 2025.
DUNN, Suzie. Technology-facilitated gender-based violence: An overview. Centre for International Governance Innovation: Supporting a Safer Internet Paper No 1, 2020. Disponível em: https://www.cigionline.org/static/documents/SaferInternet_Paper_no_1_coverupdate.pdf. Acesso em: 15 jan. 2025.
EUROPEAN INSTITUTE FOR GENDER EQUALITY. Combating Cyber Violence against Women and Girls, 2022. Disponível em: https://eige.europa.eu/sites/default/files/documents/combating_cyber_violence_against_women_and_girls.pdf. Acesso em: 15 jan. 2025.
EUROPEAN COMMISSION. Advisory Committee on Equal Opportunities for Women and Men. Opinion on combating online violence against women, 2020. Disponível em: https://commission.europa.eu/publications/opinions-advisory-committee-equal-opportunities-women-and-men_en. Acesso em: 15 jan. 2025.
EUROPEAN COMMISSION. EU survey on gender-based violence against woman and other forms of inter-personal violence (EU-GBV) - first results. Eurostat, 2022. Disponível em: https://ec.europa.eu/eurostat/documents/7870049/15323622/KS-FT-22-005-EN-N.pdf/315d443b-ba8d-e607-3ce0-845f642a8c00?version=1.0&t=1669371271599. Acesso em: 15 jan. 2025.
FARCI, Manolo; RIGHETTI, Nicola. Italian Men’s Rights Activism and the Online Backlash Against Feminism. Rassegna Italiana di Sociologia, v. 4, p. 765-781, 2019. https://doi.org/10.1423/96115
FARCI, Manolo; SCARCELLI, Cosimo Marco. Media digitali, genere e sessualità. Milano: Mondadori, 2021.
FARCI, Manolo; RICCI, Oscar. I Do Not Believe that Talking About this Kind of Stuff is a Way to Diminish Feminist Battles: An Online Controversy in the Italian Manosphere. In: GRAHAM, Roderick et al. (org.). The Routledge International Handbook of Online Deviance. London: Routledge, 2025. p. 321-338.
FLORIDI, Luciano. The Onlife Manifesto: Being Human in a Hyperconnected Era. Springer Nature: New York, 2015.
GARCÍA-ULL, Francisco José; MELERO-LÁZARO, Mónica. Gender stereotypes in AI-generated images. Profesional de la información, v. 32, n. 5, 2023. https://doi.org/10.3145/epi.2023.sep.05
GENGLER, Eva Johanna. Sexism, racism, and classism: social biases in text-to-image generative AI in the context of power, success, and beauty. Wirtschaftsinformatik 2024 Proceedings, v. 48, 2024. Disponível em: https://aisel.aisnet.org/wi2024/48. Acesso em: 15 jan. 2025.
GILL, Rosalind. Gender and the Media. London: Polity, 2007.
GIUS, Chiara. (Re)thinking gender in cyber-violence. Insights from awareness-raising campaigns on online violence against women and girls in Italy. Media Education, v. 14, n. 2, p. 95-106, 2023. https://doi.org/10.36253/me-14896
HEPP, Andreas. Deep mediatization. London: Routledge, 2020.
HONG, Joo-Wha; CHOI, Sukyoung; WILLIAMS, Dmitri. Sexist AI: an experiment integrating CASA and ELM. International Journal of Human–Computer Interaction, v. 36, n. 20, p. 1928-1941, 2020. https://doi.org/10.1080/10447318.2020.1801226
HORVÁT, Emoke-Ágnes; GONZÁLEZ-BAILÓN, Sandra. Quantifying gender disparities and bias online: editors’ introduction to “Gender Gaps in Digital Spaces”. Journal of Computer-Mediated Communication, v. 29, n. 1, zmad054, 2024. https://doi.org/10.1093/jcmc/zmad054
JANDRIĆ, Petar et al. Postdigital science and education. Educational Philosophy and Theory, v. 50, n. 10, p. 893-899, 2018. https://doi.org/10.1080/00131857.2018.1454000
JIANG, Jialei; VETTER, Matthew A.; LUCIA, Brent. Toward a ‘More-Than-Digital’AI Literacy: Reimagining Agency and Authorship in the Postdigital Era with ChatGPT. Postdigital Science and Education, v. 6, n. 3, p. 922-939, 2024. https://doi.org/10.1007/s42438-024-00477-1
LAESTADIUS, Linnea et al. Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika. New Media & Society, v. 26, n. 10, 2022. https://doi.org/10.1177/14614448221142007.
LALLI, Pina di. L’amore non uccide. Femminicidio e discorso pubblico: cronaca, tribunali, politiche. Bologna: il Mulino, 2021.
LAMENSCH, Marie. Generative AI Tools Are Perpetuating Harmful Gender Stereotypes. These new systems reflect the inequitable, racist and sexist biases of their source material. CIGI, 14 jun. 2023. Disponível em: https://www.cigionline.org/articles/generative-ai-tools-are-perpetuating-harmful-gender-stereotypes/. Acesso em: 15 jan. 2025.
LIVINGSTONE, Sonia. Media literacy and the challenge of new information and communication technologies. Communication Review, v. 7, n. 1, p. 3-14, 2004. https://doi.org/10.1080/10714420490280152
LONG, Duri; MAGERKO, Brian. What is AI literacy? Competencies and design considerations. In: Proceedings of the 2020 CHI conference on human factors in computing systems, abr. 2020. p. 1-16.
MARWICK, Alice E., CAPLAN, Robyn. Drinking male tears: Language, the manosphere, and networked harassment. Feminist Media Studies, v. 18, n. 4, p. 543-559, 2018. https://doi.org/10.1080/14680777.2018.1450568
NATALE, Simone; DEPOUNTI, Iliana. Artificial sociality. Human-Machine Communication Journal, v. 7, n. 5, p. 83-98, 2024. Disponível em: https://stars.library.ucf.edu/hmc/vol7/iss1/5/. Acesso em: 15 jan. 2025.
NG, Davy Tsz Kit et al. Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, v. 2, 100041, 2021. https://doi.org/10.1016/j.caeai.2021.100041
PENTINA, Iryna; HANCOCK, Tyler; XIE, Tianling. Exploring relationship development with social chatbots: A mixed-method study of replika. Computers in Human Behavior, v. 140, 107600, 2023. https://doi.org/10.1016/j.chb.2022.107600
POTTER, James W. Seven skills of Media literacy. Thousand Oaks: Sage, 2019.
RICAURTE, Paola. Ethics for the majority world: AI and the question of violence at scale. Media, Culture & Society, v. 44, n. 4, p. 726-745, 2022. https://doi.org/10.1177/01634437221099
SIMONOVIC, Dubravka. Report of the special rapporteur on violence against women, its causes and consequences on online violence against women and girls from a human rights perspective. Geneva: United Nations, 2020. Disponível em: https://digitallibrary.un.org/record/1641160. Acesso em: 15 jan. 2025.
SKJUVE, Marita et al. My Chatbot companion: a study of humanchatbot relationships. International Journal of Human-Computer Studies, v. 149, 102601, 2021. https://doi.org/10.1016/j.ijhcs.2021.102601
SONG, Xia; XU, Bo; ZHAO, Zhenzhen. Can people experience romantic love for artificial intelligence? An empirical study of intelligent assistants. Information & Management, v. 59, n. 2, 103595, 2022. https://doi.org/10.1016/j.im.2022.103595
TADDEO, Gabriella. Artificial intelligence literacy: aspetti sociali e educative di una nuova frontiera dell’educazione. In: RICUCCI, Roberta; ROSA, Alessia. (org.). Didattica per competenze e orizzonti educativi. La prospettiva Lifelong Learning. Lecce: Pensa Multimedia, 2024. p. 105-122.
TIROCCHI, Simona. Digital education. Dalla scuola digitale all’intelligenza artificiale. DigitCult: Scientific Journal on Digital Cultures, v. 8, n. 2, p. 75-89, 2024. https://doi.org/10.36158/97888929589205
TIROCCHI, Simona. Más allá del “hype”. Aspectos críticos e implicaciones sociales de la alfabetización en IA. In: DEFENSORÍA DEL PÚBLICO DE ARGENTINA (org.). Alfabetización Mediática e Informacional en la era de la IA. forthcoming.
TIROCCHI, Simona; SCOCCO, Marta; CRESPI, Isabella. Generation Z and cyberviolence: between digital platforms use and risk awareness. International Review of Sociology, v. 32, n. 3, p. 443-462, 2022. https://doi.org/10.1080/03906701.2022.2133408
UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION. L’etica dell’IA: Modellare il future delle nostre società. Roma: Unesco, 2021. Disponível em: https://www.unesco.it/wp-content/uploads/2023/11/Brochure-su-Raccomandazione-UNESCO-sullIntelligenza-Artificiale-1.pdf. Acesso em: 15 jan. 2025.
WHAT is gender-based violence? European Commission, 2024. Disponível em: https://commission.europa.eu/strategy-and-policy/policies/justice-and-fundamental-rights/gender-equality/gender-based-violence/what-gender-based-violence_en. Acesso em: 15 jan. 2025.
WORLD HEALTH ORGANIZATION. Violence against women, 2024. Disponível em: https://www.who.int/news-room/fact-sheets/detail/violence-against-women. Acesso em: 15 jan. 2025.
XIE, Tianling; PENTINA, Iryna. Attachment theory as a framework to understand relationships with social chatbots: a case study of Replika. In: Proceedings of the 55th Hawaii international conference on system sciences, 2022.
YI, Yumi. Establishing the concept of AI literacy: Focusing on competence and purpose. JAHR European Journal of Bioethics, v. 12, n. 2, p. 353-368, 2021. https://doi.org/10.21860/j.12.2.8
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Simona Tirocchi
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
I authorize the publication of the submitted article and soon the copyrights to the magazine, in the printed and electronic version, if it is approved after the evaluation of the reviewers.
I understand that readers may use this article without prior request, provided the source and authorship are mentioned. Readers are not authorized to use this article for reproduction, in whole or in part, for commercial purposes.
