Findings of the Association for Computational Linguistics: EACL 2023
In this paper, we focus on the topics of misinformation and racial hoaxes from a perspective deri... more In this paper, we focus on the topics of misinformation and racial hoaxes from a perspective derived from both social psychology and computational linguistics. In particular, we consider the specific case of antiimmigrant feeling as a first case study for addressing racial stereotypes. We describe the first corpus-based study for multilingual racial stereotype identification in social media conversational threads. Our contributions are: (i) a multilingual corpus of racial hoaxes, (ii) a set of common guidelines for the annotation of racial stereotypes in social media texts, and a multi-layered, fine-grained scheme, psychologically grounded on the work by Fiske et al., including not only stereotype presence, but also contextuality, implicitness, and forms of discredit, (iii) a multilingual dataset in Italian, Spanish, and French annotated following the aforementioned guidelines, and crosslingual comparative analyses taking into account racial hoaxes and stereotypes in online discussions. The analysis and results show the usefulness of our methodology and resources, shedding light on how racial hoaxes are spread, and enable the identification of negative stereotypes that reinforce them.
Despite the large number of computational resources for emotion recognition, there is a lack of d... more Despite the large number of computational resources for emotion recognition, there is a lack of data sets relying on appraisal models. According to Appraisal theories, emotions are the outcome of a multi-dimensional evaluation of events. In this paper, we present APPReddit, the first corpus of non-experimental data annotated according to this theory. After describing its development, we compare our resource with enISEAR, a corpus of events created in an experimental setting and annotated for appraisal. Results show that the two corpora can be mapped notwithstanding different typologies of data and annotations schemes. A SVM model trained on APPReddit predicts four appraisal dimensions without significant loss. Merging both corpora in a single training set increases the prediction of 3 out of 4 dimensions. Such findings pave the way to a better performing classification model for appraisal prediction.
Inside the NLP community there is a considerable amount of language resources created, annotated ... more Inside the NLP community there is a considerable amount of language resources created, annotated and released every day with the aim of studying specific linguistic phenomena. Despite a variety of attempts in order to organize such resources has been carried on, a lack of systematic methods and of possible interoperability between resources are still present. Furthermore, when storing linguistic information, still nowadays, the most common practice is the concept of "gold standard", which is in contrast with recent trends in NLP that aim at stressing the importance of different subjectivities and points of view when training machine learning and deep learning methods. In this paper we present O-Dang!: The Ontology of Dangerous Speech Messages, a systematic and interoperable Knowledge Graph (KG) for the collection of linguistic annotated data. O-Dang! is designed to gather and organize Italian datasets into a structured KG, according to the principles shared within the Linguistic Linked Open Data community. The ontology has also been designed to account a perspectivist approach, since it provides a model for encoding both gold standard and single-annotator labels in the KG. The paper is structured as follows. In Section 1. the motivations of our work are outlined. Section 2. describes the O-Dang! Ontology, that provides a common semantic model for the integration of datasets in the KG. The Ontology Population stage with information about corpora, users, and annotations is presented in Section 3.. Finally, in Section 4. an analysis of offensiveness across corpora is provided as a first case study for the resource.
Abusive language is becoming a problematic issue for our society. The spread of messages that rei... more Abusive language is becoming a problematic issue for our society. The spread of messages that reinforce social and cultural intolerance could have dangerous effects in victims’ life. State-of-the-art technologies are often effective on detecting explicit forms of abuse, leaving unidentified the utterances with very weak offensive language but a strong hurtful effect. Scholars have advanced theoretical and qualitative observations on specific indirect forms of abusive language that make it hard to be recognized automatically. In this work, we propose a battery of statistical and computational analyses able to support these considerations, with a focus on creative and cognitive aspects of the implicitness, in texts coming from different sources such as social media and news. We experiment with transformers, multi-task learning technique, and a set of linguistic features to reveal the elements involved in the implicit and explicit manifestations of abuses, providing a solid basis for c...
and all the group of research in Turin who supported and encouraged my investigations, believing ... more and all the group of research in Turin who supported and encouraged my investigations, believing in me. And finally, I would like to thank all my colleagues for everything that they taught me and for everything that they will teach me!
Important issues, such as abortion governmental laws, are discussed everyday online involving dif... more Important issues, such as abortion governmental laws, are discussed everyday online involving different opinions that could be favorable or not. Often the debates change tone and become more aggressive undermining the discussion. In this paper, we analyze the relation between abusive language and the stances of disapproval toward some controversial issues that involve specific groups of people (such as women), which are commonly also targets of hate speech. We analyzed the tweets about the feminist movement and the legalization of abortion events released by the organizers of Stance Detection shared task at SemEval 2016. An interesting finding is the usefulness of semantic and lexical features related to misogynistic and sexist speech which improve considerably the sensitivity of the system of stance classification toward the feminist movement. About the abortion issue, we found that the majority of the expressions relevant for the classification are negative and aggressive. The improvements in terms of precision, recall and f-score are confirmed by the analysis of the correct predicted unfavorable tweets, which are featured by expressions of hatred against women. The promising results obtained in this initial study demonstrate indeed that disapproval is often expressed using abusive language. It suggests that the monitoring of hate speech and abusive language during the stance detection process could be exploited to improve the quality of the debates in social media.
The possibility of raising awareness about misbehaviour online, such as hate speech, especially i... more The possibility of raising awareness about misbehaviour online, such as hate speech, especially in young generations could help society to reduce their impact, and thus, their consequences. The Computer Science Department of the University of Turin has designed various technologies that support educational projects and activities in this perspective. We implemented an annotation platform for Italian tweets employed in a laboratory called #DEACTIVHATE, specifically designed for secondary school students. The laboratory aims at countering hateful phenomena online and making students aware of technologies that they use on a daily basis. We describe our teaching experience in high schools and the usefulness of the technologies and activities tested.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. IronITA is a new shared task in the EVALITA 2018 evaluation campaign, focused on the aut... more English. IronITA is a new shared task in the EVALITA 2018 evaluation campaign, focused on the automatic classification of irony in Italian texts from Twitter. It includes two tasks: 1) irony detection and 2) detection of different types of irony, with a special focus on sarcasm identification. We received 17 submissions for the first task and 7 submissions for the second task from 7 teams. Italiano. IronITA è un nuovo esercizio di valutazione della campagna di valutazione EVALITA 2018, specificamente dedicato alla classificazione automatica dell'ironia presente in testi estratti da Twitter. Comprende due task: 1) riconoscimento dell'ironia e 2) riconoscimento di diversi tipi di ironia, con particolare attenzione all'identificazione del sarcasmo. Abbiamo ricevuto 17 sottomissioni per il primo task e 7 per il secondo, da parte di 7 gruppi partecipanti.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. The automatic misogyny identification (AMI) task proposed at IberEval and EVALITA 2018 i... more English. The automatic misogyny identification (AMI) task proposed at IberEval and EVALITA 2018 is an example of the active involvement of scientific Research to face up the online spread of hate contents against women. Considering the encouraging results obtained for Spanish and English in the precedent edition of AMI, in the EVALITA framework we tested the robustness of a similar approach based on topic and stylistic information on a new collection of Italian and English tweets. Moreover, to deal with the dynamism of the language on social platforms, we also propose an approach based on automatically-enriched lexica. Despite resources like the lexica prove to be useful for a specific domain like misogyny, the analysis of the results reveals the limitations of the proposed approaches. lessici risultano utili per domini specifici come quello della misoginia, analizzando i risultati emergono i limiti degli approcci proposti.
EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020, 2020
The Hate Speech Detection (HaSpeeDe 2) task is the second edition of a shared task on the detecti... more The Hate Speech Detection (HaSpeeDe 2) task is the second edition of a shared task on the detection of hateful content in Italian Twitter messages. HaSpeeDe 2 is composed of a Main task (hate speech detection) and two Pilot tasks, (stereotype and nominal utterance detection). Systems were challenged along two dimensions: (i) time, with test data coming from a different time period than the training data, and (ii) domain, with test data coming from the news domain (i.e., news headlines). Overall, 14 teams participated in the Main task, the best systems achieved a macro F1-score of 0.8088 and 0.7744 on the indomain in the out-of-domain test sets, respectively; 6 teams submitted their results for Pilot task 1 (stereotype detection), the best systems achieved a macro F1-score of 0.7719 and 0.7203 on in-domain and outof-domain test sets. We did not receive any submission for Pilot task 2.
English. In the domain of Natural Language Processing (NLP), the interest in figurative language ... more English. In the domain of Natural Language Processing (NLP), the interest in figurative language is enhanced, especially in the last few years, thanks to the amount of linguistic data provided by web and social networks. Figurative language provides a non-literary sense to the words, thus the utterances require several interpretations disclosing the play of signification. In order to individuate different meaning levels in case of ironic texts detection, it is necessary a computational model appropriated to the complexity of rhetorical artifice. In this paper we describe our rulebased system of irony detection as it has been presented to the SENTIPOLC task of EVALITA 2016, where we ranked third on twelve participants. Italiano. Nell’ambito del Natural Language Processing (NLP) l’interesse per il linguaggio figurativo è particolarmente aumentato negli ultimi anni, grazie alla quantità d’informazione linguistica messa a disposizione dal web e dai social network. Il linguaggio figurati...
Nowadays, misogynistic abuse online has become a serious issue due, especially, to anonymity and ... more Nowadays, misogynistic abuse online has become a serious issue due, especially, to anonymity and interactivity of the web that facilitate the increase and the permanence of the offensive comments on the web. In this paper, we present an approach based on stylistic and specific topic information for the detection of misogyny, exploring the several aspects of misogynistic Spanish and English user generated texts on Twitter. Our method has been evaluated in the framework of our participation in the AMI shared task at IberEval 2018 obtaining promising results.
The importance of the detection of aggressiveness in social media is due to real effects of viole... more The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior online. For this reason, hate speech online is a real problem in modern society and the necessity of control of usergenerated contents has become one of the priorities for governments, social media platforms and Internet companies. Current methodologies are far from solving this problem. Indeed, several aggressive comments are also disguised as sarcastic. In this perspective, this research proposal wants to investigate the role played by creative linguistic devices, especially sarcasm, in hate speech in multilingual context.
The importance of the detection of aggressiveness in social media is due to real effects of viole... more The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior online. Indeed, this kind of legal cases are increasing in the last years. For this reason, the necessity of controlling user-generated contents has become one of the priorities for many Internet companies, although current methodologies are far from solving this problem. Therefore, in this work we propose an innovative approach that combines deep learning framework with linguistic features specific for this issue. This approach has been evaluated and compared with other ones in the framework of the MEX-A3T shared task at IberEval on aggressiveness analysis in Spanish Mexican tweets. In spite of our novel approach, we obtained low results.
The lack of understanding of figurative language online, such as ironic messages, is a common cau... more The lack of understanding of figurative language online, such as ironic messages, is a common cause of error for systems that analyze automatically the users’ opinions online detecting sentiment, emotions or stance. In order to deal with this problem of automatic processing of natural language, IroSvA shared task at IberLef 2019 asks participants to detect, for the first time, irony in short texts written in Spanish language, considering the three linguistic variants from Spain, Mexico and Cuba. Another novelty of this task is the presence of labels specifying the context of the utterance, such as current political or social issues discussed online. In the context of this shared task, we approached irony detection in Spanish short texts trying to exploit the provided topic information. In addition, we investigated the usefulness of stylistic, lexical and affective features during the development of the irony detection models for the three Spanish variants. Experimental results and f...
Findings of the Association for Computational Linguistics: EACL 2023
In this paper, we focus on the topics of misinformation and racial hoaxes from a perspective deri... more In this paper, we focus on the topics of misinformation and racial hoaxes from a perspective derived from both social psychology and computational linguistics. In particular, we consider the specific case of antiimmigrant feeling as a first case study for addressing racial stereotypes. We describe the first corpus-based study for multilingual racial stereotype identification in social media conversational threads. Our contributions are: (i) a multilingual corpus of racial hoaxes, (ii) a set of common guidelines for the annotation of racial stereotypes in social media texts, and a multi-layered, fine-grained scheme, psychologically grounded on the work by Fiske et al., including not only stereotype presence, but also contextuality, implicitness, and forms of discredit, (iii) a multilingual dataset in Italian, Spanish, and French annotated following the aforementioned guidelines, and crosslingual comparative analyses taking into account racial hoaxes and stereotypes in online discussions. The analysis and results show the usefulness of our methodology and resources, shedding light on how racial hoaxes are spread, and enable the identification of negative stereotypes that reinforce them.
Despite the large number of computational resources for emotion recognition, there is a lack of d... more Despite the large number of computational resources for emotion recognition, there is a lack of data sets relying on appraisal models. According to Appraisal theories, emotions are the outcome of a multi-dimensional evaluation of events. In this paper, we present APPReddit, the first corpus of non-experimental data annotated according to this theory. After describing its development, we compare our resource with enISEAR, a corpus of events created in an experimental setting and annotated for appraisal. Results show that the two corpora can be mapped notwithstanding different typologies of data and annotations schemes. A SVM model trained on APPReddit predicts four appraisal dimensions without significant loss. Merging both corpora in a single training set increases the prediction of 3 out of 4 dimensions. Such findings pave the way to a better performing classification model for appraisal prediction.
Inside the NLP community there is a considerable amount of language resources created, annotated ... more Inside the NLP community there is a considerable amount of language resources created, annotated and released every day with the aim of studying specific linguistic phenomena. Despite a variety of attempts in order to organize such resources has been carried on, a lack of systematic methods and of possible interoperability between resources are still present. Furthermore, when storing linguistic information, still nowadays, the most common practice is the concept of "gold standard", which is in contrast with recent trends in NLP that aim at stressing the importance of different subjectivities and points of view when training machine learning and deep learning methods. In this paper we present O-Dang!: The Ontology of Dangerous Speech Messages, a systematic and interoperable Knowledge Graph (KG) for the collection of linguistic annotated data. O-Dang! is designed to gather and organize Italian datasets into a structured KG, according to the principles shared within the Linguistic Linked Open Data community. The ontology has also been designed to account a perspectivist approach, since it provides a model for encoding both gold standard and single-annotator labels in the KG. The paper is structured as follows. In Section 1. the motivations of our work are outlined. Section 2. describes the O-Dang! Ontology, that provides a common semantic model for the integration of datasets in the KG. The Ontology Population stage with information about corpora, users, and annotations is presented in Section 3.. Finally, in Section 4. an analysis of offensiveness across corpora is provided as a first case study for the resource.
Abusive language is becoming a problematic issue for our society. The spread of messages that rei... more Abusive language is becoming a problematic issue for our society. The spread of messages that reinforce social and cultural intolerance could have dangerous effects in victims’ life. State-of-the-art technologies are often effective on detecting explicit forms of abuse, leaving unidentified the utterances with very weak offensive language but a strong hurtful effect. Scholars have advanced theoretical and qualitative observations on specific indirect forms of abusive language that make it hard to be recognized automatically. In this work, we propose a battery of statistical and computational analyses able to support these considerations, with a focus on creative and cognitive aspects of the implicitness, in texts coming from different sources such as social media and news. We experiment with transformers, multi-task learning technique, and a set of linguistic features to reveal the elements involved in the implicit and explicit manifestations of abuses, providing a solid basis for c...
and all the group of research in Turin who supported and encouraged my investigations, believing ... more and all the group of research in Turin who supported and encouraged my investigations, believing in me. And finally, I would like to thank all my colleagues for everything that they taught me and for everything that they will teach me!
Important issues, such as abortion governmental laws, are discussed everyday online involving dif... more Important issues, such as abortion governmental laws, are discussed everyday online involving different opinions that could be favorable or not. Often the debates change tone and become more aggressive undermining the discussion. In this paper, we analyze the relation between abusive language and the stances of disapproval toward some controversial issues that involve specific groups of people (such as women), which are commonly also targets of hate speech. We analyzed the tweets about the feminist movement and the legalization of abortion events released by the organizers of Stance Detection shared task at SemEval 2016. An interesting finding is the usefulness of semantic and lexical features related to misogynistic and sexist speech which improve considerably the sensitivity of the system of stance classification toward the feminist movement. About the abortion issue, we found that the majority of the expressions relevant for the classification are negative and aggressive. The improvements in terms of precision, recall and f-score are confirmed by the analysis of the correct predicted unfavorable tweets, which are featured by expressions of hatred against women. The promising results obtained in this initial study demonstrate indeed that disapproval is often expressed using abusive language. It suggests that the monitoring of hate speech and abusive language during the stance detection process could be exploited to improve the quality of the debates in social media.
The possibility of raising awareness about misbehaviour online, such as hate speech, especially i... more The possibility of raising awareness about misbehaviour online, such as hate speech, especially in young generations could help society to reduce their impact, and thus, their consequences. The Computer Science Department of the University of Turin has designed various technologies that support educational projects and activities in this perspective. We implemented an annotation platform for Italian tweets employed in a laboratory called #DEACTIVHATE, specifically designed for secondary school students. The laboratory aims at countering hateful phenomena online and making students aware of technologies that they use on a daily basis. We describe our teaching experience in high schools and the usefulness of the technologies and activities tested.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. IronITA is a new shared task in the EVALITA 2018 evaluation campaign, focused on the aut... more English. IronITA is a new shared task in the EVALITA 2018 evaluation campaign, focused on the automatic classification of irony in Italian texts from Twitter. It includes two tasks: 1) irony detection and 2) detection of different types of irony, with a special focus on sarcasm identification. We received 17 submissions for the first task and 7 submissions for the second task from 7 teams. Italiano. IronITA è un nuovo esercizio di valutazione della campagna di valutazione EVALITA 2018, specificamente dedicato alla classificazione automatica dell'ironia presente in testi estratti da Twitter. Comprende due task: 1) riconoscimento dell'ironia e 2) riconoscimento di diversi tipi di ironia, con particolare attenzione all'identificazione del sarcasmo. Abbiamo ricevuto 17 sottomissioni per il primo task e 7 per il secondo, da parte di 7 gruppi partecipanti.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. The automatic misogyny identification (AMI) task proposed at IberEval and EVALITA 2018 i... more English. The automatic misogyny identification (AMI) task proposed at IberEval and EVALITA 2018 is an example of the active involvement of scientific Research to face up the online spread of hate contents against women. Considering the encouraging results obtained for Spanish and English in the precedent edition of AMI, in the EVALITA framework we tested the robustness of a similar approach based on topic and stylistic information on a new collection of Italian and English tweets. Moreover, to deal with the dynamism of the language on social platforms, we also propose an approach based on automatically-enriched lexica. Despite resources like the lexica prove to be useful for a specific domain like misogyny, the analysis of the results reveals the limitations of the proposed approaches. lessici risultano utili per domini specifici come quello della misoginia, analizzando i risultati emergono i limiti degli approcci proposti.
EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020, 2020
The Hate Speech Detection (HaSpeeDe 2) task is the second edition of a shared task on the detecti... more The Hate Speech Detection (HaSpeeDe 2) task is the second edition of a shared task on the detection of hateful content in Italian Twitter messages. HaSpeeDe 2 is composed of a Main task (hate speech detection) and two Pilot tasks, (stereotype and nominal utterance detection). Systems were challenged along two dimensions: (i) time, with test data coming from a different time period than the training data, and (ii) domain, with test data coming from the news domain (i.e., news headlines). Overall, 14 teams participated in the Main task, the best systems achieved a macro F1-score of 0.8088 and 0.7744 on the indomain in the out-of-domain test sets, respectively; 6 teams submitted their results for Pilot task 1 (stereotype detection), the best systems achieved a macro F1-score of 0.7719 and 0.7203 on in-domain and outof-domain test sets. We did not receive any submission for Pilot task 2.
English. In the domain of Natural Language Processing (NLP), the interest in figurative language ... more English. In the domain of Natural Language Processing (NLP), the interest in figurative language is enhanced, especially in the last few years, thanks to the amount of linguistic data provided by web and social networks. Figurative language provides a non-literary sense to the words, thus the utterances require several interpretations disclosing the play of signification. In order to individuate different meaning levels in case of ironic texts detection, it is necessary a computational model appropriated to the complexity of rhetorical artifice. In this paper we describe our rulebased system of irony detection as it has been presented to the SENTIPOLC task of EVALITA 2016, where we ranked third on twelve participants. Italiano. Nell’ambito del Natural Language Processing (NLP) l’interesse per il linguaggio figurativo è particolarmente aumentato negli ultimi anni, grazie alla quantità d’informazione linguistica messa a disposizione dal web e dai social network. Il linguaggio figurati...
Nowadays, misogynistic abuse online has become a serious issue due, especially, to anonymity and ... more Nowadays, misogynistic abuse online has become a serious issue due, especially, to anonymity and interactivity of the web that facilitate the increase and the permanence of the offensive comments on the web. In this paper, we present an approach based on stylistic and specific topic information for the detection of misogyny, exploring the several aspects of misogynistic Spanish and English user generated texts on Twitter. Our method has been evaluated in the framework of our participation in the AMI shared task at IberEval 2018 obtaining promising results.
The importance of the detection of aggressiveness in social media is due to real effects of viole... more The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior online. For this reason, hate speech online is a real problem in modern society and the necessity of control of usergenerated contents has become one of the priorities for governments, social media platforms and Internet companies. Current methodologies are far from solving this problem. Indeed, several aggressive comments are also disguised as sarcastic. In this perspective, this research proposal wants to investigate the role played by creative linguistic devices, especially sarcasm, in hate speech in multilingual context.
The importance of the detection of aggressiveness in social media is due to real effects of viole... more The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior online. Indeed, this kind of legal cases are increasing in the last years. For this reason, the necessity of controlling user-generated contents has become one of the priorities for many Internet companies, although current methodologies are far from solving this problem. Therefore, in this work we propose an innovative approach that combines deep learning framework with linguistic features specific for this issue. This approach has been evaluated and compared with other ones in the framework of the MEX-A3T shared task at IberEval on aggressiveness analysis in Spanish Mexican tweets. In spite of our novel approach, we obtained low results.
The lack of understanding of figurative language online, such as ironic messages, is a common cau... more The lack of understanding of figurative language online, such as ironic messages, is a common cause of error for systems that analyze automatically the users’ opinions online detecting sentiment, emotions or stance. In order to deal with this problem of automatic processing of natural language, IroSvA shared task at IberLef 2019 asks participants to detect, for the first time, irony in short texts written in Spanish language, considering the three linguistic variants from Spain, Mexico and Cuba. Another novelty of this task is the presence of labels specifying the context of the utterance, such as current political or social issues discussed online. In the context of this shared task, we approached irony detection in Spanish short texts trying to exploit the provided topic information. In addition, we investigated the usefulness of stylistic, lexical and affective features during the development of the irony detection models for the three Spanish variants. Experimental results and f...
Uploads
Papers by Simona Frenda