On the difficulty of automatically detecting irony: beyond a simple case of negation

The final publication is available at Springer via http://dx.doi.org/10.1007/s10115-013-0652-8It is well known that irony is one of the most subtle devices used to, in a refined way and without a negation marker, deny what is literally said. As such, its automatic detection would represent valuable knowledge regarding tasks as diverse as sentiment analysis, information extraction, or decision making. The research described in this article is focused on identifying key values of components to represent underlying characteristics of this linguistic phenomenon. In the absence of a negation marker, we focus on representing the core of irony by means of three conceptual layers. These layers involve 8 different textual features. By representing four available data sets with these features, we try to find hints about how to deal with this unexplored task from a computational point of view. Our findings are assessed by human annotators in two strata: isolated sentences and entire documents. The results show how complex and subjective the task of automatically detecting irony could be.The research work of Paolo Rosso was done in the framework of the European Commission WIQ-EI Web Information Quality Evaluation Initiative (IRSES grant no. 269180) project within the FP 7 Marie Curie People, the DIANA-APPLICATIONS - Finding Hidden Knowledge in Texts: Applications (TIN2012-38603-C02-01) project, and the VLC/CAMPUS Microcluster on Multimodal Interaction in Intelligent Systems.Reyes Pérez, A.; Rosso, P. (2014). On the difficulty of automatically detecting irony: beyond a simple case of negation. Knowledge and Information Systems. 40(3):595-614. https://doi.org/10.1007/s10115-013-0652-8595614403Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput Linguistics 34(4):555–596Atserias J, Casas B, Comelles E, González M, Padró L, Padró M (2006) Freeling 1.3: syntactic and semantic services in an open-source nlp library. In: Proceedings of the 5th international conference on language resources and evaluation, pp 48–55Attardo S (2007) Irony as relevant inappropriateness. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 135–174Banerjee S, Agarwal N (2012) Analyzing collective behavior from blogs using swarm intelligence. Knowl Inf Syst. doi: 10.1007/s10115-012-0512-yBeydoun G, Hoffmann A (2012) Dynamic evaluation of the development process of knowledge-based information systems. Knowl Inf Syst. doi: 10.1007/s10115-012-0491-zBurfoot C, Baldwin T (2009) Automatic satire detection: are you having a laugh? In: ACL-IJCNLP ’09: proceedings of the ACL-IJCNLP 2009 conference short papers, pp 161–164Carvalho P, Sarmento L, Silva M, de Oliveira E (2009) Clues for detecting irony in user-generated contents: oh...!! It’s “so easy”; -). In: TSA ’09: proceeding of the 1st international CIKM workshop on topic-sentiment analysis for mass opinion. ACM, Hong Kong, China, pp 53–56Clark H, Gerrig R (1984) On the pretense theory of irony. J Exp Psychol Gen 113(1):121–126Colston H (2007) On necessary conditions for verbal irony comprehension. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 97–134Colston H, Gibbs R (2007) A brief history of irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 3–24Curcó C (2007) Irony: negation, echo, and metarepresentation. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 269–296Davidov D, Tsur O, Rappoport A (2010) Semi-supervised recognition of sarcastic sentences in Twitter and Amazon. In: Proceedings of the 14th conference on computational natural language learning, CoNLL ’10. Association for Computational Linguistics, Stroudsburg, PA, USA, pp 107–116Francisco V, Gervás P, Peinado F (2010) Ontological reasoning for improving the treatment of emotions in text. Knowl Inf Syst 24(2):23Gibbs R (2007) Irony in talk among friends. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 339–360Gibbs R, Colston H (2007) The future of irony studies. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, LondonGiora R (1995) On irony and negation. Discourse Process 19(2):239–264Giora R, Balaban N, Fein O, Alkabets I (2005) Negation as positivity in disguise. In: Colston H, Katz A (eds) Figurative language comprehension: social and cultural influences. Erlbaum, Hillsdale, pp 233–258Giora R, Federman S, Kehat A, Fein O, Sabah H (2005) Irony aptness. Humor 18:23–39Grice H (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics, vol 3. Academic Press, New York, pp 41–58Horn L, Kato Y (2000) Introduction: negation and polarity at the millennium. In: Horn L, Kato Y (eds) Studies in negation and polarity. Oxford University Press, Oxford, pp 1–19Kaup B, Lüdtke J, Zwaan R (2006) Processing negated sentences with contradictory predicates: is a door that is not open mentally closed? J Pragmat 38:1033–1050Kisilevich S, Ang CS, Last M (2011) Large-scale analysis of self-disclosure patterns among online social networks users: A Russian context. Knowl Inf Syst. doi: 10.1007/s10115-011-0443-zKreuz R (2001) Using figurative language to increase advertising effectiveness. In: Office of Naval Research Military Personnel Research Science Workshop. University of Memphis, Memphis, TNKumon-Nakamura S, Glucksberg S, Brown M (2007) How about another piece of pie: the allusional pretense theory of discourse irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, LondonLangacker R (1991) Concept, image and symbol, the cognitive basis of grammar. Mounton de Gruyter, BerlinLiu J, Wang K (2012) Anonymizing bag-valued sparse data by semantic similarity-based clustering. Knowl Inf Syst. doi: 10.1007/s10115-012-0515-8Lucariello J (2007) Situational irony: a concept of events gone away. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 467–498Miller G (1995) Wordnet: a lexical database for english. Commun ACM 38(11):39–41Pang B, Lee L (2004) A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the ACL, pp 271–278Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, Morristown, NJ, USA, pp 79–86Pedersen T, Patwardhan S, Michelizzi J (2004) Wordnet:similarity—measuring the relatedness of concepts. In: Proceeding of the 9th national conference on artificial intelligence (AAAI-04). Association for Computational Linguistics, Morristown, NJ, USA, pp 1024–1025Reyes A, Rosso P (2011) Mining subjective knowledge from customer reviews: a specific case of irony detection. In: Proceedings of the 2nd workshop on computational approaches to subjectivity and sentiment analysis (WASSA 2.011). Association for Computational Linguistics, pp 118–124Reyes A, Rosso P (2012) Making objective decisions from subjective data: detecting irony in customers reviews. Decis Support Syst 53(4):754–760. doi: 10.1016/j.dss.2012.05.027Reyes A, Rosso P, Buscaldi D (2012) From humor recognition to irony detection: the figurative language of social media. Data Knowl Eng 74:1–12. doi: 10.1016/j.datak.2012.02.005Sarmento L, Carvalho P, Silva M, de Oliveira E (2009) Automatic creation of a reference corpus for political opinion mining in user-generated content, In: TSA ’09: proceeding of the 1st international CIKM workshop on topic-sentiment analysis for mass opinion. ACM, Hong Kong, China, pp 29–36Sperber D, Wilson D (1992) On verbal irony. Lingua 87:53–76Tsur O, Davidov D, Rappoport A (2010) ICWSM—a great catchy name: semi-supervised recognition of sarcastic sentences in online product reviews. In: Cohen WW, Gosling S (eds) Proceedings of the 4t international AAAI conference on weblogs and social media. The AAAI Press, Washington, DC, pp 162–169Utsumi A (1996) A unified theory of irony and its computational formalization. In: Proceedings of the 16th conference on computational linguistics. Association for Computational Linguistics, Morristown, NJ, USA, pp 962–967Veale T, Hao Y (2009) Support structures for linguistic creativity: a computational analysis of creative irony in similes. In: Proceedings of CogSci 2009, the 31st annual meeting of the cognitive science society, pp 1376–1381Veale T, Hao Y (2010) Detecting ironic intent in creative comparisons. In: Proceedings of 19th European conference on artificial intelligence—ECAI 2010. IOS Press, Amsterdam, The Netherlands, pp 765–770Whissell C (2009) Using the revised dictionary of affect in language to quantify the emotional undertones of samples of natural language. Psychol Rep 105(2):509–521Wilson D, Sperber D (2007) On verbal irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 35–56Zagibalov T, Belyatskaya K, Carroll J (2010) Comparable English-Russian book review corpora for sentiment analysis. In: Proceedings of the 1st workshop on computational approaches to subjectivity and sentiment analysis. Lisbon, Portugal, pp 67–7