MultiGranCNN: An Architecture for General Matching of Text Chunks on Multiple Levels of Granularity

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 63–73, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics MultiGranCNN: An Architecture for General Matching of Text Chunks on Multiple Levels of Granularity Wenpeng Yin and Hinrich Sch¨utze Center for Information and Language Processing University of Munich, Germany wenpeng@cis.uni-muenchen.de Abstract We present MultiGranCNN, a general deep learning architecture for matching text chunks. MultiGranCNN supports multigranular comparability of represen- tations: shorter sequences in one chunk can be directly compared to longer se- quences in the other chunk. Multi- GranCNN also contains a flexible and modularized match feature component that is easily adaptable to different types of chunk matching. We demonstrate state- of-the-art performance of MultiGranCNN on clause coherence and paraphrase iden- tification tasks. 1 Introduction Many natural language processing (NLP) tasks can be posed as classifying the relationship be- tween two TEXTCHUNKS (cf. Li et al. (2012), Bordes et al. (2014b)) where a TEXTCHUNK can be a sentence, a clause, a paragraph or any other sequence of words that forms a unit. Paraphrasing (Figure 1, top) is one task that we address in this paper and that can be formalized as classifying a TEXTCHUNK relation. The two classes correspond to the sentences being (e.g., the pair <p, q+>) or not being (e.g., the pair <p, q−>) paraphrases of each other. Another task we look at is clause coherence (Figure 1, bot- tom). Here the two TEXTCHUNK relation classes correspond to the second clause being (e.g., the pair <x, y+>) or not being (e.g., the pair <x, y−>) a discourse-coherent continuation of the first clause. Other tasks that can be formalized as TEXTCHUNK relations are question answering (QA) (is the second chunk an answer to the first?), textual inference (does the first chunk imply the second?) and machine translation (are the two chunks translations of each other?). p PDC will also almost certainly fan the flames of speculation about Longhorn’s release. q+ PDC will also almost certainly reignite speculation about release dates of Microsoft ’s new products. xq− PDC is indifferent to the release of Longhorn. The dollar suffered its worst one-day loss in a month, y+ falling to 1.7717 marks . . . from 1.7925 marks yesterday. y− up from 112.78 yen in late New York trading yesterday. Figure 1: Examples for paraphrasing and clause coherence tasks In this paper, we present MultiGranCNN, a gen- eral architecture for TEXTCHUNK relation classi- fication. MultiGranCNN can be applied to a broad range of different TEXTCHUNK relations. This is a challenge because natural language has a com- plex structure – both sequential and hierarchical – and because this structure is usually not parallel in the two chunks that must be matched, further increasing the difficulty of the task. A successful detection algorithm therefore needs to capture not only the internal structure of TEXTCHUNKS, but also the rich pattern of their interactions. MultiGranCNN is based on two innovations that are …

Content truncated to 500 words