Lexical Coherence Graph Modeling Using Word Embeddings

Proceedings of NAACL-HLT 2016, pages 1414–1423, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Lexical Coherence Graph Modeling Using Word Embeddings Mohsen Mesgar and Michael Strube Heidelberg Institute for Theoretical Studies gGmbH Schloss-Wolfsbrunnenweg 35 69118 Heidelberg, Germany (mohsen.mesgar|michael.strube)@h-its.org Abstract Coherence is established by semantic connec- tions between sentences of a text which can be modeled by lexical relations. In this pa- per, we introduce the lexical coherence graph (LCG), a new graph-based model to represent lexical relations among sentences. The fre- quency of subgraphs (coherence patterns) of this graph captures the connectivity style of sentence nodes in this graph. The coherence of a text is encoded by a vector of these fre- quencies. We evaluate the LCG model on the readability ranking task. The results of the ex- periments show that the LCG model obtains higher accuracy than state-of-the-art coher- ence models. Using larger subgraphs yields higher accuracy, because they capture more structural information. However, larger sub- graphs can be sparse. We adapt Kneser-Ney smoothing to smooth subgraphs’ frequencies. Smoothing improves performance. 1 Introduction The concept of coherence is based on cohesive se- mantic relations connecting elements of a text. Co- hesive relations are expressed through grammar and the vocabulary of a language. The former is referred to as grammatical coherence, the latter as lexical coherence (Halliday and Hasan, 1976). Grammat- ical coherence encompasses coreference, substitu- tion, ellipsis, etc. Lexical coherence comprises se- mantic connections among words of a text. In this paper we measure text coherence by mod- eling lexical coherence. Lexical relations spec- ify cohesive relations over the sentences of a text. These lexical relations can be any kind of seman- tic relation: repetition, synonymy, hyperonymy, meronymy, etc. These lexical items may or may not have the same reference (Halliday and Hasan, 1976). Why does the little boy wriggle all the time? Girls don’t. In this example the lexical items boy and girls are semantically related. Although they do not re- fer to the same entity, they still connect these two sentences. There is coherence between any pair of lexi- cal items that stand to each other in some lexico- semantic relation (Halliday and Hasan, 1976). For textual purposes it is not required to determine the type of the relation. It is only necessary to recog- nize semantically related lexical items, and these re- lations can be learned by cooccurring lexical items. One can use world knowledge resources to deter- mine semantic relations. This way is expensive in terms of determining the best resource, e.g. Word- Net vs. Freebase. WordNet lacks broad coverage in particular with proper names, Freebase is restricted to nominal concepts and entities. Recent improvements in embedding representa- tions of words let us efficiently compute semantic relations among lexical items in the vector space. These models use a vector of numbers to encode the meaning of words. We use these vectors to check the existence of any kind of semantic relations between two words. In the following example the sentences are con- …

Content truncated to 500 words