Learning Document-Level Semantic Properties from Free-Text Annotations

Proceedings of ACL-08: HLT, pages 263–271, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Learning Document-Level Semantic Properties from Free-text Annotations S.R.K. Branavan Harr Chen Jacob Eisenstein Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {branavan, harr, jacobe, regina}@csail.mit.edu Abstract This paper demonstrates a new method for leveraging free-text annotations to infer se- mantic properties of documents. Free-text an- notations are becoming increasingly abundant, due to the recent dramatic growth in semi- structured, user-generated online content. An example of such content is product reviews, which are often annotated by their authors with pros/cons keyphrases such as “a real bar- gain” or “good value.” To exploit such noisy annotations, we simultaneously find a hid- den paraphrase structure of the keyphrases, a model of the document texts, and the underly- ing semantic properties that link the two. This allows us to predict properties of unannotated documents. Our approach is implemented as a hierarchical Bayesian model with joint in- ference, which increases the robustness of the keyphrase clustering and encourages the doc- ument model to correlate with semantically meaningful properties. We perform several evaluations of our model, and find that it sub- stantially outperforms alternative approaches. 1 Introduction A central problem in language understanding is transforming raw text into structured representa- tions. Learning-based approaches have dramatically increased the scope and robustness of this type of automatic language processing, but they are typi- cally dependent on large expert-annotated datasets, which are costly to produce. In this paper, we show how novice-generated free-text annotations avail- able online can be leveraged to automatically infer document-level semantic properties. With the rapid increase of online content cre- ated by end users, noisy free-text annotations have pros/cons: great nutritional value ... combines it all: an amazing product, quick and friendly service, cleanliness, great nutrition ... pros/cons: a bit pricey, healthy ... is an awesome place to go if you are health con- scious. They have some really great low calorie dishes and they publish the calories and fat grams per serving. Figure 1: Excerpts from online restaurant reviews with pros/cons phrase lists. Both reviews discuss healthiness, but use different keyphrases. become widely available (Vickery and Wunsch- Vincent, 2007; Sterling, 2005). For example, con- sider reviews of consumer products and services. Often, such reviews are annotated with keyphrase lists of pros and cons. We would like to use these keyphrase lists as training labels, so that the proper- ties of unannotated reviews can be predicted. Hav- ing such a system would facilitate structured access and summarization of this data. However, novice- generated keyphrase annotations are incomplete de- scriptions of their corresponding review texts. Fur- thermore, they lack consistency: the same under- lying property may be expressed in many ways, e.g., “healthy” and “great nutritional value” (see Fig- ure 1). To take advantage of such noisy labels, a sys- tem must both uncover their hidden clustering into properties, and learn to predict these properties from review text. This paper presents a model that addresses both …

Content truncated to 500 words