Robust Semantic Analysis for Unseen Data in FrameNet

Proceedings of Recent Advances in Natural Language Processing, pages 628–633, Hissar, Bulgaria, 12-14 September 2011. Robust Semantic Analysis for Unseen Data in FrameNet Alexis Palmer, Afra Alishahi, Caroline Sporleder Computational Linguistics and Phonetics Saarland University, Germany {apalmer,afra,csporled}@coli.uni-saarland.de Abstract We present a novel method for FrameNet- based semantic role labeling (SRL), fo- cusing on limitations posed by the limited coverage of available annotated data. Our SRL model is based on Bayesian cluster- ing and has the advantage of being very ro- bust in the face of unseen and incomplete data. Frame labeling and role labeling are modeled in like fashions, allowing cascad- ing classification scenarios. The model is shown to perform especially well on un- seen data. In addition, we show that for seen data, predicting semantic types for roles improves role labeling performance. 1 Introduction The majority of recent work in semantic role la- beling (SRL) has been carried out on PropBank- style semantic argument annotations (Palmer et al., 2005), rather than on FrameNet-style annota- tions (Ruppenhofer et al., 2006). FrameNet dif- fers from PropBank in that FrameNet annotations are more strongly semantically driven. FrameNet generalizes over different parts of speech and can assign the same sense (frame) to a noun and a verb as in (1), where both competition and play are as- signed the COMPETITION frame. Also, FrameNet assigns semantic roles not only to syntactic argu- ments of the target but also to constituents which are not directly syntactically dependent on the tar- get but can be semantically understood as filling a role, e.g., Wivenhoe Town in (1a). (1) a. [Wivenhoe Town]Participant1 have never won the competitionCompetition. b. [Olympiakos]Participant1 playsCompetition [against Aris Salonica]Participant1 [in Piraeus]Place. A major challenge for FrameNet-style SRL is posed by the limited coverage of available anno- tated data. The FrameNet lexicographic corpus was annotated on a frame-by-frame basis, select- ing individual example sentences for each lexical unit (LU), or pairing of lemma and frame. This means that many common lemmas are missing from FrameNet, and for those that are included the number of example sentences is often rela- tively small and not in accordance with distribu- tions found in naturally-occurring texts. FrameNet’s well-known coverage gaps translate directly to drops in labeling performance, motivat- ing the development of systems which are more robust in the face of sparse data. For example, the supervised SRL system Shalmaneser (Erk and Pad´o, 2006) obtains a frame labeling accuracy of 93% on FrameNet 1.2 (with a 90-10 training-test split), but the same system’s performance drops to 47% accuracy when trained on FrameNet 1.3 and tested on texts with full frame-semantic annota- tions (Palmer and Sporleder, 2010). Similarly, Das et al. (2010) report a 60% frame labeling F-Score on SemEval-07 data, but of 210 unseen lemmas, their system predicts just four frames correctly.1 In general the term unseen could refer to un- seen frames, unseen lemmas, or unseen LUs. As further discussed in Section 4, we are interested in unseen LUs: cases in which the system has not been exposed …

Content truncated to 500 words