BBN: Description of the SIFT System as Used for MUC-7

1 ALGORITHMS THAT LEARN TO EXTRACT INFORMATION BBN: DESCRIPTION OF THE SIFT SYSTEM AS USED FOR MUC-7 Scott Miller, Michael Crystal, Heidi Fox, Lance Ramshaw, Richard Schwartz, Rebecca Stone, Ralph Weischedel, and the Annotation Group BBN Technologies 70 Fawcett Street Cambridge, MA 02138 weischedel@bbn.com ABSTRACT For MUC-7, BBN has for the first time fielded a fully-trained system for NE, TE, and TR; results are all the output of statistical language models trained on annotated data, rather than programs executing hand- written rules. Such trained systems have some significant advantages: • They can be easily ported to new domains by simply annotating data with semantic answers. • The complex interactions that make rule-based systems difficult to develop and maintain can here be learned automatically from the training data. We believe that the results in this evaluation are evidence that such trained systems, even at their current level of development, can perform roughly on a par with rules hand-tailored by experts. Since MUC-3, BBN has been steadily increasing the proportion of the information extraction process that is statistically trained. Already in MET-1, our name-finding results were the output of a fully statistical, HMM-based model, and that statistical Identifinder™ model was also used for the NE task in MUC-7. For the MUC-7 TE and TR tasks, BBN developed SIFT, a new model that represents a significant further step along this path, replacing PLUM, a system requiring handwritten patterns, with SIFT, a single integrated trained model. SIFT: AN INTEGRATED TE/TR EXTRACTION SYSTEM Introduction At the sentence level, the SIFT system (“Statistics for Information From Text”) employs a unified statistical process to map from words to semantic structures. That is, part-of-speech determination, name- finding, parsing, and relationship-finding all happen as part of the same process. This allows each element of the model to influence the others, and avoids the assembly-line trap of having to commit to a particular part-of-speech choice, say, early on in the process, when only local information is available to inform the choice. The SIFT sentence-level model was trained from two sources: • General knowledge of English sentence structure was learned from the Penn Treebank corpus of one million words of Wall Street Journal text. • Specific knowledge about how the target entities and relations are expressed in English was learned from about 500 K words of on-domain text annotated with named entities, descriptors, and semantic relations. For extraction in a new domain, the names and descriptors of relevant items (persons, organizations, locations, and artifacts) are marked, as well as the target relationships between them that are signaled syntactically. For example, in the phrase “GTE Corp. of Stamford”, the annotation would record a 2 “location-of” connection between the company and the city. The model can thus learn the structures that are typically used in English to convey the target relationships. While the bulk of the TE/TR task happens within the sentence-level decoder, some further processing was still required to produce TE and TR answer templates. After the names, descriptors, and local relationships …

Content truncated to 500 words