Edit Detection and Parsing for Transcribed Speech

Edit Detection and Parsing for Transcribed Speech Eugene Charniak and Mark Johnson Deparments of Computer Science and Cognitive and Linguistic Sciences Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University, Providence, RI 02912 ec,mj@cs.brown.edu ∗ Abstract We present a simple architecture for parsing transcribed speech in which an edited-word de- tector first removes such words from the sen- tence string, and then a standard statistical parser trained on transcribed speech parses the remaining words. The edit detector achieves a misclassification rate on edited words of 2.2%. (The NULL-model, which marks everything as not edited, has an error rate of 5.9%.) To evalu- ate our parsing results we introduce a new eval- uation metric, the purpose of which is to make evaluation of a parse tree relatively indifferent to the exact tree position of EDITED nodes. By this metric the parser achieves 85.3% precision and 86.5% recall. 1 Introduction While significant effort has been expended on the parsing of written text, parsing speech has received relatively little attention. The comparative neglect of speech (or transcribed speech) is understandable, since parsing tran- scribed speech presents several problems absent in regular text: “um”s and “ah”s (or more formally, filled pauses), frequent use of par- entheticals (e.g., “you know”), ungrammatical constructions, and speech repairs (e.g., “Why didn’t he, why didn’t she stay home?”). In this paper we present and evaluate a simple two-pass architecture for handling the problems of parsing transcribed speech. The first pass tries to identify which of the words in the string are edited (“why didn’t he,” in the above exam- ple). These words are removed from the string given to the second pass, an already existing sta- tistical parser trained on a transcribed speech ∗ This research was supported in part by NSF grant LIS SBR 9720368 and by NSF ITR grant 20100203. corpus. (In particular, all of the research in this paper was performed on the parsed “Switch- board” corpus as provided by the Linguistic Data Consortium.) This architecture is based upon a fundamen- tal assumption: that the semantic and prag- matic content of an utterance is based solely on the unedited words in the word sequence. This assumption is not completely true. For example, Core and Schubert [8] point to coun- terexamples such as “have the engine take the oranges to Elmira, um, I mean, take them to Corning” where the antecedent of “them” is found in the EDITED words. However, we be- lieve that the assumption is so close to true that the number of errors introduced by this assump- tion is small compared to the total number of errors made by the system. In order to evaluate the parser’s output we compare it with the gold-standard parse trees. For this purpose a very simple third pass is added to the architecture: the hypothesized edited words are inserted into the parser output (see Section 3 for details). To the degree that our fundamental assumption holds, a “real” ap- plication would ignore this last step. This architecture has …

Content truncated to 500 words