Lightweight Grammatical Annotation in the TEI: New Perspectives

Lightweight Grammatical Annotation in the TEI: New Perspectives Piotr Bański1, Susanne Haaf2, Martin Mueller3 1Institute for the German Language, Mannheim (Germany) 2Berlin-Brandenburg Academy of Sciences and Humanities, Berlin (Germany) 3Northwestern University, Chicago (USA) 1banski@ids-mannheim.de, 2haaf@bbaw.de, 3martinmueller@northwestern.edu Abstract In mid-2017, as part of our activities within the TEI Special Interest Group for Linguists (LingSIG), we submitted to the TEI Technical Council a proposal for a new attribute class that would gather attributes facilitating simple token-level linguistic annotation. With this proposal, we addressed community feedback complaining about the lack of a specific tagset for lightweight linguistic annotation within the TEI. Apart from @lemma and @lemmaRef, up till now TEI encoders could only resort to using the generic attribute @ana for inline linguistic annotation, or to the quite complex system of feature structures for robust linguistic annotation, the latter requiring relatively complex processing even for the most basic types of linguistic features. As a result, there now exists a small set of basic descriptive devices which have been made available at the cost of only very small changes to the TEI tagset. The merit of a predefined TEI tagset for lightweight linguistic annotation is the homogeneity of tagging and thus better interoperability of simple linguistic resources encoded in the TEI. The present paper introduces the new attributes, makes a case for one more addition, and presents the advantages of the new system over the legacy TEI solutions. Keywords: linguistic annotation, lightweight annotation, TEI, TEI LingSIG 1. Introduction In July 2017, as part of our activities within the TEI Special Interest Group for Linguists (LingSIG), we submitted to the TEI Technical Council a proposal for the definition of a new attribute class that gathers token-level attributes facilitating simple linguistic annotation. With this proposal, we addressed repeated requests from various corpus projects to facilitate combined annotation of customary TEI text structures and basic token-level linguistic features. Most of our proposed modifications have been accepted and are part of the new release of the TEI Guidelines (3.3.0, published in January 2018).1 The present paper is therefore partially a report of success and a review of new features that the encoder has at her disposal, and in part, it provides arguments for extending the descriptive power of the Guidelines even further, towards supporting a complete chain of token-level grammatical analysis in various kinds of text collections. In the out-of-the-box TEI markup, it is now possible to use the following attributes of the elements <w> (« word ») and <pc> (« punctuation character »): • @pos (for part-of-speech) • @msd (for morpho-syntactic description) • @join (to signal string concatenation with a neighbouring element) These attributes – together with @lemma and @lemmaRef, previously defined inside <w> – are now encapsulated in a new attribute class, att.linguistic, supplying means crucial for basic grammatical annotation at the token level, i.e., for lightweight grammatical annotation.2 1 The changes were merged in a pull request which also references detailed discussion, see https://github.com/TEIC/TEI/pull/1671 and issue #1670. 2 We enclose element names in angle brackets …

Content truncated to 500 words