Automatic Insertion of Accents in French Text

Automatic Insertion of Accents m French Text Michel Simard Laboratoire de recherche appliquee en linguistique informatique (RALI) Universit€ de Montreal simardmmiro.UMontreal.CA Abstract Automatic accent inser·tion (AAI) is the problem of re-inserting accents (diacritics) into a text where they are missing. Unaccented French texts are still quite common in electronic media, as a result of a long his- tory of character encoding problems and the lack of well-established conventions for typing accented char- acters on computer keyboards. We present an AAI method for French, based on a stochastic language model. This method was implemented into a program and G library of functions, which are now commer- cially available. Our experiments show that French text processed with this program contains less than one accent error per 130 words. We also show how our AAI method can be nsed to do on-the-fly accent in- sertions within a word-processing environment, which makes it possible to write in French without having to type accents. A prototype of such a_ s'ystem was integrated into the Emacs editor, and iS now avail- able to all students and employees of t}1e Universite de Montreal's computer science department. 1 Introduction Even in this era of flashy, high-speed multimC::dia in- formation, unaccented French texts (i.e texts without diacritics) are still routinely encountered in electronic media. Two factors account for this: first, the com- puter field has long suffered from a lack of sufficiently widespread standards for encoding accented charac- ters, which has resulted in a plethora of problems in the electronic transfer and processing of French texts. Even now, it is not uncommon for one of the soft- ware links in an E-mail distribution chain to delib- erately remove accents in order to avoid subsequent problems. Secondly, when nsing a computer keyboard that is not specifically designed for French, keying in French accented characters can turn out to be a laborious activity. This is a matter of both stan- dards and ergonomics. As a result, a large number of French-speaking users systematically avoid using ac- cented characters, at least in informal communication. If this situation remains tolerable in practice, it is essentially because it is extremely rare that the ab- sence of accents renders a French text incomprehen- sible to the human reader. Cases of ambiguity do nonetheless occur: for instance, "Ce chantier ferme a cause des emeutes" could be interpreted as '\Ce chan tier ferme a cause des CmeuteS 11 C'This work-site is closing because of the riots") or \\Ce chan tier fermi a cause des Cmeutesn ("This closed work-site [more naturally put, this work-site closure] has caused riot- s',). From a linguistic point of view, the lack of accents in French simply increases the relative degree of am- biguity inherent in the language. At worst, it slows down reading and proves awkward, much as a text . written entirely in capital letters might do. 27 The fact remains, however, that while unaccented ri·ench text may be tolerated under certain circum- stances, it is not acceptable in common usage, …

Content truncated to 500 words