Evaluating and comparing three text-production techniques

Evaluating and comparing three text-production techniques .Ios~ Coch GSI-Erli l, pl. dos Mal:seillais F-94227 Charcnton-lc-Pont Codex France jose. coch@erli, fr Abstract What al+c the benefits of ttsing Natural [~anguagc (;cneratio,t in an industrial apl+lication? We have attempt t<) answer part (}f this qttcsti{}n with at descripti(}n of an assessment {}f three techniques for producing multiscntcntial text: senti- atutomatic fill-in-lhc-blank interfacing, automalic linguistic-and-tcmphltes hybrid generation, and hunlall writing. This asscssIllol]l used a black b(}x motlmdology, with ain independetlt blind- tested jury that gave difforent quality levels in relation to a sot o1' criteria. The texts used for tile assessliicnt wcfc business reply letters. 1 Introduction Thct'c arc many m{}re industrial proiccts in Analysis than in Natural l,anguagc (;cneration. Therclorc the bencl:ils {}f using applied N1 ,(; would a]}pcal + a crucial issue. We have l}r{}vidcd a partial rCSl)onsc It} this issue by analysing the asscsslnent o[" three different tcclmiqucs for producing multiscntential text (in this case, business reply letters). In the lollowing section, we have describe{l tile three techniques under assessment: semi-automatic non- linguistic fill-in-the-blank intcrlhcing, atut(}matic linguistic-and-tentphtte hybrid gerlct'ation, and human wril:ing. The third section deals with the black-b{}x mcthodol(}gy and qttality critcria used for tile aISSBSSIlICII[, "['lie lk)urth section descl+ibcs the results {}f the alsscssntcn[. The fiftll section givcs examl}les of letters prt~luccd by both the semi-autonutlic system, and the lilU_Btistic-and-tempIate hybrid system. The hlst section analyses tile results of tile assessment. 2 Three techniques for producing multisentential text This section describes tile three text-production techniques under assesslllelt[. 2.1 Fill-in-the-blank semi-automatic technique Since 1975, the mail department el' lea Redoule (a l~;uropcan mail-order colnpany) has been using a semi ;automatic reply system, referred to below as "SA", consisting of a nutnbel + of predelined attd fill- in-the-blank sentences or paragraphs which are identified by codes that the writers memorisc. Writing a letter thcrcfore involves typing the code that corresponds to the desired pm'agraph and inserting the relevant elcnlents. The sentences or paragraphs thus produced are thcl'clbre concatenations o1' predefined and illSertcd texts. l. A relatively high number of prcdefined sentences and paragraphs have to be provided, to cover the writers' needs, but: 2. In fact, writers use only a reduced set of predefined 13aragraphs, Ihe nunlber of which depending on tile writer. 3. The quality of tile t+esulting style of reply varies widely. 2.2 Automatic Hybrid Generation (IAnguistic + Template approach) lea Rcdoutc and GSI-Erli have developed a real- situation pilot system (for details on this project, see (Coch, David, and Magnolcr, 1995)) which builds up a text (i.e. a letter) fronl data entered by tile human operator who processes the request; a custonlcf database; and knowledge bases. It uses GSi-Erli's AlcthGen text generation toolbox (see (Coch, 1996)). The overall system is composed of two Inain modules: thc I)ccision module and the Generation module. The Decision module has the following functions: 249 • it allows the writer (who reads the request letter) to identify the author and subject of the request letter; • it asks the writer for relevant …

Content truncated to 500 words