Multilingual Dictionary Based Construction of Core Vocabulary

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 405–412 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 405 Adjusting Image Attributes of Localized Regions with Low-level Dialogue Tzu-Hsiang Lin1, Alexander Rudnicky1, Trung Bui2, Doo Soon Kim2, Jean Oh1 1Carnegie Mellon University, 2Adobe Research tzuhsial@alumni.cmu.edu, {air,jeanoh}@cmu.edu, {bui,dkim}@adobe.com Abstract Natural Language Image Editing (NLIE) aims to use natural language instructions to edit images. Since novices are inexperienced with image editing techniques, their instructions are often ambiguous and contain high-level abstractions which require complex editing steps. Motivated by this inexperience aspect, we aim to smooth the learning curve by teaching the novices to edit images using low-level command terminologies. Towards this end, we develop a task-oriented dialogue system to investigate low-level instructions for NLIE. Our system grounds language on the level of edit operations, and suggests options for users to choose from. Though compelled to express in low-level terms, user evaluation shows that 25% of users found our system easy-to-use, resonating with our motivation. Analysis shows that users generally adapt to utilizing the proposed low-level language interface. We also identified object segmentation as the key factor to user satisfaction. Our work demonstrates advantages of low-level, direct language-action mapping approach that can be applied to other problem domains beyond image editing such as audio editing or industrial design. Keywords: dialogue, image editing, natural language understanding, low-level 1. Introduction Image editing has long been in demand since the invention of photography. However, learning image editing is time- consuming. It involves a wide assortment of features, and combinations of these features to achieve a desired effect. As of today, most novices edit their images by requesting experts for help. These requests come in the form of Im- age Edit Requests (IER) – natural language descriptions which express desired changes to be made. When experts see these requests, they can accurately edit these images via a two-stage process: (i) First, experts look at the original image and the IER and interprets the high-level concepts expressed by the novice. (ii) Second, they come up with one or more low-level edit operations for these concepts and apply these edits. Can we build machines that do the same thing? This motivates the study on Natural Language Image Editing (NLIE). There are mainly two approaches towards NLIE. The two-stage approach follows the editing process of ex- perts. Several works have developed semantic parsers for the first stage, with datasets collected from crowd- sourcing (Manuvinakurike et al., 2018a), online image edit- ing communities (Mohapatra, 2018), or spoken conversa- tion with experts (Manuvinakurike et al., 2018b). Though these semantic parsers are able to capture the high-level in- tent of novices, little has been discussed about the second stage – how to infer edits with these parsers. These datasets still require expert annotations for generating edits, which is a difficult one-to-many mapping. On the other hand, the image generation approach directly generates an edited im- age given the original image and an IER using …

Content truncated to 500 words