Aspect-augmented Adversarial Networks for Domain Adaptation

Aspect-augmented Adversarial Networks for Domain Adaptation Yuan Zhang, Regina Barzilay, and Tommi Jaakkola Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {yuanzh, regina, tommi}@csail.mit.edu Abstract We introduce a neural method for transfer learning between two (source and target) clas- sification tasks or aspects over the same do- main. Rather than training on target la- bels, we use a few keywords pertaining to source and target aspects indicating sentence relevance instead of document class labels. Documents are encoded by learning to em- bed and softly select relevant sentences in an aspect-dependent manner. A shared classi- fier is trained on the source encoded docu- ments and labels, and applied to target en- coded documents. We ensure transfer through aspect-adversarial training so that encoded documents are, as sets, aspect-invariant. Ex- perimental results demonstrate that our ap- proach outperforms different baselines and model variants on two datasets, yielding an improvement of 27% on a pathology dataset and 5% on a review dataset.1 1 Introduction Many NLP problems are naturally multitask classi- fication problems. For instance, values extracted for different fields from the same document are often dependent as they share the same context. Exist- ing systems rely on this dependence (transfer across fields) to improve accuracy. In this paper, we con- sider a version of this problem where there is a clear dependence between two tasks but annotations are available only for the source task. For example, 1The code is available at https://github.com/ yuanzh/aspect_adversarial. Pathology report: • Final diagnosis: BREAST (LEFT) … Invasive ductal carcinoma: identified. Carcinoma tumor size: num cm. Grade: 3. … Lymphatic vessel invasion: identified. Blood vessel invasion: Suspicious. Margin of invasive carcinoma … Diagnosis results: Source (IDC): Positive Target (LVI): Positive Figure 1: A snippet of a breast pathology report with diagnosis results for two types of disease (aspects): carcinoma (IDC) and lymph invasion (LVI). Note how the same phrase indicating positive results (e.g. identified) is applicable to both aspects. A transfer model learns to map other key phrases (e.g. Grade 3) to such shared indicators. the target goal may be to classify pathology reports (shown in Figure 1) for the presence of lymph in- vasion but training data are available only for car- cinoma in the same reports. We call this problem aspect transfer as the objective is to learn to classify examples differently, focusing on different aspects, without access to target aspect labels. Clearly, such transfer learning is possible only with auxiliary in- formation relating the tasks together. The key challenge is to articulate and incorpo- rate commonalities across the tasks. For instance, in classifying reviews of different products, sentiment words (referred to as pivots) can be shared across the products. This commonality enables one to align feature spaces across multiple products, enabling useful transfer (?). Similar properties hold in other contexts and beyond sentiment analysis. Figure 1 515 Transactions of the Association for Computational Linguistics, vol. 5, pp. 515–528, 2017. Action Editor: Hal Daum´e III . Submission batch: 12/2016; Revision batch: 6/2017; Published 12/2017. c⃝2017 …

Content truncated to 500 words