Don’t Eclipse Your Arts Due to Small Discrepancies: Boundary Repositioning with a Pointer Network for Aspect Extraction

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 351–357 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 351 Text Classification with Negative Supervision Sora Ohashi†, Junya Takayama†, Tomoyuki Kajiwara‡, Chenhui Chu‡, Yuki Arase† † Graduete School of Information Science and Technology, Osaka University ‡ Institute of Datability Science, Osaka University † {ohashi.sora, takayama.junya, arase}@ist.osaka-u.ac.jp ‡ {kajiwara, chu}@ids.osaka-u.ac.jp Abstract Advanced pre-trained models for text repre- sentation have achieved state-of-the-art per- formance on various text classification tasks. However, the discrepancy between the seman- tic similarity of texts and labelling standards affects classifiers, i.e. leading to lower perfor- mance in cases where classifiers should assign different labels to semantically similar texts. To address this problem, we propose a sim- ple multitask learning model that uses negative supervision. Specifically, our model encour- ages texts with different labels to have distinct representations. Comprehensive experiments show that our model outperforms the state- of-the-art pre-trained model on both single- and multi-label classifications, sentence and document classifications, and classifications in three different languages. 1 Introduction Text classification generally consists of two pro- cesses: an encoder that converts texts to numeri- cal representations and a classifier that estimates hidden relations between the representations and class labels. The text representations are gener- ated using N-gram statistics (Wang and Manning, 2012), word embeddings (Joulin et al., 2017; Wang et al., 2018), convolutional neural networks (Kalch- brenner et al., 2014; Zhang et al., 2015; Shen et al., 2018), and recurrent neural networks (Yang et al., 2016, 2018). Recently, powerful pre-trained models for text representations, e.g. Bidirec- tional Encoder Representations from Transformers (BERT) (Devlin et al., 2019), have shown state- of-the-art performance on text classification tasks using only the simple classifier of a fully connected layer. However, a problem occurs when a classification task is adversarial to text encoders. Encoders aim to represent the meanings of texts; hence, seman- Sentence Label BERT A cold is a legit disease. – Cold Oh my god! I caught a cold! Cold Cold Table 1: Examples of BERT classification for labelling a disease contracted by a writer. Both sentences are about the common cold. Only the second example in- dicates that the writer had a cold. BERT misclassified the first sentence. tically similar texts tend to have closer representa- tions. Meanwhile, a classifier should distinguish subtle differences that lead to different label assign- ments, although the texts are semantically similar. Table 1 shows an example of classification results using BERT for the MedWeb dataset (Wakamiya et al., 2017). This task requires the labelling of a disease contracted by the writer of a text. Although both texts in Table 1 refer to the common cold, only the second example implies that the writer had a cold. BERT mistakenly labelled both texts as Cold1, likely owing to their semantic related- ness. When the standard of class label assignments disagrees with the semantic similarity, the classi- fier tends to be error-prone owing to the excessive effects of the semantic similarity. To address this …

Content truncated to 500 words