Numeracy-600K: Learning Numeracy for Detecting Exaggerated Information in Market Comments

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6307–6313 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6307 Numeracy-600K: Learning Numeracy for Detecting Exaggerated Information in Market Comments Chung-Chi Chen,1 Hen-Hsen Huang,2,4 Hiroya Takamura,3 Hsin-Hsi Chen1,4 1 Department of Computer Science and Information Engineering National Taiwan University, Taiwan 2 Department of Computer Science, National Chengchi University, Taiwan 3 Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology, Japan 4 MOST Joint Research Center for AI Technology and All Vista Healthcare, Taiwan cjchen@nlg.csie.ntu.edu.tw, hhhuang@nccu.edu.tw, takamura.hiroya@aist.go.jp, hhchen@ntu.edu.tw Abstract In this paper, we attempt to answer the ques- tion of whether neural network models can learn numeracy, which is the ability to predict the magnitude of a numeral at some specific position in a text description. A large bench- mark dataset, called Numeracy-600K, is pro- vided for the novel task. We explore several neural network models including CNN, GRU, BiGRU, CRNN, CNN-capsule, GRU-capsule, and BiGRU-capsule in the experiments. The results show that the BiGRU model gets the best micro-averaged F1 score of 80.16%, and the GRU-capsule model gets the best macro- averaged F1 score of 64.71%. Besides dis- cussing the challenges through comprehensive experiments, we also present an important ap- plication scenario, i.e., detecting exaggerated information, for the task. 1 Introduction As a prior research from a dataset obtained from Reuters, one of the largest international news agencies, over 65.66% of market comments con- tain numerals. Without the numerals in market comments, we will miss a lot of useful informa- tion. Table 1 lists some instances of real-time mar- ket comments. The topics include the descriptions of market data (S1), financial statements (S2), products (S3), analyst reports (S4), and events (S5). From the table, we can see that numerals provide more detailed information than words do. For example, in comment (S1) we can learn that the share price of Apple Inc. (AAPL) has fallen, but we cannot obtain the percentage change or the price quote without the numerals. Furthermore, (S3) provides crucial information such as the date (Q2) and the amount of sales with numerals (4.6 (S1) <AAPL> SHARES DOWN 4 PCT AT $113.7IN MORNING TRADE (S2) <AAPL> Q1 REV VIEW $75.08 BLN (S3) <AAPL> - Q2 MAC SALES OF 4.6 MLN UNITS VS 4.1 MLN UNITS LAST YEAR (S4) <AAPL>: CANACCORD GENUITY RAISES PRICE TARGET TO $600 (S5) <AAPL> CFO SAYS REVENUE EX- PECTED TO BE DOWN BETWEEN 5-10% IN CONSTANT CURRENCY FOR Q1 Table 1: Instances of market comments. (S6) S&P 500 <.SPX> UP 1.53 POINTS AT AFTER MARKET OPEN (S7) DOW JONES <.DJI> UP 8.70 POINTS AT AFTER MARKET OPEN (S8) U.S. Q3 GDP rises pct Table 2: Instances for the proposed task. and 4.1). These examples show the crucial roles of numerals in financial narratives. Table 2 lists three market comments selected from our dataset as examples. Investors would know from their experiences that the blanks in (S6) and (S7) should be filled with quotes of …

Content truncated to 500 words