ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning in Language Using Neuro-symbolic Cloud Computing

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 645–652 Barcelona, Spain (Online), December 12, 2020. 645 ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning in Language using neuro-symbolic cloud computing Len Yabloko Next Generation Software, New City, New York, USA lenyabloko@gmail.com Abstract I present ETHAN: Experimental Testing of Hybrid AI Node implemented entirely on free cloud computing infrastructure. The ultimate goal of this research is to create modular reusable hybrid neuro-symbolic architecture for Artificial Intelligence. As a test case I model natural language comprehension of causal relations from open domain text corpus that combines semi-supervised language model (Huggingface Transformers) with constituency and dependency parsers (Allen Institute for Artificial Intelligence). The experimental results presented in this paper show potential for SOTA-level performance and demonstrate hybrid Artificial Intelligence in action. On SemEval- 2020 Task 5 Subtask 1 ETHAN achieves F1 0.878 which is the second best result at the time of my writing. On Subtask 2 ETHAN currently scores 0.673 which is within the top ten results. All code used for experiments is available as open source and can be executed directly from the web browser without any installation or configuration. I briefly discuss some theoretical approaches that led to the proposed solutions and future directions of this research. 1 Introduction Several hard open problems of machine learning and AI are intrinsically related to causality (?). One of them is the algorithmization of counterfactuals (Pearl, 2011): what algorithms would allow machines to recognize crucial semantic difference between sentences like If Oswald didn’t kill Kennedy, someone else did, If Oswald hadn’t killed Kennedy, someone else would have.(Adams, 1975) What additional knowledge would a machine require to do such a binary classification? But going even further, what linguistic features of the two sentences would need to be isolated and analyzed before any form of causal inference can be performed? If the first question conveniently re-frames the linguistic problem into statistical one, then the second question requires machine to identify some basic causal relation in the text, namely the antecedent and consequent. SemEval-2020 Task 5(Yang et al., 2020) asks these two question in a form of subtasks 1 and 21. Even for a single language (English) the task so fundamental to natural language processing (NLP) that it bridges the major divide in the artificial intelligence (AI) research between the good old fashioned AI (GOFAI) and the methodology that led to the latest advances, namely the Deep Learning (DL). It is worth mentioning the intense debates2 around the future of AI which inevitably circles back to the role of causality in human and machine understanding of language or any data for that matter. My results add to the position that any successful approach must be a hybrid between symbolic and connectionist sides of the divide (Marcus, 2020) . I was able to achieve a SOTA F1 score 0.878 on the subtask 1 in a short time using open source software. However, I found subtask 2 significantly more challenging, This work is licensed under a …

Content truncated to 500 words