Learning from Explicit and Implicit Supervision Jointly For Algebra Word Problems

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 297–306, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Learning from Explicit and Implicit Supervision Jointly For Algebra Word Problems Shyam Upadhyay1 Ming-Wei Chang2 Kai-Wei Chang3 Wen-tau Yih2 1University of Illinois at Urbana-Champaign, Urbana, IL, USA 2Microsoft Research, Redmond, WA, USA 3University of Virginia, Charlottesville, VA, USA Abstract Automatically solving algebra word problems has raised considerable interest recently. Ex- isting state-of-the-art approaches mainly rely on learning from human annotated equations. In this paper, we demonstrate that it is pos- sible to efficiently mine algebra problems and their numerical solutions with little to no man- ual effort. To leverage the mined dataset, we propose a novel structured-output learning al- gorithm that aims to learn from both explicit (e.g., equations) and implicit (e.g., solutions) supervision signals jointly. Enabled by this new algorithm, our model gains 4.6% abso- lute improvement in accuracy on the ALG- 514 benchmark compared to the one without using implicit supervision. The final model also outperforms the current state-of-the-art approach by 3%. 1 Introduction Algebra word problems express mathematical rela- tionships via narratives set in a real-world scenario, such as the one below: Maria is now four times as old as Kate. Four years ago, Maria was six times as old as Kate. Find their ages now. The desired output is an equation system which ex- presses the mathematical relationship symbolically: m = 4 × n and m − 4 = 6 × (n − 4) where m and n represent the age of Maria and Kate, respectively. The solution (i.e., m = 40, n = 10) can be found by a mathematical engine given the equation systems. Building efficient automatic algebra word problem solvers have clear values for online education sce- narios. The challenge itself also provides a good test bed for evaluating an intelligent agent that un- derstands natural languages, a direction advocated by artificial intelligence researchers (Clark and Et- zioni, 2016). One key challenge of solving algebra word prob- lems is the lack of fully annotated data (i.e., the an- notated equation system associated with each prob- lem). In contrast to annotating problems with binary or categorical labels, manually solving algebra word problems to provide correct equations is time con- suming. As a result, existing benchmark datasets are small, limiting the performance of supervised learning approaches. However, thousands of alge- bra word problems have been posted and discussed in online forums, where the solutions can be easily mined, despite the fact that some of them could be incorrect. It is thus interesting to ask whether a bet- ter algebra problem solver can be learned by lever- aging these noisy and implicit supervision signals, namely the solutions. In this work, we address the technical difficulty of leveraging implicit supervision in learning an alge- bra word problem solver. We argue that the effec- tive strategy is to learn from both explicit and im- plicit supervision signals jointly. In particular, we design a novel …

Content truncated to 500 words