FastHybrid: A Hybrid Model for Efficient Answer Selection

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2378–2388, Osaka, Japan, December 11-17 2016. FastHybrid: A Hybrid Model for Efficient Answer Selection Lidan Wang IBM Watson T.J. Watson Research Ctr. Yorktown Heights, NY wangli@us.ibm.com Ming Tan IBM Watson T.J. Watson Research Ctr. Yorktown Heights, NY mingtan@us.ibm.com Jiawei Han Computer Science Dept. UIUC Urbana, IL hanj@cs.uiuc.edu Abstract Answer selection is a core component in any question-answering systems. It aims to select cor- rect answer sentences for a given question from a pool of candidate sentences. In recent years, many deep learning methods have been proposed and shown excellent results for this task. How- ever, these methods typically require extensive parameter (and hyper-parameter) tuning, which gives rise to efficiency issues for large-scale datasets, and potentially makes them less portable across new datasets and domains (as re-tuning is usually required). In this paper, we propose an extremely efficient hybrid model (FastHybrid) that tackles the problem from both an accuracy and scalability point of view. FastHybrid is a light-weight model that requires little tuning and adaptation across different domains. It combines a fast deep model (which will be introduced in the method section) with an initial information retrieval model to effectively and efficiently han- dle answer selection. We introduce a new efficient attention mechanism in the hybrid model and demonstrate its effectiveness on several QA datasets. Experimental results show that although the hybrid uses no training data, its accuracy is often on-par with supervised deep learning tech- niques, while significantly reducing training and tuning costs across different domains. 1 Introduction Open-domain question answering (QA) aims to serve a user’s information request by returning a list of direct answers. This problem has been receiving an increasingly amount of attention in the NLP and machine learning communities in the recent years (Ferrucci et al., 2012; Etzioni et al., 2011). Answer sentence selection (Yih et al., 2013; Tan et al., 2016; Yu et al., 2014; Severyn et al., 2013), which, given a user question, returns the correct sentences that contain the exact answer, is a core component in QA systems. The performance of QA systems critically depends on choosing the right candidate sentences which facilitate the extraction of final answers. To be successful, in addition to accuracy, real-world systems must be scalable and select the most accurate answer sentences in a short amount of time. However, accuracy and speed/scalability are com- peting forces that often counteract each other. It is often the case that methods developed for improving accuracy incur moderate-to-large computational costs. For example, models based on neural networks have become very popular due to their strong accuracy for this task (Yu et al., 2014). However, they are typically slower at training and test time as compared to simple models, which may limit their use on very large datasets (Joulin et al., 2016). The speed issue is particularly important when working with new domains and datasets, as the model may have to be re-trained or adapted for the new dataset. …

Content truncated to 500 words