HITSZ-ICRC: An Integration Approach for QA TempEval Challenge

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 830–834, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics HITSZ-ICRC: An Integration Approach for QA TempEval Challenge Yongshuai Hou, Cong Tan, Qingcai Chen and Xiaolong Wang Department of Computer Science and Technology Harbin Institute of Technology Shenzhen Graduate School HIT Campus, The University Town of Shenzhen, Shenzhen, 518055, China {yongshuai.hou, viptancong, qingcai.chen}@gmail.com wangxl@insun.hit.edu.cn Abstract This paper presents the HITSZ-ICRC system designed for the QA TempEval challenge in SemEval-2015. The system used an integra- tion approach to annotate temporal informa- tion by merging temporal annotation results from different temporal annotators. TIPSemB, ClearTK and TARSQI were used as temporal annotators to get candidate temporal annota- tion results. Evaluation demonstrated that our system was effective for improving the per- formance of temporal information annotation, and achieved recalls of 0.18, 0.26 and 0.19 on Blog, News and Wikipedeia test sets. 1 Introduction The QA TempEval (Llorens et al., 2015) in Se- mEval-2015 is a temporal information annotation challenge, which is a follow-up task after Tem- pEval-1 (Verhagen et al., 2007), TempEval-2 (Verhagen et al., 2010) and TempEval-3(UzZaman et al., 2013). QA TempEval task is similar to the task ABC in TempEval-3, requires participant sys- tem (1) extracting and normalizing temporal ex- pressions, (2) extracting events and (3) identifying temporal relations on plain documents. Temporal information annotation should follow TimeML scheme (Pustejovsky et al., 2003a). Difference be- tween QA TempEval task and task ABC in Tem- pEval-3 is evaluation method: in all previous TempEval tasks, annotated result was evaluated by the temporal information annotation accuracy based on manually annotated test corpus; in QA TempEval, annotated result was evaluated by tem- poral question-answering(QA) accuracy in the giv- en temporal QA system (UzZaman et al., 2012) based on temporal knowledge produced from par- ticipant’s annotation. Temporal annotation is useful in information re- trieval, QA, natural language understanding and so on. A lot of researches have been attracted on this topic in the past years. Many methods were pro- posed and many toolkits were implemented for temporal information annotation. TIMEN (Llorens et al., 2012a) is a community- driven tool using rule-based method based on knowledge base to solve the temporal expression normalization problem. TARSQI Toolkit (Verha- gen and Pustejovsky, 2008) is a modular system for automatic temporal information annotation. The toolkit can extract temporal expressions, events and recognize temporal relations by its dif- ferent components. Llorens et al. (2010) used CRF models based on semantic information to annotate temporal information according to TimeML scheme, and their TIPSem system got outstanding performance results in TempEval-2. Steve (2013) piped machine-learning models in his ClearTK system to annotate temporal information using a small set of features. His system got best perfor- mance for temporal relation identification in Tem- pEval-3. The TIMEN toolkit was integrated into the ClearTK system for temporal expression nor- malization. Llorens et al. (2012b) proposed an au- tomatic method to improve the correctness of each individual annotation by merging different annota- tion results with …

Content truncated to 500 words