What’s Wrong, Python? – A Visual Differ and Graph Library for NLP in Python

What’s Wrong, Python? – A Visual Differ and Graph Library for NLP in Python Bal´azs Indig12, Andr´as Simonyi1, No´emi Ligeti-Nagy1 1MTA-PPKE Hungarian Language Technology Research Group 2P´azm´any P´eter Catholic University, Faculty of Information Technology and Bionics H-1083 Budapest, Pr´ater street 50/A {indig.balazs, simonyi.andras, ligeti-nagy.noemi}@itk.ppke.hu Abstract The correct analysis of the output of a program based on supervised learning is inevitable in order to be able to identify the errors it produced and characterise its error types. This task is fairly difficult without a proper tool, especially if one works with complex data structures such as parse trees or sentence alignments. In this paper, we present a library that allows the user to interactively visualise and compare the output of any program that yields a well-known data format. Our goal is to create a tool granting the total control of the visualisation to the user, including extensions, but also have the common primitives and data-formats implemented for typical cases. We describe the common features of the common NLP tasks from the viewpoint of visualisation in order to specify the essential primitive functions. We enumerate many popular off-the-shelf NLP visualisation programs to compare with our implementation, which unifies all of the profitable features of the existing programs adding extendibility as a crucial feature to them. Keywords: Tools, Visualisation, Python 1. Introduction In natural language processing (NLP), it is obviously fundamental to achieve good performance on the well- researched tasks; however, on a task currently under devel- opment, it is even more relevant to be aware of what have happened and where the program has gone wrong. The thorough analysis of the errors can lead to new ideas and methods, and finally, to better results. In most of the NLP tasks, this error inspection is carried out manually, while the number of tools supporting this work phase is low. The main reason of that is that for visual comparing there needs to be a good graphing library that displays output data cor- rectly, which can be also hijacked to display visual differ- ences in a form that helps human processing. Finally, a well-visualised good example can tell more of the inner- working of an algorithm than a thousand words or tables. In scientific computing, especially in NLP, two program- ming languages serve as the source language of the major- ity of the new tools. JAVA is popular because of its ma- turity and cross-platform compatibility. In the job market, one can find many well-trained programmers developing in JAVA, the universities are still putting a great emphasis on teaching JAVA, and finally, the slowly evolving industrial applications are still using JAVA for new projects because of its adoption. However, Python is still gaining more and more popularity on the basis of its simplicity allowing fast prototyping and its ability to hide the performance of the critical parts – which are usually implemented in C++ – under a newbie-friendly Python API without sacrificing the overall running or development performance. The cheap work power of fast-trained …

Content truncated to 500 words