Reusable workflows for gender prediction

Reusable workflows for gender prediction Matej Martinc, Senja Pollak Joˇzef Stefan Institute Jamova cesta 39, Ljubljana, Slovenia {matej.martinc, senja.pollak}@ijs.si Abstract This paper presents a system for author profiling (AP) modeling that reduces the complexity and time of building a sophisticated model for a number of different AP tasks. The system is implemented in a cloud-based visual programming platform ClowdFlows and is publicly available to a wider audience. In the platform, we also implemented our already existing state of the art gender prediction model and tested it on a number of cross-genre tasks. The results show that the implemented model, which was trained on tweets, achieves results comparable to state of the art models for cross-genre gender prediction. There is however a noticeable decrease in accuracy when the genre of a test set is different from the genre of the train set. Keywords: author profiling, workflows, reusability and replicability 1. Introduction In recent years, the data from the user-generated content (UGC) has become a popular resource for studies in natu- ral language processing (NLP) and text mining, especially due to its accessibility, size and a near real-time publishing. Messages from the social media by personal users display a trend towards more personal content (Naaman et al., 2010), which makes it a useful dataset for learning about the users’ characteristics, related to demographics, psychological pro- file or mental health. The field that uses text documents to discover users’ attributes automatically is known as author profiling (AP) and includes tasks, such as the prediction of author’s gender (Rangel et al., 2017), which is also at the core of this paper, but also the prediction of author’s age (Rangel et al., 2016), personality type (Verhoeven et al., 2016) or language variety (Rangel et al., 2017). While deep learning approaches are gradually taking over different areas of NLP, the best approaches to AP still use more traditional classifiers and require extensive feature en- gineering (Rangel et al., 2017). This test-driven approach relies heavily on trying out different feature and param- eter configurations, which can be very time consuming. To accelerate this process of model building, we propose the implementation of a system that enables faster model prototyping by employing the visual programming (VP) paradigm. The VP paradigm simplifies the representation of complex procedures into visual arrangements of their building blocks and consequently enables faster and more efficient model prototyping and model fine-tuning. The contribution of this paper is two-fold and is anchored in recent trends in reusability and replicability in NLP. Our AP system implementation is built in an online visual program- ming environment, which is to the best of our knowledge the first of its kind. The proposed approach to construct- ing prototype AP models through visual workflows of NLP components contributes to the reusability in NLP on the level of specific components, as well as on the level of the entire gender prediction models. The pretrained models for five languages (four of which we developed for PAN 2017 (Martinc et al., 2017)) can …

Content truncated to 500 words