Story Trees: Representing Documents using Topological Persistence

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2413–2429 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2413 Story Trees: Representing Documents using Topological Persistence Pantea Haghighatkhah♦, Antske Fokkens♣♦, Pia Sommerauer♣, Bettina Speckmann♦, Kevin Verbeek♦ ♦ Eindhoven University of Technology, Dep. of Mathematics & Computer Science ♣ Vrije Universiteit Amsterdam, Computational Linguistics and Text Mining Lab {p.haghighatkhah, b.speckmann, k.a.b.verbeek}@tue.nl, {antske.fokkens, pia.sommerauer}@vu.nl Abstract Topological Data Analysis (TDA) focuses on the inherent shape of (spatial) data. As such, it may provide useful methods to explore spatial representations of linguistic data (embeddings) which have become central in NLP. In this paper we aim to introduce TDA to researchers in language technology. We use TDA to represent document structure as so-called story trees. Story trees are hierarchical representations created from semantic vector representations of sentences via persistent homology. They can be used to identify and clearly visualize prominent components of a story line. We showcase their potential by using story trees to create extractive summaries for news stories. Keywords: Topical Data Analysis, Semantic Vectors, Document level discourse 1. Introduction Topological data analysis (TDA) provides insights into the shape of (spatial) data. TDA is built on a strong mathematical foundation and thus delivers transparent and explainable results. Rather than analyzing data point by point, TDA captures the overall shape of the data, ignoring the inherent noise present in individual data points. As such, TDA can represent point data in terms of global geometric or topological structures that capture information about the data as a whole. It has received significant attention in a variety of scientific fields, such as biomedicine (Nielson et al., 2015), chem- istry (Lee et al., 2017), and material science (Nakamura et al., 2015). Spatial data has become increasingly central in NLP, as most approaches rely on embedding representations of linguistic units. Hence previous work has explored the possibilities of TDA for text analysis in various ways. For example, several papers use TDA to derive superior features from high-dimensional data that can be used as input for machine learning, with some reporting good re- sults compared to baselines (Doshi and Zadrozny, 2018; Kushnareva et al., 2021). Another set of papers uses TDA directly to explore a text’s structure for the purpose of clustering or finding key phrases. However, to the best of our knowledge, these attempts use only basic lexical overlap as input for their analyses. In general the use of TDA so far remains limited to isolated attempts; the NLP community as a whole is still largely unaware of the advanced, transparent analysis methods that TDA can provide. In this paper, we illustrate how to apply TDA to high- dimensional linguistic data representations, while pro- viding a transparent model for investigating a text’s structure and salient components. To this end we in- troduce story trees which capture the shape of a text via the persistence of the connected components of its sentences, represented as points in a semantic space. By tracking the …

Content truncated to 500 words