On Generative Spoken Language Modeling from Raw Audio

Modeling Content and Context with Deep Relational Learning Maria Leonor Pacheco and Dan Goldwasser Department of Computer Science Purdue University West Lafayette, IN 47907 {pachecog, dgoldwas}@purdue.edu Abstract Building models for realistic natural language tasks requires dealing with long texts and ac- counting for complicated structural depen- dencies. Neural-symbolic representations have emerged as a way to combine the reasoning capabilities of symbolic methods, with the expressiveness of neural networks. However, most of the existing frameworks for combining neural and symbolic representations have been designed for classic relational learning tasks that work over a universe of symbolic entities and relations. In this paper, we present DRAIL, an open-source declarative framework for spe- cifying deep relational models, designed to support a variety of NLP scenarios. Our frame- work supports easy integration with expressive language encoders, and provides aninterfaceto study the interactions between representation, inference and learning. 1 Introduction Understanding natural language interactions in realistic settings requires models that can deal with noisy textual inputs, reason about the depen- dencies between different textual elements, and leverage the dependenciesbetween textual content and the context from which it emerges. Work in linguistics and anthropology has defined context as a frame that surrounds a focal communicative event and provides resources for its interpretation (Gumperz, 1992; Duranti and Goodwin, 1992). As a motivating example, consider the interac- tions in the debate network described in Figure 1. Given a debate claim (t1), and two consecutive posts debating it (p1, p2), we define a textual in- ference task, determining whether a pair of text elements hold the same stance in the debate (denoted using the relation Agree(X,Y)). This task is similar to other textual inference tasks (Bowman et al., 2015) that have been successfully approached using complex neural representations (Peters et al., 2018; Devlin et al., 2019). In add- ition, we can leverage the dependencies between these decisions. For example, assuming that one post agrees with the debate claim (Agree(t1, p2)), and the other one does not (¬Agree(t1, p1)), the disagreement between the two posts can be inferred: ¬Agree(t1, p1) ∧ Agree(t1, p2) → ¬ Agree(p1, p2). Finally, we consider the social context of the text. The disagreement between the posts can reflect a difference in the perspectives their authors hold on the issue. This informa- tion might not be directly observed, but it can be inferred using the authors’ social interactions and behavior, given the principle of social homophily (McPherson et al., 2001), stating that people with strong social ties are likely to hold similar views and authors’ perspectives can be captured by rep- resenting their social interactions. Exploiting this information requires models that can align the social representation with the linguistic one. Motivated by these challenges, we introduce DRAIL1, a Deep Relational Learning framework, which uses a combined neuro-symbolic repre- sentation for modeling the interaction between multiple decisions in relational domains. Similar to other neuro-symbolic approaches (Mao et al., 2019; Cohen et al., 2020), our goal is to exploit the complementary strengths of the two modeling paradigms. Symbolic …

Content truncated to 500 words