Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss German
14973Authors
nan
Abstract
AbstractThis paper presents SwissCrawl, the largest Swiss German text corpus to date. Composed of more than half a million sentences, it was generated using a customized web scraping tool that could be applied to other low-resource languages as well. The approach demonstrates how freely available web pages can be used to construct comprehensive text corpora, which are of fundamental importance for natural language processing. In an experimental evaluation, we show that using the new corpus leads to significant improvements for the task of language modeling.
Keywords
Full Content
Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 254–258 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 254 A Seed Corpus of Hindu Temples in India Priya Radhakrishnan* AI Labs, American Express Priya.Radhakrishnan@aexp.com Abstract Temples are an integral part of culture and heritage of India and are centers of religious practice for practicing Hindus. A scientific study of temples can reveal valuable insights into Indian culture and heritage. However to the best of our knowledge, learning resources that aid such a study are either not publicly available or non-existent. In this endeavour we present our initial efforts to create a corpus of Hindu temples in India. In this paper, we present a simple, re-usable platform that creates temple corpus from web text on temples. Curation is improved using classifiers trained on textual data in Wikipedia articles on Hindu temples. The training data is verified by human volunteers. The temple corpus consists of 4933 high accuracy facts about 573 temples. We make the corpus and the platform freely available. We also test the re-usability of the platform by creating a corpus of museums in India. We believe the temple corpus will aid scientific study of temples and the platform will aid in construction of similar corpora. We believe both these will significantly contribute in promoting research on culture and heritage of a region. Keywords: Corpus Creation, Information Extraction, Crowd Sourcing 1. Introduction Motivation : Temples remain an integral part of culture and heritage of India and are centres of religious practice for practicing Hindus (Trouillet, 2017). A scientific study of temples can reveal valuable insights into the Indian cul- ture and heritage. Much of the information about temples is available as text in the open web, which can be utilized to conduct such a study. However this information is not in the form of a learning resource, which can be readily used for such studies. Two sources of readily available facts on temples is Wikipedia page infoboxes and Google knowledgegraph. But we find these sources to cover only a limited number of popular temples. For instance, as per 2001 census, India has more than 2 million Hindu temples1 out of which only 518 have wikipedia articles. Wikipedia articles are present only for the popular temples. Even for popular temple like ‘Brahmapureeswarar Temple’ which has a Wikipedia arti- cle, Google knowledge graph does not return any facts. In this paper we present our initial efforts towards the cre- ation of a corpus of Hindu temples in India, using textual data on temples in the web. Figure 1 depicts temple cor- pus creation. Consider the temple ‘Tirumala Venkateswara Temple’ as an example. The input webpage (shown on left side in Figure 1) is textual data on the temple. Our platform converts this into the facts i.e.location, deity and mange- ment, and stores (shown on right side of Figure 1) as temple corpus. Data Collection : Recent advancements in reading com- prehension and question …
Content truncated to 500 words
Labeled Text
AbstractThis paper presents SwissCrawl, the largest Swiss German text corpus to date. Composed of more than half a million sentences, it was generated using a customized web scraping tool that could be applied to other low-resource languages as well. The approach demonstrates how freely available web pages can be used to construct comprehensive text corpora, which are of fundamental importance for natural language processing. In an experimental evaluation, we show that using the new corpus leads to significant improvements for the task of language modeling.
Publication Details
- Paper ID:
- 14973
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan