Systematic TextRank Optimization in Extractive Summarization

Proceedings of Recent Advances in Natural Language Processing, pages 117–123 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_013 117 Evaluating Large Language Models in Relationship Extraction from Unstructured Data: Empirical Study from Holocaust Testimonies Isuri Anuradha Nanomi Arachchige University of Wolverhampton/ UK isurianuradha96@gmail.com Le An Ha University of Wolverhampton/ UK ha.l.a@wlv.ac.uk Ruslan Mitkov Lancaster University / UK r.mitkov@lancaster.ac.uk Vinitar Nahar University of Wolverhampton, UK vinita.nahar@wlv.ac.uk Abstract Relationship extraction from unstructured data remains one of the most challenging tasks in the field of Natural Language Process- ing (NLP). The complexity of relationship extraction arises from the need to compre- hend the underlying semantics, syntactic struc- tures, and contextual dependencies within the text. Unstructured data poses challenges with diverse linguistic patterns, implicit relation- ships, contextual nuances, complicating accu- rate relationship identification and extraction. The emergence of Large Language Models (LLMs), such as GPT (Generative Pre-trained Transformer), has indeed marked a significant advancement in the field of NLP.In this work, we assess and evaluate the effectiveness of LLMs in relationship extraction in the Holo- caust testimonies within the context of the His- torical realm. By delving into this domain- specific context, we aim to gain deeper in- sights into the performance and capabilities of LLMs in accurately capturing and extracting relationships within the Holocaust domain by developing a novel knowledge graph to visu- alise the relationships of the Holocaust. To the best of our knowledge, there is no exist- ing study which discusses relationship extrac- tion in Holocaust testimonies. The majority of current approaches for Information Extraction (IE) in historic documents are either manual or Optical Character Recognition (OCR) based. Moreover, in this study, we found that the Subject-Object-Verb extraction using GPT3- based relations produced more meaningful re- sults compared to the Semantic Role labeling- based triple extraction. 1 Introduction Understanding unstructured texts using compu- tational methods is considered a challenging task due to the complexity of the natural lan- guage. Holocaust testimonies are firsthand ac- counts provided by survivors, witnesses, and oth- ers who experienced or observed the atrocities of the Holocaust during World War II (Isuri A. Nanomi Arachchige, 2023). Holocaust testi- monies also belong to the category of unstructured texts, which presents unique challenges for com- putational assessment. These testimonies are often emotionally charged and contain highly-sensitive and personal information which is scattered ev- erywhere in the testimony. Moreover, Holocaust testimonies contain a range of linguistic complex- ities, such as archaic language, regional dialects, and highly specialised terminology related to the Holocaust, which can be challenging to parse us- ing traditional NLP techniques. Extracting relationships from Holocaust testi- monies is essential for historians as these firsthand accounts provide valuable information about the Holocaust. By uncovering hidden connections and associations between entities from the testimonies, historians can gain deeper insights into the his- torical context, dynamics between individuals and groups, and the broader narrative of the Holocaust. This information helps to enhance the understand- ing of the events, identify patterns, and shed light on the social, political, and cultural aspects of this tragic period …

Content truncated to 500 words