Механический Турок

Human linguistic annotation is crucial for many natural language processing tasks but can be expensive and time-consuming. We ex- plore the use of Amazon’s Mechanical Turk system, a significantly cheaper …...

Searching and reading the Web is one of the principal methods used to seek out infor- mation to resolve problems about technol- ogy in general and digital devices in partic- …...

Manual evaluation of translation quality is generally thought to be excessively time consuming and expensive. We explore a fast and inexpensive way of doing it using Amazon’s Mechanical Turk to …...

AbstractThis paper investigates whether high-quality annotations for tasks involving semantic disambiguation can be obtained without a major investment in time or expense. We examine the use of untrained human volunteers …...

AbstractThe emergence of crowdsourcing as a commonly used approach to collect vast quantities of human assessments on a variety of tasks represents nothing less than a paradigm shift. This is …...

AbstractCorpus-based approaches to machine translation (MT) rely on the availability of parallel corpora. To produce user-acceptable translation outputs, such systems need high quality data to be efficiency trained, optimized and …...

Recent work has established the efficacy of Amazon’s Mechanical Turk for constructing parallel corpora for machine translation re- search. We apply this to building a collec- tion of parallel corpora …...

Human evaluation of machine translation quality is a key element in the develop- ment of machine translation systems, as automatic metrics are validated through correlation with human judgment. How- ever, …...

We present a systematic study of the effect of crowdsourced translations on Machine Trans- lation performance. We compare Machine Translation systems trained on the same data but with translations obtained …...

This paper presents G-FLOW, a novel system for coherent extractive multi-document sum- marization (MDS).1 Where previous work on MDS considered sentence selection and or- dering separately, G-FLOW introduces a joint …...

AbstractThis paper presents a multi-dialect, multi-genre, human annotated corpus of dialectal Arabic. We collected utterances in five Arabic dialects: Levantine, Gulf, Egyptian, Iraqi and Maghrebi. We scraped newspaper websites for …...

AbstractThe language used in online forums differs in many ways from that of traditional language resources such as news. One difference is the use and frequency of nonliteral, subjective dialogue …...