Диагностические эксперименты
task
ID: diagnosticexperiments
Related Papers
Would you describe a leopard as yellow? Evaluating crowd-annotations with justified and informative disagreement
AbstractSemantic annotation tasks contain ambiguity and vagueness and require varying degrees of world knowledge. Disagreement is an important indication of these phenomena. Most traditional evaluation methods, however, critically hinge upon …...
Diagnosing Vision-and-Language Navigation: What Really Matters
AbstractVision-and-language navigation (VLN) is a multimodal task where an agent follows natural language instructions and navigates in visual environments. Multiple setups have been proposed, and researchers apply new model architectures …...