Recognizing Personal Characteristics of Readers using Eye-Movements and Text Features

Proceedings of COLING 2012: Technical Papers, pages 1747–1762, COLING 2012, Mumbai, December 2012. Recognizing personal characteristics of readers using eye-movements and text features Pascual Martínez-Gómez1,2 Tadayoshi Hara2 Akiko Aizawa1,2 (1) The University of Tokyo (2) National Institute of Informatics {pascual, harasan, aizawa}@nii.ac.jp ABSTRACT In the present work we raise the hypothesis that eye-movements when reading texts reveal task performance, as measured by the level of understanding of the reader. With the objective of testing that hypothesis, we introduce a framework to integrate geometric information of eye-movements and text layout into natural language processing models via image processing techniques. We evidence the patterns in reading behavior between subjects with similar task performance using principal component analysis and quantify the likelihood of our hypothesis using the concept of linear separability. Finally, we point to potential applications that could benefit from these findings. KEYWORDS: eye-tracking, natural language processing, image recognition. TITLE AND ABSTRACT IN JAPANESE 視 視 視線 線 線の の の動 動 動き き きと と とテ テ テキ キ キス ス スト ト ト素 素 素性 性 性を を を用 用 用い い いた た た読 読 読み み み手 手 手の の の個 個 個性 性 性認 認 認識 識 識 本研究では、テキストを読む際の視線の動きから、テキストの理解度によって測定され るような読み手のタスクパフォーマンスが予測可能である、という仮説を立てる。この 仮説を検証するため、我々はまず、画像処理技術を介して、視線の動きとテキスト配置 に関する位置情報を、自然言語処理のモデルとして統合する枠組を導入する。次に我々 は、近いタスクパフォーマンスの被験者間に共通した読解行動のパターンを主成分分析 によって同定し、この線形分離可能性を求めることで我々の仮説の蓋然性を定量的に示 す。最後に、我々はこれらの発見から恩恵を受け得る応用例について述べる。 KEYWORDS: 視線追跡、自然言語処理、画像認識. 1747 1 Introduction Reading is a common activity that is part of the process of information transfer between humans. However, despite of the important role it has played in recent history and its current wide use, this process of information transfer is not well understood. The difficulty in modelling the reading activity stems from the direct unobservability of human mental states and how the information is decoded and integrated into the brain, or simply forgotten. The reading act can be seen as an interaction between the reader and the writer where the information and other aspects of the communication are transferred via the document. If we could observe the detailed editing process of a writer, that would indeed give us valuable information about the writer, useful to interpret the message to be sent. However, information on the editing process is usually not available, but the final result in the form of a document is. On the other side of the communication channel, the reader does not necessarily evidence any reading actions except for the movements of the eyes, and that is all we have to understand the reading process. Several psycholinguistic studies (Rayner, 1998; McDonald and Shillcock, 2003) have shown that document characteristics influence on cognitive processing and that they are reflected on eye-movements in an on-line manner. There have been attempts to also quantify the influence of textual linguistic characteristics on reading behavior for certain types of reading tasks (Martínez- Gómez et al., 2012b), noting that although linguistic features of documents can be used to explain eye-movements and reading behavior, there might be other influencing factors. In our work, we consider the document and the eye-movements of the reader as the only observable variables in the reading act, and …

Content truncated to 500 words