Reading the Reader

Project Desciption

Overview

Imagine reading a text that knows you are looking at it – imagine reading a text that adapts to how you read it and supports you as you read. How we read today has changed little since the invention of printing. However, new technologies can make it possible to identify individual reading patterns and adjust the reading material according to the reader, making reading easier and more accessible also for reading impaired to improve comprehension. This needs a robust set of machine learning methods that integrate and classify features from complex, noisy, multimodal data of biological origin in real-time.

We apply an interdisciplinary approach of data- and vision science, typographic design and software engineering to develop new robust methods to extract features of reading from real-time measures of eye-movements, pupil dilation and e.g., posture, allowing us to adopt and assess the impact of individualized typographic settings and support the changing needs of the user.

Fig 1: An overview of the system. The user interacts with a reading device, controlled by a reading app. Simultaneously an eye tracker records gaze points, pupil size, and posture, from which additional features (saccades, fixations, and regressions) are extracted. A classifier inspects this stream as well as signals from the reader (originating from the stimuli and from user-generated input) and generates intervention signals from this and a trained dataset. (From ( Citation: Hansen, Bækgaard & al., 2023 Hansen, J., Bækgaard, P., Valgeirsdottir, D. & Beier, S. (2023). Universal design of gaze interactive applications for people with special needs. ) ).

Background

More than 216 m people have moderate to severe vision impairment ( Citation: Bourne, Flaxman & al., 2017 Bourne, R., Flaxman, S., Braithwaite, T., Cicinelli, M., Das, A., Jonas, J., Keeffe, J., Kempen, J., Leasher, J., Limburg, H. & (2017). Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: A systematic review and meta-analysis. The Lancet Global Health, 5(9). e888–e897. ) and many are challenged readers ( Citation: Legge, Rubin & al., 1985 Legge, G., Rubin, G., Pelli, D. & Schleske, M. (1985). Psychophysics of reading—II. Low vision. Vision research, 25(2). 253–265. ) . However, reading has changed little since the invention of printing, although new technology makes it possible to identify individual reading patterns and personalize the presentation of text, making reading easier and text more accessible and comprehensible.

Now, imagine reading a text that knows you are looking at it and adapts to how you read, actively supporting your style of reading while you read.

We will develop a robust set of automated methods that

• integrate,
• classify,
• interpret and
• respond

to complex multimodal observations of user behavior in real-time.

This requires interdisciplinary work across data science, software engineering, typography design, psychology and vision science with the aim to extract user behavior data in real time by tracking fixations, eye movements, pupil changes, user interaction, time, text progression, reading patterns and more.

This allows us to characterize individual reading style, to identify typographic preferences and to adapt the presentation of text, in order to enable a new level of adaptive support for readers with impaired vision or cognition.

The project extends a recently initiated strategic collaboration between the Technical University of Denmark (DTU) and the Royal Danish Academy (RDA) that explores synergies between art, science, and technology to enrich society.

It involves pre-graduate and post-graduate students at both institutions and provides a vital element of research-based training. The developed tools have generic qualities, which subsequently can be repurposed for the assistance of individuals with other types of (reading) disabilities than the specific vision disorders included in this protocol.

The majority of the description above is courtesy the original funding application authored by Sofie Beier and Per Bækgaard.

Additional Reading

You can read more about the project and related work here:

• AI will help people with impaired vision to read better,
• Artificial intelligence will help to improve reading opportunities for millions of people
• Reading the Reader
• Hansen, Bækgaard, Valgeirsdottir and Beier: Universal Design of Gaze Interactive Applications for People with Special Needs

Thesis works related to the project:

• Jürgens, C.E.S.: Automated detection of reading pattern differences for normal readers compared to people with dyslexia (MSc Thesis, 2023)
• Refsgaard, L.K.B., Farooq, A.A.: Reading The Reader: an intelligent system for optimizing typography for central vision loss readers (MSc Thesis, 2023)
• Klerk, M.B.: Improving reading amongst dyslexics by identifying specific issues using eye tracking (BSc Thesis, 2024)
• Deniz, R.: Designing a reading aid for AMD patients (MSc Thesis, 2024)
• Andrei, M.-V., Pereira, L.E.: Designing a Typography-Adaptive Reading Interface for Individuals with Central Vision Loss (MSc Thesis, 2024)
• Noor, Sifat-E: Unveiling Reading Patterns: Exploring the Impact of Writing Styles on Reading Behaviour through Eye Tracking (MSc Thesis, 2024)
• Palinec, M: Reactive Reading: Crafting an Adaptive Frontend Interface for Dynamic Machine Learning (MSc Thesis, 2024)
• Jensen, H.E.: Intervening without interrupting: A study on the optimal level of intervention in reading applications with adaptive user interfaces (MSc Thesis, 2025)
• Farkas, R: Utilizing Transformer-Based Encoding on Eye-Tracking Data to Enhance Reading for CVL Patients (MSc Thesis, 2025)

Bibliography