Document Type

Conference Paper

Publication Date

2024

DOI

10.1145/3678957.3685714

Publication Title

ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction

Pages

498-507

Conference Name

ICMI '24: International Conference on Multimodal Interaction, 4-8 November 2024, San Jose, Costa Rica

Abstract

Data chart visualizations and text are often paired in news articles, online blogs, and academic publications to present complex data. While chart visualizations offer graphical summaries of the data, the accompanying text provides essential context and explanation. Associating information from text and charts is straightforward for sighted users but presents significant challenges for individuals with low vision, especially on small-screen devices such as smartphones. The visual nature of charts coupled with the layout of the text inherently makes it difficult for low vision users to mentally associate chart data with text and comprehend the content due to their dependence on screen magnifier assistive technology, which only displays a small portion of the screen at any instant due to content enlargement. To address this problem, in this paper, we present a smartphone-based multimodal mixed-initiative interface that transforms static data charts and the accompanying text into an interactive slide show featuring frames containing “magnified views” of relevant data point combinations. The interface also includes a narration component that delivers tailored information for each “magnified view”. The design of our interface was informed by a user study with 10 low-vision participants, aimed at uncovering low vision interaction challenges and user-interface requirements with multimodal documents that integrate text and chart visualizations. Our interface was also evaluated in a subsequent study with 12 low-vision participants, where we observed significant improvements in chart usability compared to both status-quo screen magnifiers and a state-of-the-art solution.

Rights

© 2024 Copyright held by the owner/authors.

This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.

Original Publication Citation

Prakash, Y., Kolgar Nayak, A., Alyaan, S. M., Khan, P. A., Lee, H. N., & Ashok, V. (2024). Improving usability of data charts in multimodal documents for low vision users. In H. Hung, C. Oertel, M. Soleymani, T. Chaspari, H. Dibeklioglu, J. Shakla, & K. Truong (Eds.), ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction (pp. 498-507). Association for Computing Machinery. https://doi.org/10.1145/3678957.3685714

ORCID

0009-0000-9992-9706 (Nayak), 0009-0002-7353-4698 (Alyaan), 0009-0000-1186-6780 (Khan), 0000-0002-4772-1265 (Ashok)

Download

Share

COinS