Document Type
Conference Paper
Publication Date
2025
DOI
10.1145/3744257.3744275
Publication Title
W4A '25: Proceedings of the 22nd International Web for All Conference
Pages
36-47
Conference Name
W4A '25: 22nd International Web for All Conference, 2025, April 28-29, 2025, Sydney, Australia
Abstract
Online food ordering has become commonplace due to its convenience. The wide variety of culinary choices, combined with fast and economical door-delivery services, encourages more people to order food online. To facilitate this process, food vendors, including restaurants, often provide full menus on their websites, typically in visual formats such as images or PDFs. While this is convenient for sighted users, blind and visually impaired (BVI) individuals face significant challenges accessing these visual menus with their screen reader assistive technology. An interview study with 12 BVI screen reader users revealed that present assistive tools do not adequately satisfy the needs of these users, with issues ranging from text-ordering errors, to inaccurate inferences (e.g., incorrectly categorizing a Caesar salad with anchovies as vegetarian), to misinterpretation of symbols and legends. Moreover, the users expressed a need for a screen reader-tailored interface to access the information in menus. To address these access barriers and users’ needs, we present AccessMenu, a browser extension that automatically detects visual menus in restaurant websites, uses multi-modal large language models to extract and analyze the menu content, and re-renders it in a conveniently navigable HTML format accessible with screen readers. AccessMenu also enables BVI users to issue natural language queries, allowing them to efficiently distill specific information from the menus. In a user evaluation with 10 blind participants, AccessMenu significantly outperformed a state-of-the-art solution in usability and task workload, by providing convenient menu navigation and query-based menu filtering capabilities.
Rights
© 2025 Copyright held by the owner/authors.
This work is licensed under a Creative Commons Attribution International 4.0 (CC BY 4.0) License.
Original Publication Citation
Venkatraman, N., Kolgar Nayak, A., Dahal, S., Prakash, Y., Lee, H. N., & Ashok, V. (2025). In B. Norton & T. Kinsmill (Eds.), W4A '25: Proceedings of the 22nd International Web for All Conference (pp. 36-47). Association for Computing Machinery. https://doi.org/10.1145/3744257.3744275
Repository Citation
Venkatraman, N., Kolgar Nayak, A., Dahal, S., Prakash, Y., Lee, H. N., & Ashok, V. (2025). In B. Norton & T. Kinsmill (Eds.), W4A '25: Proceedings of the 22nd International Web for All Conference (pp. 36-47). Association for Computing Machinery. https://doi.org/10.1145/3744257.3744275
ORCID
0009-0008-5428-5052 (Venkatraman), 0009-0000-9992-9706 (Nayak), 0009-0003-3785-6013 (Dahal), 0000-0001-8593-327X (Prakash), 0000-0002-4772-1265 (Ashok)