Document Type

Article

Publication Date

2026

DOI

10.1007/s10278-026-02111-x

Publication Title

Journal of Imaging Informatics in Medicine

Pages

26 pp.

Abstract

In recent years, vision-language models (VLMs) have been applied to various fields, including healthcare, education, finance, and manufacturing, with remarkable performance. However, concerns remain regarding VLMs’ consistency and uncertainty, particularly in critical applications such as healthcare, which demand a high level of trust and reliability. This paper proposes a novel approach to evaluate uncertainty in VLMs’ responses using a convex hull approach on a healthcare application for visual question answering (VQA). For any VLM, temperature refers to a sampling parameter used in probabilistic generation, which controls the randomness of the model’s output. The LLM-CXR model is selected as the medical VLM utilized to generate responses for a given prompt at different temperature settings. According to the results, the LLM-CXR VLM shows high uncertainty at higher temperature settings, which can be characterized geometrically in feature space. Experimental results emphasize the importance of uncertainty in VLMs’ responses, especially in healthcare applications.

Rights

© 2026 The Authors.

This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if you modified the licensed material. You do not have permission under this license to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Data Availability

Article states: "The datasets used in this study are publicly available. The source code used for uncertainty analysis is available at: https://github.com/ocatak/VLMUncertainty."

Original Publication Citation

Catak, F. O., Kuzlu, M., Patrick, T., & Audette, M. (2026). Improving medical diagnostics with vision-language models: Convex hull-based uncertainty analysis. Journal of Imaging Informatics in Medicine. Advance online publication. https://doi.org/10.1007/s10278-026-02111-x

ORCID

0000-0002-8719-2353 (Kuzlu), 0000-0003-0011-1731 (Audette)

Download

Share

COinS