AI DOCTOR – VOICE & VISION

Abstract

The rapid growth of AI-driven digital health platforms and increasing complexity of healthcare accessibility challenges have created significant demand for intelligent, multimodal healthcare assistants capable of delivering real-time medical guidance. Traditional healthcare systems often fail to bridge the gap between patients and timely medical support, relying on manual processes that are slow and limited in scope. This paper presents the design and implementation of an AI-Powered Multimodal Healthcare Assistant, AI Doctor – Voice & Vision, which leverages Multi-Agent AI architecture and RetrievalAugmented Generation (RAG) to automate end-to-end preliminary healthcare guidance. The system integrates a Voice Processing Agent, NLP Agent, Computer Vision Agent, and Disease Prediction Agent, orchestrated through a sequential pipeline. The NLP Agent processes symptoms extracted from voice or text input. The Computer Vision Agent analyses uploaded medical images using CNN models for disease detection. The Disease Prediction Agent synthesizes findings and generates healthcare suggestions via RAG-enhanced knowledge retrieval. A Gradio-based frontend provides voice input, image upload, diagnosis results, and text-to-speech responses. Results demonstrate effective multimodal interaction with approximately 4-second end-to-end pipeline execution and 89% average RAG retrieval relevance.