To help clients and audiences envision how artificial intelligence could transform healthcare, KPMG partnered with Eagerworks to build an AI-powered conversational medical triage application that reimagines the first point of contact between patients and medical systems.

 

Designed to be used at fairs, events, and client demonstrations, this immersive application simulates a medical triage process through natural, conversational interaction with a virtual medical assistant, bringing to life a realistic vision of future hospitals.

The challenge was to simulate a future-facing medical triage process fully driven by artificial intelligence, replacing the traditional initial patient assessment with a conversational AI system. 

 

The interaction needed to feel human and fluid, guided by an avatar that asks questions about symptoms, medical history, and medications, and then intelligently determines the most appropriate specialist for the patient. 

 

From a technical standpoint, the experience had to operate seamlessly across VR headsets, tablets, and mobile devices. The main technical challenges included:

 

- real-time audio processing

- accurately understanding spoken input

- generating coherent and context-aware responses

- returning them as natural-sounding speech

 

All of this needed to happen within a strict performance requirement: responses had to be generated in under six seconds from the moment the user finished speaking, to ensure a fluid, natural conversation that feels as close as to a real-life interaction as possible.

While the visual layer, avatar design, and user experience were developed by FS Studio, Eagerworks was responsible for building the intelligence behind the system, the “brain” of the experience. 

 

This involved designing and implementing the AI backbone that powers the entire conversational flow of the triage experience.

 

The system manages the complete interaction loop by:

 

- receiving audio input from the user

- processing and understanding speech through AI models

- applying conversational and triage logic

- generating coherent, context-aware responses

- transforming those responses into a realistic voice output that is delivered through the avatar

 

To achieve this, Eagerworks developed a robust and scalable architecture using:

 

- OpenAI models for language understanding and reasoning, 

- ElevenLabs for high-quality voice synthesis, 

- Python as the core language orchestrating the logic of the system

- AWS infrastructure was used to ensure performance, scalability, and reliability

 

The result is a real-time conversational AI system that feels natural, responsive, and technically credible as a vision of future healthcare.

 

KPMG now has a fully interactive AI-powered medical triage system that allows them to demonstrate, in a tangible and engaging way, how artificial intelligence can improve patient intake efficiency, decision-making, and patient experience in healthcare settings. 

 

Beyond its technical depth, the system became a powerful storytelling and sales asset for KPMG, helping position themselves as a forward-thinking leader in healthcare innovation while making complex AI concepts accessible and compelling for clients and stakeholders.