Build notes ยท Voice biomarker prototype

How a voice becomes a screening signal.

An experimental pipeline that listens to a short voice sample three different ways. One model hears how you sound. Another captures what you said. A third reasons over both and names where they agree and where they diverge.

Experimental prototype. This produces a screening signal, not a medical diagnosis. It is not HIPAA-compliant and has never been used with real patient data.

The prototype in use. Upload, analysis (sped up), and results. Playback does not start automatically when reduced motion is requested.
Infographic of the voice biomarker pipeline. Top tier: speak, normalize, then two parallel lanes, how they sound and what they said, merging into what it may mean together. Bottom tier: browser and static hosting, CDN, compute server with the acoustic model, managed transcription, and a reasoning model, with a five-layer security band along the request path.
Select the image to open it full size.

The idea

Three independent systems. One reconciled view.

Acoustic and semantic evidence stay in separate lanes and only meet at the end, in the open, where disagreement is visible.

How they sound

Acoustic model

Kintsugi's open model scores tone, pace, and vocal patterns for depression and anxiety. It never sees a single word.

What they said

Transcription

Amazon Transcribe converts the same audio to text. It has no idea what the acoustic model concluded.

What it may mean together

Reasoning model

Claude on Amazon Bedrock compares both lanes, names agreement and divergence, and never diagnoses.

Security

A prototype still gets defense in depth.

  1. Password gate. Limits who can run an analysis at all.
  2. HTTPS only. The browser never speaks plain HTTP to anything.
  3. Network firewall. The server accepts traffic only from CloudFront.
  4. Origin secret. Proves a request came through this specific CDN. The app refuses to start without it.
  5. No retention. Staged audio is deleted as soon as transcription finishes.

Design decisions

The small calls that matter.

  • Asymmetric scales stay asymmetric. Depression has three bands, anxiety has four, so each gets its own component.
  • Disagreement is a feature. When tone and words diverge, the interpretation says so instead of averaging it away.
  • Show the raw output. Raw scores sit beside the interpretation, so nobody has to take the language model's word for it.

Before production

What a real deployment would add.

  • A HIPAA-eligible architecture, business associate agreements, and end-to-end audit logging
  • Per-user authentication and rate limiting in place of a shared password
  • Scale-to-zero compute and clinical validation before any score informs a real decision

See it live

Want a walkthrough?

The live demo is password protected. If you are working through how to put AI into a regulated care environment, I am happy to show it and talk through the tradeoffs.

Book a conversation Open the live demo

Password protected. Access on request.

Back to all demos