Brief #18 · AI & LLM
Bangla Voice Assistant
Medium⏱ 14-20 hSoon
A voice assistant you can speak to in Bangla: speech to text, an LLM answer, and speech back, all inside a strict latency budget so it feels like a conversation.
Outlines are free · building with tracking needs Pro
BThe briefFrom the team
Many of our customers would rather speak than type, mostly in Bangla, sometimes mixed with English. Our first prototype took six seconds to answer and people hung up. Build a voice assistant that starts answering in under 1.5 seconds and understands Bangla-English mixed speech.
What you'll build
- 1A browser page that streams microphone audio to your server over a WebSocket.
- 2Speech-to-text with voice activity detection, so you know when the person has stopped speaking.
- 3Stream the LLM's answer and start text-to-speech on the first sentence instead of waiting for the whole reply.
- 4Let the user interrupt: when they start speaking, stop the audio and listen.
- 5Log a timing breakdown for every turn: listening end, transcript, first token, first audio.
Step-by-step walkthrough
Walkthrough coming soon
This project's step-by-step walkthrough, with real outputs at every step, is still being written. You can read the brief and try it on your own meanwhile; tracking opens when the walkthrough is ready.
Done when
- Median time from end of speech to first audio is under 1.5 s, measured over 50 turns.
- Word error rate measured on 50 recorded Bangla and mixed-language sentences you transcribed by hand.
- Interrupting the assistant stops its speech within 300 ms.
- A latency chart showing which stage of the pipeline takes the most time.
Stretch goals
- Run speech-to-text locally and compare accuracy and latency with a hosted API.
- Give it one tool, like checking an order status by voice.