MCPcopy Index your code
hub / github.com/Ejb503/ai-voice-generation

github.com/Ejb503/ai-voice-generation @main

Chat with this repo
repository ↗ · DeepWiki ↗ · + Follow
19 symbols 37 edges 13 files 0 documented · 0%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

AI Powered Voice Chat Demo

logo By tyingshoelaces.com License Contributors

Overview

This is a simple version of OpenAI's voice functionality using free APIs. This demo lets you talk, listen, and converse with LLMs.

Original blog post is here: - Blog: Blog Post Youtube video explainer is here: YouTube Video

Feel free to play around!

Tech Stack

  • LLM Host: Groq
  • LLM: LLAMA 3
  • TTS: DeepGram
  • STT: SpeechRecognition API
  • Web Framework: NextJS (React front-end, Express API)

How to use

  1. download the repo
  2. npm i
  3. setup .env.local with DEEPGRAM_API_KEY and GROQ_API_KEY
  4. npm run dev

You might want to edit all the prompts to change the tone of the response.

The architecture is simple, Voice -> Text -> LLM -> Text -> Voice. Rag and all sorts of fun creative things can be used to spice up the LLM.

Hints and tricks

You'll probably want to switch out SpeechRecognition for Whisper AI if you want non-chrome APIs or something more stable.

There is a lot of investment needed in handling state in the AudioPlayer, not necessary for this demo.

Playing with the prompts and context going to Groq is the key for personalisation.

Contact me for feedback!

What I Did

I built a demo where you can:

  1. Talk into the browser using the WebSpeechRecognitionAPI.
  2. Stream the transcribed text to Groq for processing.
  3. Stream the response from Groq to DeepGram for text-to-speech conversion.

  4. Play the generated audio response in the browser.

  5. NextJS: ★★★★★ - Wonderful technology, simplifies client and server-side development.

  6. Groq: ★★★★★ - New benchmarks in speed and cost.
  7. Llama3: ★★★★☆ - Noticeable difference from GPT-io, great for cheap requests and demos.
  8. DeepGram: ★★★☆☆ - Generous starting credits, good latency. Still green as a tech.

Links

Edward Ejb503, Tying Shoelaces Blog

Extension points exported contracts — how you extend this code

SpeechAnimationProps (Interface)
(no doc)
src/app/components/speech-animation.tsx
ControlsProps (Interface)
(no doc)
src/app/components/controls.tsx
Window (Interface)
(no doc)
src/app/hooks/useSpeechRecognition.ts

Core symbols most depended-on inside this repo

checkSpeechRecognitionAndMicrophonePermission
called by 1
src/app/components/audio-widget.tsx
postTranscript
called by 1
src/app/components/audio-widget.tsx
postSubscribe
called by 1
src/app/components/utils/utils.ts
postStream
called by 1
src/app/components/utils/utils.ts
extractJson
called by 1
src/app/components/utils/utils.ts
processAudioStream
called by 1
src/app/components/utils/utils.ts
useSpeechRecognition
called by 1
src/app/hooks/useSpeechRecognition.ts
stopRecognition
called by 1
src/app/hooks/useSpeechRecognition.ts

Shape

Function 16
Interface 3

Languages

TypeScript100%

Modules by API surface

src/app/components/utils/utils.ts4 symbols
src/app/hooks/useSpeechRecognition.ts3 symbols
src/app/components/audio-widget.tsx3 symbols
src/app/components/speech-animation.tsx2 symbols
src/app/components/controls.tsx2 symbols
src/app/page.tsx1 symbols
src/app/layout.tsx1 symbols
src/app/hooks/utils/utils.ts1 symbols
src/app/api/v1/groq/route.ts1 symbols
src/app/api/v1/deepgram/route.ts1 symbols

For agents

$ claude mcp add ai-voice-generation \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact

Ask about this repo answers extend the page