hub / github.com/RetellAI/retell-custom-llm-node-demo

github.com/RetellAI/retell-custom-llm-node-demo @main

68 symbols 107 edges 10 files 0 documented · 0%

What it actually does AI analysis from the code graph — generated when you open this

loading…

README

retell-custom-llm-node-demo

This backend repo demonstrates how to start a WebSocket server that Retell server will connect to. Retell will send live transcripts and other updates to the server, and get responses from this server. see API Docs for walkthrough.

The protocol of messages we send and expect to receive is documented here.

To set up inbound, make phone calls, see API Docs for a walkthrough.

This repo contains azure OpenAI, OpenAI, and OpenRouter, modify the import inside src/server.ts to switch between which one to use.

Check this Youtube Tutorial containing a walkthrough using the Frontend Demo and this repo.

Steps to run locally to test

Add Retell and your LLM API key (Azure OpenAI / OpenAI / OpenRouter) to ".env.development".
Azure OpenAI is pretty fast and stable: guide for setup
OpenAI is the most widely used one, although the latency can vary.
OpenRouter allows you to choose between tons of Open Source AI Models.
Install dependencies

npm install

In another bash, use ngrok to expose this port to the public network

ngrok http 8080

Start the server

npm run dev

You should see a fowarding address like https://dc14-2601-645-c57f-8670-9986-5662-2c9a-adbd.ngrok-free.app, and you are going to take the hostname dc14-2601-645-c57f-8670-9986-5662-2c9a-adbd.ngrok-free.app, prepend it with wss://, postpend with /llm-websocket (the route setup to handle LLM websocket connection in the code) to create the url to use in the dashboard to create a new agent. Now the agent you created should connect with your localhost.

The custom LLM URL would look like wss://dc14-2601-645-c57f-8670-9986-5662-2c9a-adbd.ngrok-free.app/llm-websocket

Run in prod

To run in prod, you probably want to customize your LLM solution, host the code in a cloud, and use that IP to create the agent.

Extension points exported contracts — how you extend this code

Utterance (Interface)

(no doc)

src/types.ts

PingPongRequest (Interface)

(no doc)

src/types.ts

CallDetailsRequest (Interface)

(no doc)

src/types.ts

UpdateOnlyRequest (Interface)

(no doc)

src/types.ts

ResponseRequiredRequest (Interface)

(no doc)

src/types.ts

Core symbols most depended-on inside this repo

handleRetellLlmWebSocket

src/llms/llm_openai_func_call.ts

ConversationToChatRequestMessages

called by 1

src/llms/llm_openai_func_call.ts

PreparePrompt

called by 1

src/llms/llm_openai_func_call.ts

DraftResponse

called by 1

src/llms/llm_openai_func_call.ts

ConversationToChatRequestMessages

called by 1

src/llms/llm_openai.ts

Shape

Method 39

Class 16

Interface 13

Languages

TypeScript100%

Modules by API surface

src/types.ts13 symbols

src/llms/llm_azure_openai_func_call_end_call.ts8 symbols

src/llms/llm_azure_openai_func_call.ts8 symbols

src/llms/llm_openrouter.ts7 symbols

src/llms/llm_openai_func_call.ts7 symbols

src/llms/llm_openai.ts7 symbols

src/llms/llm_azure_openai.ts7 symbols

src/server.ts6 symbols

src/llms/llm_dummy_mock.ts5 symbols

For agents

$ claude mcp add retell-custom-llm-node-demo \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact