Serverless Azure OpenAI Quick Start with LlamaIndex

(Python)

This sample shows how to quickly get started with LlamaIndex.ai on Azure. The application is hosted on Azure Container Apps. You can use it as a starting point for building more complex RAG applications.

(Like and fork this sample to receive lastest changes and updates)

Features

This project demonstrates how to build a simple LlamaIndex application using Azure OpenAI. The app is set up as a chat interface that can answer questions about your data. You can add arbitrary data sources to your chat, like local files, websites, or data retrieved from a database. The app will ingest any supported files you put in ./data/ directory. This sample app includes an example pdf in the data folder that contains information about standards for sending letters, cards, flats, and parcels in the mail. The app also uses LlamaIndex.TS that is able to ingest any PDF, text, CSV, Markdown, Word and HTML files.

Architecture Diagram

• This application has two main components:

• A Python backend built using FastAPI

• A Javascript frontend built with Next.js

It is hosted on Azure Container Apps in just a few commands.

• The app uses Azure OpenAI to answer questions about the data you provide. The app is set up to use the gpt-35-turbo model and embeddings to provide the best and fastest answers to your questions.

Getting Started

You have a few options for getting started with this template. The quickest way to get started is GitHub Codespaces, since it will setup all the tools for you, but you can also set it up locally. You can also use a VS Code dev container

This template uses gpt-35-turbo version 1106 which may not be available in all Azure regions. Check for up-to-date region availability and select a region during deployment accordingly. We recommend using swedencentral.

GitHub Codespaces

You can run this template virtually by using GitHub Codespaces. The button will open a web-based VS Code instance in your browser:

1. Open the template (this may take several minutes)
2. Open a terminal window

3. Sign into your Azure account:

   shell azd auth login 4. Provision the Azure resources and deploy your code:

   shell azd up

   Once your deployment is complete you can begin to set up your python environment.

4. Create a python virtual environment and install the python dependencies:

   Linux and MacOS venv activation: bash cd backend python3 -m venv venv source venv/bin/activate

   Install dependencies with poetry: bash poetry install

   You will also need to ensure the environment variables are accessible. You can do this by running the following command:

   bash azd env get-values > .env Confirm that this step has happened successfuly by checking if a .env file has been added to the backend folder.

5. We can now generate the embeddings of the documents in the ./data directory. In this sample it contains a pdf file with mail standards.

   bash poetry run generate

6. Next, we can install the frontend dependencies:

   bash cd ../frontend npm install

The app is now ready to run! To test it, run the following commands:

1. First start the Flask server bash cd ../backend python main.py (If you see a Traceloop error ignore it as we will not be using it for this example.)

2. Make ports in Github Codespaces public

   Because the Flask server and the frontend web app server are running on different ports, you will need to use public ports in codespaces. To do this look for the ports tab at the top of your terminal in vscode. If the port visibilities of the available ports are already public skip this step. If they are private look for port 8000, right click on it, select Port Visibility and set it to public. Do the same for port 3000.

   

3. Next open a new terminal and launch the web app

bash cd frontend npm run dev

Open the URL http://localhost:3000 in your browser to interact with the bot.

Congratulations! Your RAG app is now working. An example question to ask is 'Can you tell me how much it costs to send a large parcel to France?'

VS Code Dev Containers

A related option is VS Code Dev Containers, which will open the project in your local VS Code using the Dev Containers extension:

1. Start Docker Desktop (install it if not already installed)
2. Open the project:
3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window.

4. Sign into your Azure account:

   shell azd auth login 5. Provision the Azure resources and deploy your code:

   shell azd up

Once your deployment is complete, you should see a .env file in the .azure\env_name folder. This file contains the environment variables needed to run the application using Azure resources. Move this file to the backend\app folder for the variables to be loaded into the correct enivornment.

1. Create a python virtual environment and install the python dependencies:

   bash cd backend python3 -m venv venv source venv/bin/activate poetry install

   You will also need to ensure the environment variables are accessible. You can do this by running the following command:

   bash azd env get-values > .env Confirm that this step has happened successfuly by checking if a .env file has been added to the backend folder.

2. We can now generate the embeddings of the documents in the ./data directory. In this sample it contains a pdf file with mail standards.

   bash poetry run generate

3. Install the frontend dependencies:

   bash cd .. cd frontend npm install 9. Configure a CI/CD pipeline:

   shell azd pipeline config

The app is now ready to run! To test it, run the following commands:

1. First run the Flask development server

cd ../backend
python main.py

1. Next open a new terminal and launch the web app

cd frontend
npm run dev

Open the URL http://localhost:3000 in your browser to interact with the bot. An example question to ask is 'Can you tell me how much it costs to send a large parcel to France?'

Local Environment

Prerequisites

You need to install following tools to work on your local machine:

• Python 3.9, 3.10, or 3.11
• Poetry
• Node.js LTS
• Azure Developer CLI
• Git
• PowerShell 7+ (for Windows users only)
• Important: Ensure you can run pwsh.exe from a PowerShell command. If this fails, you likely need to upgrade PowerShell.
• Instead of Powershell, you can also use Git Bash or WSL to run the Azure Developer CLI commands.
• This template uses gpt-35-turbo version 1106 which may not be available in all Azure regions. Check for up-to-date region availability and select a region during deployment accordingly
• We recommend using swedencentral

Then you can get the project code:

1. Fork the project to create your own copy of this repository.
2. On your forked repository, select the Code button, then the Local tab, and copy the URL of your forked repository.
3. Open a terminal and run this command to clone the repo: git clone <your-repo-url>

Using the template locally

1. Bring down the template code:

   shell azd init --template llama-index-python

   This will perform a git clone

2. Sign into your Azure account:

   shell azd auth login

3. Create a python virtual environment and install the python dependencies:

   bash cd backend python3 -m venv venv source venv/bin/activate poetry install

4. Provision and deploy the project to Azure:

   shell azd up You will also need to ensure the environment variables are accessible. You can do this by running the following command:

   bash azd env get-values > .env Confirm that this step has happened successfuly by checking if a .env file has been added to the backend folder.

5. We can now generate the embeddings of the documents in the ./data directory. In this sample it contains a pdf file with mail standards.

   bash poetry run generate

6. Install the frontend dependencies:

   bash cd .. cd frontend npm install

7. Configure a CI/CD pipeline:

   shell azd pipeline config

The app is now ready to run! To test it, run the following commands:

1. First run the Flask development server bash cd ../backend python main.py

2. Next open a new terminal and launch the web app

bash cd frontend npm run dev

Open the URL http://localhost:3000 in your browser to interact with the bot. An example question to ask is 'Can you tell me how much it costs to send a large parcel to France?'

Personalize The App

1. Use your own data:
2. Add any data you want to include in the ./backend/data folder.
3. cd ./backend and then run poetry run generate
4. then run the fastapi dev server using python [main.py](http://main.py/)

5. open a new terminal cd into frontend and run npm run dev

6. Change the look of the app:

7. Change app name in header.tsx
8. Change app avatar by adding a new image in ./frontend/public and replace the places in header.tsx and chat-avatar.tsx that have llama.png with your image name.
9. Edit colors on the page in global.css , background colors can be changed by making changes to .background-gradient

Guidance

Region Availability

This template uses gpt-35-turbo version 1106 which may not be available in all Azure regions. Check for up-to-date region availability and select a region during deployment accordingly * We recommend using swedencentral

Costs

Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. However, you can use the Azure pricing calculator fo