Sign in

Azure-Samples /
azure-search-openai-demo Public

Code Issues 499

Files azure-
main
search-
openai-
demo
/
README.m
d

pamelafox last week

281 lines (188 loc) · 19.4 KB

Preview Code Blame

ChatGPT-like app
with your data
using Azure
OpenAI and Azure
AI Search
(Python)
This solution's backend is written in
Python. There are also JavaScript,
.NET, and Java samples based on this
one. Learn more about developing AI
apps using Azure AI Services.

GITHUB CODESPACES OPEN

DEV CONTAINERS OPEN

Table of Contents

Features
Azure account requirements
Cost estimation
Getting Started
GitHub Codespaces
VS Code Dev Containers
Local environment
Deploying
Deploying again
Sharing environments
Running locally
Using the app
Clean up
Guidance
Customizing the UI and data
Monitoring with Application
Insights
Productionizing
Troubleshooting
Resources

📺 Watch a video overview of the app.

This sample demonstrates a few

approaches for creating ChatGPT-like
experiences over your own data using
the Retrieval Augmented Generation
pattern. It uses Azure OpenAI Service
to access a GPT model (gpt-35-turbo),
and Azure AI Search for data indexing
and retrieval.

The repo includes sample data so it's

ready to try end to end. In this sample
application we use a fictitious company
called Contoso Electronics, and the
experience allows its employees to ask
questions about the benefits, internal
policies, as well as job descriptions and
roles.

Features
Chat (multi-turn) and Q&A (single
turn) interfaces
Renders citations and thought
process for each answer
Includes settings directly in the UI
to tweak the behavior and
experiment with options
Integrates Azure AI Search for
indexing and retrieval of
documents, with support for many
document formats as well as
integrated vectorization
Optional usage of GPT-4 with
vision to reason over image-heavy
documents
Optional addition of speech
input/output for accessibility
Optional automation of user login
and data access via Microsoft Entra
Performance tracing and
monitoring with Application Insights

Architecture Diagram

Azure account
requirements
IMPORTANT: In order to deploy and run
this example, you'll need:

Azure account. If you're new to

Azure, get an Azure account for
free and you'll get some free Azure
credits to get started. See guide to
deploying with the free trial.
Azure subscription with access
enabled for the Azure OpenAI
service. You can request access
with this form. If your access
request to Azure OpenAI service
doesn't match the acceptance
criteria, you can use OpenAI public
API instead. Learn how to switch to
an OpenAI instance.
Azure account permissions:
Your Azure account must have
Microsoft.Authorization/role
Assignments/write
permissions, such as Role
Based Access Control
Administrator, User Access
Administrator, or Owner. If you
don't have subscription-level
permissions, you must be
granted RBAC for an existing
resource group and deploy to
that existing group.
Your Azure account also needs
Microsoft.Resources/deployme
nts/write permissions on the
subscription level.

Cost estimation
Pricing varies per region and usage, so
it isn't possible to predict exact costs for
your usage. However, you can try the
Azure pricing calculator for the
resources below.

Azure App Service: Basic Tier with

1 CPU core, 1.75 GB RAM. Pricing
per hour. Pricing
Azure OpenAI: Standard tier, GPT
and Ada models. Pricing per 1K
tokens used, and at least 1K tokens
are used per question. Pricing
Azure AI Document Intelligence:
SO (Standard) tier using pre-built
layout. Pricing per document page,
sample documents have 261 pages
total. Pricing
Azure AI Search: Basic tier, 1
replica, free level of semantic
search. Pricing per hour. Pricing
Azure Blob Storage: Standard tier
with ZRS (Zone-redundant
storage). Pricing per storage and
read operations. Pricing
Azure Monitor: Pay-as-you-go tier.
Costs based on data ingested.
Pricing

To reduce costs, you can switch to free

SKUs for various services, but those
SKUs have limitations. See this guide
on deploying with minimal costs for
more details.

⚠️To avoid unnecessary costs,

remember to take down your app if it's
no longer in use, either by deleting the
resource group in the Portal or running
azd down .

Getting Started
You have a few options for setting up
this project. The easiest way to get
started is GitHub Codespaces, since it
will setup all the tools for you, but you
can also set it up locally if desired.

GitHub Codespaces
You can run this repo virtually by using
GitHub Codespaces, which will open a
web-based VS Code in your browser:

GITHUB CODESPACES OPEN

Once the codespace opens (this may

take several minutes), open a terminal
window.

VS Code Dev Containers

A related option is VS Code Dev
Containers, which will open the project
in your local VS Code using the Dev
Containers extension:

1. Start Docker Desktop (install it if

not already installed)

2. Open the project:

DEV CONTAINERS OPEN

3. In the VS Code window that opens,

once the project files show up (this
may take several minutes), open a
terminal window.

Local environment
1. Install the required tools:

Azure Developer CLI

Python 3.9, 3.10, or 3.11
Important: Python and
the pip package manager
must be in the path in
Windows for the setup
scripts to work.
Important: Ensure you
can run python --version
from console. On Ubuntu,
you might need to run
sudo apt install python-
is-python3 to link python
to python3 .
Node.js 18+
Git
Powershell 7+ (pwsh) - For
Windows users only.
Important: Ensure you
can run pwsh.exe from a
PowerShell terminal. If
this fails, you likely need
to upgrade PowerShell.

2. Create a new folder and switch to it

in the terminal.

3. Run this command to download the

project code:

azd init -t azure-search-o

Note that this command will

initialize a git repository, so you do
not need to clone this repository.

Deploying

Follow these steps to provision Azure

resources and deploy the application
code:

1. Login to your Azure account:

azd auth login

2. Create a new azd environment:

azd env new

Enter a name that will be used for

the resource group. This will create
a new folder in the .azure folder,
and set it as the active
environment for any calls to azd
going forward.

3. (Optional) This is the point where

you can customize the deployment
by setting environment variables, in
order to use existing resources,
enable optional features (such as
auth or vision), or deploy to free
tiers.

4. Run azd up - This will provision

Azure resources and deploy this
sample to those resources,
including building the search index
based on the files found in the
./data folder.

Important: Beware that the

resources created by this
command will incur immediate
costs, primarily from the AI
Search resource. These
resources may accrue costs
even if you interrupt the
command before it is fully
executed. You can run azd
down or delete the resources
manually to avoid
unnecessary spending.
You will be prompted to select
two locations, one for the
majority of resources and one
for the OpenAI resource, which
is currently a short list. That
location list is based on the
OpenAI model availability table
and may become outdated as
availability changes.

5. After the application has been

successfully deployed you will see
a URL printed to the console. Click
that URL to interact with the
application in your browser. It will
look like the following:

NOTE: It may take 5-10 minutes

after you see 'SUCCESS' for the
application to be fully deployed. If
you see a "Python Developer"
welcome screen or an error page,
then wait a bit and refresh the
page. See guide on debugging
App Service deployments.

Deploying again
If you've only changed the
backend/frontend code in the app
folder, then you don't need to re-
provision the Azure resources. You can
just run:

azd deploy

If you've changed the infrastructure files

( infra folder or azure.yaml ), then
you'll need to re-provision the Azure
resources. You can do that by running:

azd up

Sharing environments

To give someone else access to a

completely deployed and existing
environment, either you or they can
follow these steps:

1. Install the Azure CLI

2. Run azd init -t azure-search-
openai-demo or clone this
repository.
3. Run azd env refresh -e
{environment name} They will need
the azd environment name,
subscription ID, and location to run
this command. You can find those
values in your .azure/{env
name}/.env file. This will populate
their azd environment's .env file
with all the settings needed to run
the app locally.
4. Set the environment variable
AZURE_PRINCIPAL_ID either in that
.env file or in the active shell to
their Azure ID, which they can get
with az ad signed-in-user show .
5. Run ./scripts/roles.ps1 or
.scripts/roles.sh to assign all of
the necessary roles to the user. If
they do not have the necessary
permission to create roles in the
subscription, then you may need to
run this script for them. Once the
script runs, they should be able to
run the app locally.

Running locally

You can only run locally after having

successfully run the azd up command.
If you haven't yet, follow the deploying
steps above.

1. Run azd auth login

2. Change dir to app
3. Run ./start.ps1 or ./start.sh
or run the "VS Code Task: Start
App" to start the project locally.

See more tips in the local development

guide.

Using the app

In Azure: navigate to the Azure

WebApp deployed by azd. The
URL is printed out when azd
completes (as "Endpoint"), or you
can find it in the Azure portal.
Running locally: navigate to
127.0.0.1:50505

Once in the web app:

Try different topics in chat or Q&A

context. For chat, try follow up
questions, clarifications, ask to
simplify or elaborate on answer,
etc.
Explore citations and sources
Click on "settings" to try different
options, tweak prompts, etc.

Clean up
To clean up all the resources created by
this sample:

1. Run azd down

2. When asked if you are sure you
want to continue, enter y
3. When asked if you want to
permanently delete the resources,
enter y

The resource group and all the

resources will be deleted.

Guidance

Besides the tips below, you can find

extensive documentation in the docs
folder.

Customizing the UI and data

Once you successfully deploy the app,
you can start customizing it for your
needs: changing the text, tweaking the
prompts, and replacing the data.
Consult the app customization guide as
well as the data ingestion guide for
more details.

Monitoring with Application

Insights
By default, deployed apps use
Application Insights for the tracing of
each request, along with the logging of
errors.

To see the performance data, go to the

Application Insights resource in your
resource group, click on the "Investigate
-> Performance" blade and navigate to
any HTTP request to see the timing
data. To inspect the performance of chat
requests, use the "Drill into Samples"
button to see end-to-end traces of all
the API calls made for any chat request:

To see any exceptions and server

errors, navigate to the "Investigate ->
Failures" blade and use the filtering
tools to locate a specific exception. You
can see Python stack traces on the
right-hand side.

You can also see chart summaries on a

dashboard by running the following
command:

azd monitor

Productionizing
This sample is designed to be a starting
point for your own production
application, but you should do a
thorough review of the security and
performance before deploying to
production. Read through our
productionizing guide for more details.

Troubleshooting
Here are the most common failure
scenarios and solutions:

1. The subscription
( AZURE_SUBSCRIPTION_ID ) doesn't
have access to the Azure OpenAI
service. Please ensure
AZURE_SUBSCRIPTION_ID matches
the ID specified in the OpenAI
access request process.

2. You're attempting to create

resources in regions not enabled
for Azure OpenAI (e.g. East US 2
instead of East US), or where the
model you're trying to use isn't
enabled. See this matrix of model
availability.

3. You've exceeded a quota, most

often number of resources per
region. See this article on quotas
and limits.

4. You're getting "same resource

name not allowed" conflicts. That's
likely because you've run the
sample multiple times and deleted
the resources you've been creating
each time, but are forgetting to
purge them. Azure keeps resources
for 48 hours unless you purge from
soft delete. See this article on
purging resources.

5. You see
CERTIFICATE_VERIFY_FAILED when
the prepdocs.py script runs.
That's typically due to incorrect
SSL certificates setup on your
machine. Try the suggestions in
this StackOverflow answer.

6. After running azd up and visiting

the website, you see a '404 Not
Found' in the browser. Wait 10
minutes and try again, as it might
be still starting up. Then try
running azd deploy and wait
again. If you still encounter errors
with the deployed app, consult the
guide on debugging App Service
deployments. Please file an issue
if the logs don't help you resolve
the error.

Resources
Additional documentation for this
app
📖 Revolutionize your Enterprise
Data with ChatGPT: Next-gen Apps
w/ Azure OpenAI and AI Search
📖 Azure AI Search
📖 Azure OpenAI Service
📖 Comparing Azure OpenAI and
OpenAI
📖 Access Control in Generative AI
applications with Azure Cognitive
Search
📺 Quickly build and deploy OpenAI
apps on Azure, infused with your
own data
📺 AI Chat App Hack series

Getting help
This is a sample built to demonstrate
the capabilities of modern Generative AI
apps and how they can be built in
Azure. For help with deploying this
sample, please post in GitHub Issues. If
you're a Microsoft employee, you can
also post in our Teams channel.

This repository is supported by the

maintainers, not by Microsoft Support,
so please use the support mechanisms
described above, and we will do our
best to help you out.

Note
Note: The PDF documents used in
this demo contain information
generated using a language model
(Azure OpenAI Service). The
information contained in these
documents is only for
demonstration purposes and does
not reflect the opinions or beliefs of
Microsoft. Microsoft makes no
representations or warranties of
any kind, express or implied, about
the completeness, accuracy,
reliability, suitability or availability
with respect to the information
contained in this document. All
rights reserved to Microsoft.