Skip to content

Commit 4ceda37

Browse files
santiatpmlsantiatpml
authored
pgml chat opensourceai (#1238)
1 parent f7401b8 commit 4ceda37

File tree

5 files changed

+838
-451
lines changed

5 files changed

+838
-451
lines changed

‎pgml-apps/pgml-chat/.env.template

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,5 @@ DATABASE_URL=<POSTGRES_DATABASE_URL starts with postgres://>
33

44
SLACK_BOT_TOKEN=<SLACK_BOT_TOKEN>
55
SLACK_APP_TOKEN=<SLACK_APP_TOKEN>
6-
DISCORD_BOT_TOKEN=<DISCORD_BOT_TOKEN>
6+
DISCORD_BOT_TOKEN=<DISCORD_BOT_TOKEN>
7+
SYSTEM_PROMPT_TEMPLATE=<SYSTEM PROMPT FOR CHAT COMPLETION MODEL. Check prompts.md file for examples>

‎pgml-apps/pgml-chat/README.md

Lines changed: 41 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ A command line tool to build and deploy a **_knowledge based_** chatbot using Po
33

44
There are two stages in building a knowledge based chatbot:
55
- Build a knowledge base by ingesting documents, chunking documents, generating embeddings and indexing these embeddings for fast query
6-
- Generate responses to user queries by retrieving relevant documents and generating responses using OpenAI API
6+
- Generate responses to user queries by retrieving relevant documents and generating responses using OpenAI and [OpenSourceAI API](https://postgresml.org/docs/introduction/apis/client-sdks/opensourceai)
77

88
This tool automates the above two stages and provides a command line interface to build and deploy a knowledge based chatbot.
99

@@ -12,7 +12,7 @@ Before you begin, make sure you have the following:
1212

1313
- PostgresML Database: Sign up for a free [GPU-powered database](https://postgresml.org/signup)
1414
- Python version >=3.8
15-
- OpenAI API key
15+
- (Optional) OpenAI API key
1616

1717

1818
# Getting started
@@ -30,24 +30,24 @@ wget https://raw.githubusercontent.com/postgresml/postgresml/master/pgml-apps/pg
3030
```
3131
3. Copy the template file to `.env`
3232

33-
4. Update environment variables with your OpenAI API key and PostgresML database credentials.
33+
4. Update environment variables with your PostgresML database credentials and OpenAI API key (optional).
3434
```bash
35-
OPENAI_API_KEY=<OPENAI_API_KEY>
3635
DATABASE_URL=<POSTGRES_DATABASE_URL starts with postgres://>
36+
OPENAI_API_KEY=<OPENAI_API_KEY> # Optional
3737
```
3838

3939
# Usage
4040
You can get help on the command line interface by running:
4141

4242
```bash
4343
(pgml-bot-builder-py3.9) pgml-chat % pgml-chat % pgml-chat --help
44-
usage: pgml-chat [-h] --collection_name COLLECTION_NAME [--root_dir ROOT_DIR] [--stage {ingest,chat}] [--chat_interface {cli,slack,discord}]
45-
[--chat_history CHAT_HISTORY] [--bot_name BOT_NAME] [--bot_language BOT_LANGUAGE] [--bot_topic BOT_TOPIC]
46-
[--bot_topic_primary_language BOT_TOPIC_PRIMARY_LANGUAGE] [--bot_persona BOT_PERSONA]
44+
usage: pgml-chat [-h] --collection_name COLLECTION_NAME [--root_dir ROOT_DIR] [--stage {ingest,chat}] [--chat_interface {cli,slack,discord}] [--chat_history CHAT_HISTORY] [--bot_name BOT_NAME]
45+
[--bot_language BOT_LANGUAGE] [--bot_topic BOT_TOPIC] [--bot_topic_primary_language BOT_TOPIC_PRIMARY_LANGUAGE] [--bot_persona BOT_PERSONA]
46+
[--chat_completion_model CHAT_COMPLETION_MODEL] [--max_tokens MAX_TOKENS] [--vector_recall_limit VECTOR_RECALL_LIMIT]
4747

4848
PostgresML Chatbot Builder
4949

50-
optional arguments:
50+
options:
5151
-h, --help show this help message and exit
5252
--collection_name COLLECTION_NAME
5353
Name of the collection (schema) to store the data in PostgresML database (default: None)
@@ -57,16 +57,21 @@ optional arguments:
5757
--chat_interface {cli,slack,discord}
5858
Chat interface to use (default: cli)
5959
--chat_history CHAT_HISTORY
60-
Number of messages from history used for generating response (default: 1)
60+
Number of messages from history used for generating response (default: 0)
6161
--bot_name BOT_NAME Name of the bot (default: PgBot)
6262
--bot_language BOT_LANGUAGE
6363
Language of the bot (default: English)
6464
--bot_topic BOT_TOPIC
6565
Topic of the bot (default: PostgresML)
6666
--bot_topic_primary_language BOT_TOPIC_PRIMARY_LANGUAGE
67-
Primary programming language of the topic (default: )
67+
Primary programming language of the topic (default: SQL)
6868
--bot_persona BOT_PERSONA
6969
Persona of the bot (default: Engineer)
70+
--chat_completion_model CHAT_COMPLETION_MODEL
71+
--max_tokens MAX_TOKENS
72+
Maximum number of tokens to generate (default: 256)
73+
--vector_recall_limit VECTOR_RECALL_LIMIT
74+
Maximum number of documents to retrieve from vector recall (default: 1)
7075
```
7176
## Ingest
7277
In this step, we ingest documents, chunk documents, generate embeddings and index these embeddings for fast query.
@@ -146,6 +151,32 @@ Once the discord app is running, you can interact with the chatbot on Discord as
146151
147152
![Discord Chatbot](./images/discord_screenshot.png)
148153
154+
# Prompt Engineering
155+
In addition to relevant context retrieved from vector search, system prompt to generate accurate responses with minimum hallucinations requires prompt engineering.
156+
Different chat completion models require different system prompts. Since the prompts including the context are long, they suffer from **lost in the middle** problem described in [this paper](https://arxiv.org/pdf/2307.03172.pdf). Below are some of the prompts that we have used for different chat completion models.
157+
158+
## Default prompt (GPT-3.5 and open source models)
159+
```text
160+
Use the following pieces of context to answer the question at the end.
161+
If you don't know the answer, just say that you don't know, don't try to make up an answer.
162+
Use three sentences maximum and keep the answer as concise as possible.
163+
Always say "thanks for asking!" at the end of the answer.
164+
```
165+
166+
## GPT-4 System prompt
167+
```text
168+
You are an assistant to answer questions about {topic}.\
169+
Your name is {name}. You speak like {persona} in {language}. Use the given list of documents to answer user's question.\
170+
Use the conversation history if it is applicable to answer the question. \n Use the following steps:\n \
171+
1. Identify if the user input is really a question. \n \
172+
2. If the user input is not related to the {topic} then respond that it is not related to the {topic}.\n \
173+
3. If the user input is related to the {topic} then first identify relevant documents from the list of documents. \n \
174+
4. If the documents that you found relevant have information to completely and accurately answers the question then respond with the answer.\n \
175+
5. If the documents that you found relevant have code snippets then respond with the code snippets. \n \
176+
6. Most importantly, don't make up code snippets that are not present in the documents.\n \
177+
7. If the user input is generic like Cool, Thanks, Hello, etc. then respond with a generic answer. \n"
178+
```
179+
149180
# Developer Guide
150181
151182
1. Clone this repository, start a poetry shell and install dependencies

0 commit comments

Comments
 (0)