From f0ad312d0d25de6ee51750599542006baaa0ff26 Mon Sep 17 00:00:00 2001 From: Averi Kitsch Date: Thu, 7 Mar 2024 10:12:15 -0800 Subject: [PATCH] docs: add github links --- docs/chat_message_history.ipynb | 274 +++++++++++++++----------------- docs/document_loader.ipynb | 19 ++- 2 files changed, 144 insertions(+), 149 deletions(-) diff --git a/docs/chat_message_history.ipynb b/docs/chat_message_history.ipynb index 1ebce2a..16d4445 100644 --- a/docs/chat_message_history.ipynb +++ b/docs/chat_message_history.ipynb @@ -11,7 +11,11 @@ "\n", "> [Cloud SQL](https://cloud.google.com/sql) is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability. It offers MySQL, PostgreSQL, and SQL Server database engines. Extend your database application to build AI-powered experiences leveraging Cloud SQL's Langchain integrations.\n", "\n", - "This notebook goes over how to use `Cloud SQL for SQL Server` to store chat message history with the `MSSQLChatMessageHistory` class." + "This notebook goes over how to use `Cloud SQL for SQL Server` to store chat message history with the `MSSQLChatMessageHistory` class.\n", + "\n", + "Learn more about the package on [GitHub](https://github.com/googleapis/langchain-google-cloud-sql-mssql-python/).\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/googleapis/langchain-google-cloud-sql-mssql-python/blob/main/docs/chat_message_history.ipynb)" ] }, { @@ -24,7 +28,9 @@ "## Before You Begin\n", "\n", "To run this notebook, you will need to do the following:\n", + "\n", " * [Create a Google Cloud Project](https://developers.google.com/workspace/guides/create-project)\n", + " * [Enable the Cloud SQL Admin API.](https://console.cloud.google.com/marketplace/product/google/sqladmin.googleapis.com)\n", " * [Create a Cloud SQL for SQL Server instance](https://cloud.google.com/sql/docs/sqlserver/create-instance)\n", " * [Create a Cloud SQL database](https://cloud.google.com/sql/docs/sqlserver/create-manage-databases)\n", " * [Create a database user](https://cloud.google.com/sql/docs/sqlserver/create-manage-users) (Optional if you choose to use the `sqlserver` user)" @@ -32,83 +38,87 @@ }, { "cell_type": "markdown", - "source": [ - "### 🦜🔗 Library Installation\n", - "The integration lives in its own `langchain-google-cloud-sql-mssql` package, so we need to install it." - ], + "id": "Mm7-fG_LltD7", "metadata": { "id": "Mm7-fG_LltD7" }, - "id": "Mm7-fG_LltD7" + "source": [ + "### 🦜🔗 Library Installation\n", + "The integration lives in its own `langchain-google-cloud-sql-mssql` package, so we need to install it." + ] }, { "cell_type": "code", - "source": [ - "%pip install --upgrade --quiet langchain-google-cloud-sql-mssql langchain-google-vertexai" - ], + "execution_count": null, + "id": "1VELXvcj8AId", "metadata": { "id": "1VELXvcj8AId" }, - "id": "1VELXvcj8AId", - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-google-cloud-sql-mssql langchain-google-vertexai" + ] }, { "cell_type": "markdown", - "source": [ - "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top." - ], + "id": "98TVoM3MNDHu", "metadata": { "id": "98TVoM3MNDHu" }, - "id": "98TVoM3MNDHu" + "source": [ + "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top." + ] }, { "cell_type": "code", + "execution_count": null, + "id": "v6jBDnYnNM08", + "metadata": { + "id": "v6jBDnYnNM08" + }, + "outputs": [], "source": [ "# # Automatically restart kernel after installs so that your environment can access the new packages\n", "# import IPython\n", "\n", "# app = IPython.Application.instance()\n", "# app.kernel.do_shutdown(True)" - ], - "metadata": { - "id": "v6jBDnYnNM08" - }, - "id": "v6jBDnYnNM08", - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", + "id": "yygMe6rPWxHS", + "metadata": { + "id": "yygMe6rPWxHS" + }, "source": [ "### 🔐 Authentication\n", "Authenticate to Google Cloud as the IAM user logged into this notebook in order to access your Google Cloud Project.\n", "\n", "* If you are using Colab to run this notebook, use the cell below and continue.\n", "* If you are using Vertex AI Workbench, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)." - ], - "metadata": { - "id": "yygMe6rPWxHS" - }, - "id": "yygMe6rPWxHS" + ] }, { "cell_type": "code", - "source": [ - "from google.colab import auth\n", - "\n", - "auth.authenticate_user()" - ], + "execution_count": 1, + "id": "PTXN1_DSXj2b", "metadata": { "id": "PTXN1_DSXj2b" }, - "execution_count": 1, "outputs": [], - "id": "PTXN1_DSXj2b" + "source": [ + "from google.colab import auth\n", + "\n", + "auth.authenticate_user()" + ] }, { "cell_type": "markdown", + "id": "NEvB9BoLEulY", + "metadata": { + "id": "NEvB9BoLEulY" + }, "source": [ "### ☁ Set Your Google Cloud Project\n", "Set your Google Cloud project so that you can leverage Google Cloud resources within this notebook.\n", @@ -118,14 +128,17 @@ "* Run `gcloud config list`.\n", "* Run `gcloud projects list`.\n", "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)." - ], - "metadata": { - "id": "NEvB9BoLEulY" - }, - "id": "NEvB9BoLEulY" + ] }, { "cell_type": "code", + "execution_count": null, + "id": "gfkS3yVRE4_W", + "metadata": { + "cellView": "form", + "id": "gfkS3yVRE4_W" + }, + "outputs": [], "source": [ "# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.\n", "\n", @@ -133,38 +146,7 @@ "\n", "# Set the project id\n", "!gcloud config set project {PROJECT_ID}" - ], - "metadata": { - "cellView": "form", - "id": "gfkS3yVRE4_W" - }, - "id": "gfkS3yVRE4_W", - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "source": [ - "### 💡 API Enablement\n", - "The `langchain-google-cloud-sql-mssql` package requires that you [enable the Cloud SQL Admin API](https://console.cloud.google.com/flows/enableapi?apiid=sqladmin.googleapis.com) in your Google Cloud Project." - ], - "metadata": { - "id": "rEWWNoNnKOgq" - }, - "id": "rEWWNoNnKOgq" - }, - { - "cell_type": "code", - "source": [ - "# enable Cloud SQL Admin API\n", - "!gcloud services enable sqladmin.googleapis.com" - ], - "metadata": { - "id": "5utKIdq7KYi5" - }, - "id": "5utKIdq7KYi5", - "execution_count": 3, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -178,17 +160,23 @@ }, { "cell_type": "markdown", - "source": [ - "### Set Cloud SQL database values\n", - "Find your database values, in the [Cloud SQL Instances page](https://console.cloud.google.com/sql?_ga=2.223735448.2062268965.1707700487-2088871159.1707257687)." - ], + "id": "OMvzMWRrR6n7", "metadata": { "id": "OMvzMWRrR6n7" }, - "id": "OMvzMWRrR6n7" + "source": [ + "### Set Cloud SQL database values\n", + "Find your database values, in the [Cloud SQL Instances page](https://console.cloud.google.com/sql?_ga=2.223735448.2062268965.1707700487-2088871159.1707257687)." + ] }, { "cell_type": "code", + "execution_count": 4, + "id": "irl7eMFnSPZr", + "metadata": { + "id": "irl7eMFnSPZr" + }, + "outputs": [], "source": [ "# @title Set Your Values Here { display-mode: \"form\" }\n", "REGION = \"us-central1\" # @param {type: \"string\"}\n", @@ -197,16 +185,14 @@ "DB_USER = \"my-username\" # @param {type: \"string\"}\n", "DB_PASS = \"my-password\" # @param {type: \"string\"}\n", "TABLE_NAME = \"message_store\" # @param {type: \"string\"}" - ], - "metadata": { - "id": "irl7eMFnSPZr" - }, - "id": "irl7eMFnSPZr", - "execution_count": 4, - "outputs": [] + ] }, { "cell_type": "markdown", + "id": "QuQigs4UoFQ2", + "metadata": { + "id": "QuQigs4UoFQ2" + }, "source": [ "### MSSQLEngine Connection Pool\n", "\n", @@ -214,7 +200,7 @@ "\n", "To create a `MSSQLEngine` using `MSSQLEngine.from_instance()` you need to provide only 6 things:\n", "\n", - "1. `project_id` : Project ID of the Google Cloud Project where the Cloud SQL instance is located.\n", + "1. `project_id` : Project ID of the Google Cloud Project where the Cloud SQL instance is located.\n", "1. `region` : Region where the Cloud SQL instance is located.\n", "1. `instance` : The name of the Cloud SQL instance.\n", "1. `database` : The name of the database to connect to on the Cloud SQL instance.\n", @@ -222,11 +208,7 @@ "1. `password` : Database password to use for built-in database authentication and login.\n", "\n", "By default, [built-in database authentication](https://cloud.google.com/sql/docs/sqlserver/users) using a username and password to access the Cloud SQL database is used for database authentication.\n" - ], - "metadata": { - "id": "QuQigs4UoFQ2" - }, - "id": "QuQigs4UoFQ2" + ] }, { "cell_type": "code", @@ -237,10 +219,10 @@ "end_time": "2023-08-28T10:04:38.077748Z", "start_time": "2023-08-28T10:04:36.105894Z" }, + "id": "4576e914a866fb40", "jupyter": { "outputs_hidden": false - }, - "id": "4576e914a866fb40" + } }, "outputs": [], "source": [ @@ -258,31 +240,35 @@ }, { "cell_type": "markdown", + "id": "qPV8WfWr7O54", + "metadata": { + "id": "qPV8WfWr7O54" + }, "source": [ "### Initialize a table\n", "The `MSSQLChatMessageHistory` class requires a database table with a specific schema in order to store the chat message history.\n", "\n", "The `MSSQLEngine` engine has a helper method `init_chat_history_table()` that can be used to create a table with the proper schema for you." - ], - "metadata": { - "id": "qPV8WfWr7O54" - }, - "id": "qPV8WfWr7O54" + ] }, { "cell_type": "code", - "source": [ - "engine.init_chat_history_table(table_name=TABLE_NAME)" - ], + "execution_count": null, + "id": "TEu4VHArRttE", "metadata": { "id": "TEu4VHArRttE" }, - "id": "TEu4VHArRttE", - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "engine.init_chat_history_table(table_name=TABLE_NAME)" + ] }, { "cell_type": "markdown", + "id": "zSYQTYf3UfOi", + "metadata": { + "id": "zSYQTYf3UfOi" + }, "source": [ "### MSSQLChatMessageHistory\n", "\n", @@ -291,14 +277,16 @@ "1. `engine` - An instance of a `MSSQLEngine` engine.\n", "1. `session_id` - A unique identifier string that specifies an id for the session.\n", "1. `table_name` : The name of the table within the Cloud SQL database to store the chat message history." - ], - "metadata": { - "id": "zSYQTYf3UfOi" - }, - "id": "zSYQTYf3UfOi" + ] }, { "cell_type": "code", + "execution_count": 7, + "id": "Kq7RLtfOq0wi", + "metadata": { + "id": "Kq7RLtfOq0wi" + }, + "outputs": [], "source": [ "from langchain_google_cloud_sql_mssql import MSSQLChatMessageHistory\n", "\n", @@ -307,13 +295,7 @@ ")\n", "history.add_user_message(\"hi!\")\n", "history.add_ai_message(\"whats up?\")" - ], - "metadata": { - "id": "Kq7RLtfOq0wi" - }, - "id": "Kq7RLtfOq0wi", - "execution_count": 7, - "outputs": [] + ] }, { "cell_type": "code", @@ -324,25 +306,25 @@ "end_time": "2023-08-28T10:04:38.929396Z", "start_time": "2023-08-28T10:04:38.915727Z" }, - "jupyter": { - "outputs_hidden": false - }, "colab": { "base_uri": "https://localhost:8080/" }, "id": "b476688cbb32ba90", + "jupyter": { + "outputs_hidden": false + }, "outputId": "f8c170e8-ea9d-4905-a9f4-bc83f9726ac5" }, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "[HumanMessage(content='hi!'), AIMessage(content='whats up?')]" ] }, + "execution_count": 8, "metadata": {}, - "execution_count": 8 + "output_type": "execute_result" } ], "source": [ @@ -351,28 +333,28 @@ }, { "cell_type": "markdown", + "id": "ss6CbqcTTedr", + "metadata": { + "id": "ss6CbqcTTedr" + }, "source": [ "#### Cleaning up\n", "When the history of a specific session is obsolete and can be deleted, it can be done the following way.\n", "\n", "**Note:** Once deleted, the data is no longer stored in Cloud SQL and is gone forever." - ], - "metadata": { - "id": "ss6CbqcTTedr" - }, - "id": "ss6CbqcTTedr" + ] }, { "cell_type": "code", - "source": [ - "history.clear()" - ], + "execution_count": 9, + "id": "3khxzFxYO7x6", "metadata": { "id": "3khxzFxYO7x6" }, - "id": "3khxzFxYO7x6", - "execution_count": 9, - "outputs": [] + "outputs": [], + "source": [ + "history.clear()" + ] }, { "cell_type": "markdown", @@ -390,16 +372,16 @@ }, { "cell_type": "code", - "source": [ - "# enable Vertex AI API\n", - "!gcloud services enable aiplatform.googleapis.com" - ], + "execution_count": null, + "id": "hYtHM3-TOMCe", "metadata": { "id": "hYtHM3-TOMCe" }, - "id": "hYtHM3-TOMCe", - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "# enable Vertex AI API\n", + "!gcloud services enable aiplatform.googleapis.com" + ] }, { "cell_type": "code", @@ -482,14 +464,14 @@ }, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "AIMessage(content=' Hello Bob, how can I help you today?')" ] }, + "execution_count": 14, "metadata": {}, - "execution_count": 14 + "output_type": "execute_result" } ], "source": [ @@ -509,14 +491,14 @@ }, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "AIMessage(content=' Your name is Bob.')" ] }, + "execution_count": 15, "metadata": {}, - "execution_count": 15 + "output_type": "execute_result" } ], "source": [ @@ -525,6 +507,10 @@ } ], "metadata": { + "colab": { + "provenance": [], + "toc_visible": true + }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", @@ -541,10 +527,6 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.1" - }, - "colab": { - "provenance": [], - "toc_visible": true } }, "nbformat": 4, diff --git a/docs/document_loader.ipynb b/docs/document_loader.ipynb index 2d4bdce..df9a193 100644 --- a/docs/document_loader.ipynb +++ b/docs/document_loader.ipynb @@ -10,6 +10,8 @@ "\n", "This notebook goes over how to use [Cloud SQL for SQL server](https://cloud.google.com/sql/sqlserver) to [save, load and delete langchain documents](https://python.langchain.com/docs/modules/data_connection/document_loaders/) with `MSSQLLoader` and `MSSQLDocumentSaver`.\n", "\n", + "Learn more about the package on [GitHub](https://github.com/googleapis/langchain-google-cloud-sql-mssql-python/).\n", + "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/googleapis/langchain-google-cloud-sql-mssql-python/blob/main/docs/document_loader.ipynb)" ] }, @@ -20,10 +22,12 @@ "## Before You Begin\n", "\n", "To run this notebook, you will need to do the following:\n", + "\n", "* [Create a Google Cloud Project](https://developers.google.com/workspace/guides/create-project)\n", + "* [Enable the Cloud SQL Admin API.](https://console.cloud.google.com/marketplace/product/google/sqladmin.googleapis.com)\n", "* [Create a Cloud SQL for SQL server instance](https://cloud.google.com/sql/docs/sqlserver/create-instance)\n", - "* [Create a Cloud SQL database](https://cloud.google.com/sql/docs/mssql/create-manage-databases)\n", - "* [Add an IAM database user to the database](https://cloud.google.com/sql/docs/sqlserver/add-manage-iam-users#creating-a-database-user) (Optional)\n", + "* [Create a Cloud SQL database](https://cloud.google.com/sql/docs/sqlserver/create-manage-databases)\n", + "* [Add an IAM database user to the database](https://cloud.google.com/sql/docs/sqlserver/create-manage-users) (Optional)\n", "\n", "After confirmed access to database in the runtime environment of this notebook, filling the following values and run the cell before running example scripts." ] @@ -204,6 +208,7 @@ "### Initialize a table\n", "\n", "Initialize a table of default schema via `MSSQLEngine.init_document_table()`. Table Columns:\n", + "\n", "- page_content (type: text)\n", "- langchain_metadata (type: JSON)\n", "\n", @@ -226,6 +231,7 @@ "### Save documents\n", "\n", "Save langchain documents with `MSSQLDocumentSaver.add_documents()`. To initialize `MSSQLDocumentSaver` class you need to provide 2 things:\n", + "\n", "1. `engine` - An instance of a `MSSQLEngine` engine.\n", "2. `table_name` - The name of the table within the Cloud SQL database to store langchain documents." ] @@ -236,8 +242,8 @@ "metadata": {}, "outputs": [], "source": [ - "from langchain_google_cloud_sql_mssql import MSSQLDocumentSaver\n", "from langchain_core.documents import Document\n", + "from langchain_google_cloud_sql_mssql import MSSQLDocumentSaver\n", "\n", "test_docs = [\n", " Document(\n", @@ -269,6 +275,7 @@ "metadata": {}, "source": [ "Load langchain documents with `MSSQLLoader.load()` or `MSSQLLoader.lazy_load()`. `lazy_load` returns a generator that only queries database during the iteration. To initialize `MSSQLDocumentSaver` class you need to provide:\n", + "\n", "1. `engine` - An instance of a `MSSQLEngine` engine.\n", "2. `table_name` - The name of the table within the Cloud SQL database to store langchain documents." ] @@ -340,6 +347,7 @@ "For table with default schema (page_content, langchain_metadata), the deletion criteria is:\n", "\n", "A `row` should be deleted if there exists a `document` in the list, such that\n", + "\n", "- `document.page_content` equals `row[page_content]`\n", "- `document.metadata` equals `row[langchain_metadata]`" ] @@ -447,6 +455,7 @@ "metadata": {}, "source": [ "We can specify the content and metadata we want to load by setting the `content_columns` and `metadata_columns` when initializing the `MSSQLLoader`.\n", + "\n", "1. `content_columns`: The columns to write into the `page_content` of the document.\n", "2. `metadata_columns`: The columns to write into the `metadata` of the document.\n", "\n", @@ -485,12 +494,14 @@ "metadata": {}, "source": [ "In order to save langchain document into table with customized metadata fields. We need first create such a table via `MSSQLEngine.init_document_table()`, and specify the list of `metadata_columns` we want it to have. In this example, the created table will have table columns:\n", + "\n", "- description (type: text): for storing fruit description.\n", "- fruit_name (type text): for storing fruit name.\n", "- organic (type tinyint(1)): to tell if the fruit is organic.\n", "- other_metadata (type: JSON): for storing other metadata information of the fruit.\n", "\n", "We can use the following parameters with `MSSQLEngine.init_document_table()` to create the table:\n", + "\n", "1. `table_name`: The name of the table within the Cloud SQL database to store langchain documents.\n", "2. `metadata_columns`: A list of `sqlalchemy.Column` indicating the list of metadata columns we need.\n", "3. `content_column`: The name of column to store `page_content` of langchain document. Default: `page_content`.\n", @@ -530,6 +541,7 @@ "metadata": {}, "source": [ "Save documents with `MSSQLDocumentSaver.add_documents()`. As you can see in this example, \n", + "\n", "- `document.page_content` will be saved into `description` column.\n", "- `document.metadata.fruit_name` will be saved into `fruit_name` column.\n", "- `document.metadata.organic` will be saved into `organic` column.\n", @@ -583,6 +595,7 @@ "We can also delete documents from table with customized metadata columns via `MSSQLDocumentSaver.delete()`. The deletion criteria is:\n", "\n", "A `row` should be deleted if there exists a `document` in the list, such that\n", + "\n", "- `document.page_content` equals `row[page_content]`\n", "- For every metadata field `k` in `document.metadata`\n", " - `document.metadata[k]` equals `row[k]` or `document.metadata[k]` equals `row[langchain_metadata][k]`\n",