Skip to content

Silas final pipelines #934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Aug 21, 2023
Merged

Silas final pipelines #934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6c14956
Remote save
SilasMarvin Jul 14, 2023
3b2e3b5
Working remote embeddings
SilasMarvin Jul 17, 2023
f1d6bf7
Compiling
SilasMarvin Jul 19, 2023
22f280e
Commit before moving everything to lazy
SilasMarvin Jul 19, 2023
ec090ca
Working lazy python
SilasMarvin Jul 21, 2023
58c01a3
Commit before moving adjusting Javascript macros
SilasMarvin Jul 21, 2023
9e7b146
Working javascript sdk
SilasMarvin Jul 22, 2023
abb4f5e
Working javascript sdk
SilasMarvin Jul 26, 2023
76ccf3a
The start of working pipelines
SilasMarvin Jul 28, 2023
cfcc66b
Working pipelines in python
SilasMarvin Aug 3, 2023
a9dcbc9
Uncomment
SilasMarvin Aug 3, 2023
8b48750
Added to_dict function
SilasMarvin Aug 3, 2023
6e3f1e6
Small changes and prep for progress bars
SilasMarvin Aug 4, 2023
5365557
Working progress bars and many other small but exciting things
SilasMarvin Aug 4, 2023
66476ff
Prepping to push to test pypi
SilasMarvin Aug 7, 2023
f2613d7
Prepping for javascript
SilasMarvin Aug 8, 2023
92c9623
Improvments to javascript and updates to the python sdk deploy script
SilasMarvin Aug 8, 2023
8a4e3cf
Prepping for real tests
SilasMarvin Aug 8, 2023
12bb3a8
Updated sql
SilasMarvin Aug 9, 2023
2b5b68b
Python examples translated to use pipelines
SilasMarvin Aug 9, 2023
447fc80
Mostly cleaned up and documented crate, and cleaned up python README …
SilasMarvin Aug 10, 2023
333c5e6
Ready for test deployments
SilasMarvin Aug 10, 2023
11bcce2
Updated manual build file for python
SilasMarvin Aug 10, 2023
845bf02
Build fast
SilasMarvin Aug 11, 2023
4904a1a
Small tweaks
SilasMarvin Aug 11, 2023
64dc7e2
Prepping for another test release
SilasMarvin Aug 11, 2023
c3b274c
Prepping to expand query_builder
SilasMarvin Aug 11, 2023
cb143a5
Massive cleanups to macros
SilasMarvin Aug 11, 2023
c66b07b
Massive cleanups to macros
SilasMarvin Aug 11, 2023
b7d4c2d
Ready to release
SilasMarvin Aug 11, 2023
a2c87b1
Formatting
SilasMarvin Aug 13, 2023
dd9c3ab
Renamed files
SilasMarvin Aug 21, 2023
5568608
Added removed file
SilasMarvin Aug 21, 2023
4a2e98d
Removed unnecessary file
SilasMarvin Aug 21, 2023
e673af4
Updated sdk version to 0.9
SilasMarvin Aug 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 48 additions & 38 deletions .github/workflows/python-sdk.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
name: deploy python sdk
on:
workflow_dispatch:
inputs:
deploy_to_pypi:
description: "Whether to deploy to PyPI (true) or TestPyPI (false)"
required: false
default: "false"
jobs:
deploy-python-sdk-linux:
strategy:
Expand All @@ -24,31 +29,33 @@ jobs:
DEBIAN_FRONTEND: noninteractive
TZ: Etc/UTC
run: |
sudo apt update
sudo apt-get -y install software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo add-apt-repository -y ppa:apt-fast/stable
sudo apt update
sudo apt-get -y install apt-fast
sudo apt-fast -y install \
python3.7 python3.7-dev \
python3.8 python3.8-dev \
python3.9 python3.9-dev \
python3.10 python3.10-dev \
python3.11 python3.11-dev \
python3-pip \
git
pip install maturin
pip install patchelf
- name: Create dist directory
run: mkdir ../../python/dist
- name: Build wheels
run: maturin build --release --strip -i python3.7 -i python3.8 -i python3.9 -i python3.10 -i python3.11 -o ../../python/pgml/dist -F python
- name: Deploy wheels
sudo apt update
sudo apt-get -y install software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo add-apt-repository -y ppa:apt-fast/stable
sudo apt update
sudo apt-get -y install apt-fast
sudo apt-fast -y install \
python3.7 python3.7-dev \
python3.8 python3.8-dev \
python3.9 python3.9-dev \
python3.10 python3.10-dev \
python3.11 python3.11-dev \
python3-pip \
git
pip install maturin
pip install patchelf
- name: Build and deploy wheels to TestPyPI
if: github.event.inputs.deploy_to_pypi == 'false'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: maturin publish -r testpypi -i python3.7 -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python
- name: Build and deploy wheels to PyPI
if: github.event.inputs.deploy_to_pypi == 'true'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
working-directory: pgml-sdks/python/pgml
run: maturin upload --skip-existing dist/*
run: maturin publish -i python3.7 -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python

deploy-python-sdk-mac:
runs-on: macos-latest
defaults:
Expand All @@ -72,15 +79,17 @@ jobs:
brew install python@3.10
brew install python@3.11
pip3 install maturin
- name: Create dist directory
run: mkdir ../../python/dist
- name: Build wheels
run: maturin build --release --strip -i python3.8 -i python3.9 -i python3.10 -i python3.11 -o ../../python/pgml/dist -F python
- name: Deploy wheels
- name: Build and deploy wheels to TestPyPI
if: github.event.inputs.deploy_to_pypi == 'false'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: maturin publish -r testpypi -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python
- name: Build and deploy wheels to PyPI
if: github.event.inputs.deploy_to_pypi == 'true'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
working-directory: pgml-sdks/python/pgml
run: maturin upload --skip-existing dist/*
run: maturin publish -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python

deploy-python-sdk-windows:
runs-on: windows-latest
strategy:
Expand All @@ -101,17 +110,18 @@ jobs:
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip3 install maturin
- name: Create dist directory
run: mkdir ..\..\python\dist
- name: Build wheels
run: maturin build --release --strip -o ..\..\python\pgml\dist -F python
- name: Deploy wheels
- name: Build and deploy wheels to TestPyPI
if: github.event.inputs.deploy_to_pypi == 'false'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: maturin publish -r testpypi -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python
- name: Build and deploy wheels to PyPI
if: github.event.inputs.deploy_to_pypi == 'true'
env:
MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
working-directory: pgml-sdks\python\pgml
run: maturin upload --skip-existing dist\*
run: maturin publish -i python3.8 -i python3.9 -i python3.10 -i python3.11 --skip-existing -F python
8 changes: 8 additions & 0 deletions pgml-extension/sql/pgml--2.7.6--2.7.7.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
-- Drop the constraint on snapshot_id for models
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gonna need to bump the version in Cargo.toml and hit build to update the lockfile.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to release the version now I mean.

ALTER TABLE pgml.models ALTER COLUMN snapshot_id DROP NOT NULL;

-- Add openai option to pgml.runtime
ALTER TYPE pgml.runtime ADD VALUE IF NOT EXISTS 'openai';

-- Add embedding option to pgml.task
ALTER TYPE pgml.task ADD VALUE IF NOT EXISTS 'embedding';
Loading