Skip to content

Docs: Quickstart Fixes #615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 31, 2025
Merged

Docs: Quickstart Fixes #615

merged 1 commit into from
Mar 31, 2025

Conversation

danehans
Copy link
Contributor

@danehans danehans commented Mar 31, 2025 •
edited
Loading

  1. The InferencePool name referenced by the quick start guide (https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/inferencepool-resources.yaml) is no longer vllm-llama2-7b:
apiVersion: inference.networking.x-k8s.io/v1alpha2
kind: InferencePool
metadata:
  labels:
  name: vllm-llama3-8b-instruct
spec:
  targetPortNumber: 8000
  selector:
    app: vllm-llama3-8b-instruct
  extensionRef:
    name: vllm-llama3-8b-instruct-epp
  1. The food-review-1 model does not exist. Updates the food-review example InferenceModel to use the correct target model name:
$ kubextl exec deploy/vllm-llama3-8b-instruct -- curl 127.0.0.1:8000/v1/models | jq .
...
    {
      "id": "food-review",
      "object": "model",
...

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 31, 2025
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 31, 2025
@netlify Netlify
Copy link

netlify bot commented Mar 31, 2025 •
edited
Loading

✅ Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 3f99e71
🔍 Latest deploy log https://app.netlify.com/sites/gateway-api-inference-extension/deploys/67ea021156c4d000081157b4
😎 Deploy Preview https://deploy-preview-615--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Mar 31, 2025
@danehans
Docs: Quickstart Fixes
3f99e71 
- Fixes the InferencePool name reference in HTTPRoute.
- Fixes target model name in InferenceModel.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans danehans changed the title Docs: Fix InferencePool Name Reference Docs: Quickstart Fixes Mar 31, 2025
@danehans danehans force-pushed the quickstart_cleanup branch from 6d3c4e7 to 3f99e71 Compare March 31, 2025 02:46
@ahg-g
Copy link
Contributor

ahg-g commented Mar 31, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 31, 2025
@kfswain
Copy link
Collaborator

kfswain commented Mar 31, 2025

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danehans, kfswain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f4c956c into kubernetes-sigs:main Mar 31, 2025
8 checks passed
kfswain pushed a commit to kfswain/llm-instance-gateway that referenced this pull request Apr 29, 2025
@danehans @kfswain
Docs: Quickstart Fixes (kubernetes-sigs#615)
bc0306a 
- Fixes the InferencePool name reference in HTTPRoute.
- Fixes target model name in InferenceModel.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
kfswain pushed a commit to kfswain/llm-instance-gateway that referenced this pull request Apr 29, 2025
@danehans @kfswain
Docs: Quickstart Fixes (kubernetes-sigs#615)
e6b6d7d 
- Fixes the InferencePool name reference in HTTPRoute.
- Fixes target model name in InferenceModel.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
kfswain pushed a commit to kfswain/llm-instance-gateway that referenced this pull request Apr 29, 2025
@danehans @kfswain
Docs: Quickstart Fixes (kubernetes-sigs#615)
574926d 
- Fixes the InferencePool name reference in HTTPRoute.
- Fixes target model name in InferenceModel.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kfswain kfswain Awaiting requested review from kfswain

@liu-cong liu-cong Awaiting requested review from liu-cong

Assignees

@ahg-g ahg-g

@kfswain kfswain

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@danehans @ahg-g @kfswain @k8s-ci-robot