24 Pull requests merged by 10 people

• Update istio version

  #780 merged May 2, 2025

• add labels to pod metadata for the use of scheduler plugins

  #779 merged May 2, 2025

• passing headers to scheduler plugins

  #775 merged May 2, 2025

• remove EndpointSlice from RBAC

  #774 merged May 2, 2025

• Add feature request link for adding Triton LoRA metric

  #773 merged May 1, 2025

• Create unit test for request handler

  #745 merged May 1, 2025

• put SchedulerConfig fields private again. added NewSchedulerConfig func

  #771 merged May 1, 2025

• Parse request x-request-id and expose it in contextual logger

  #746 merged May 1, 2025

• Add scheduler e2e latency metric

  #767 merged Apr 30, 2025

• Add queue and kv-cache scorers

  #762 merged Apr 30, 2025

• Small refactor to capture request data for route.

  #765 merged Apr 30, 2025

• fix: pass commit hash from the cloud build default variable

  #763 merged Apr 30, 2025

• chore: make SchedulerConfig fields configurable

  #764 merged Apr 30, 2025

• Add inference_extension_info metric for project metadata

  #744 merged Apr 29, 2025

• Move scheduler initialization up to the main

  #757 merged Apr 29, 2025

• feat: Initial setup for conformance test suite

  #720 merged Apr 29, 2025

• fixed error message in scheduler when no pods are available

  #759 merged Apr 29, 2025

• Request for adding Alibaba Cloud Container Service for Kubernetes (ACK) into implementations

  #748 merged Apr 29, 2025

• extract pod representation from backend/metrics to backend

  #751 merged Apr 29, 2025

• Bump the kubernetes group with 6 updates

  #754 merged Apr 29, 2025

• Add GetEnvString helper function

  #758 merged Apr 28, 2025

• add max score picker

  #752 merged Apr 28, 2025

• Weighted scorers

  #737 merged Apr 28, 2025

• fixed broken link to implementations

  #750 merged Apr 27, 2025

9 Pull requests opened by 9 people

• add regression testing docs

  #755 opened Apr 28, 2025

• [WIP]: added v1 type for experiment

  #756 opened Apr 28, 2025

• Added the ability for plugins to receive the request headers and modify them

  #760 opened Apr 30, 2025

• Amend the endpoint picker protocol to support multiple fallback endpoints

  #761 opened Apr 30, 2025

• Add prefix cache aware scheduling

  #768 opened May 1, 2025

• feat(conformance): Add initial InferencePool tests and shared Gateway setup

  #772 opened May 1, 2025

• feat: Add metric that records length of queue for each model server pods

  #776 opened May 2, 2025

• chore: update golang.google.org/grpc dep from v1.71.1 to v1.72.0

  #777 opened May 2, 2025

• [WIP] EPP architectural refactor

  #781 opened May 2, 2025

5 Issues closed by 2 people

• Docs: Getting Started Guide for istio resulting in a broken example

  #740 closed May 2, 2025

• Configure x-request-id support in the default ootb examples

  #556 closed May 1, 2025

• Include metadata metric

  #579 closed Apr 29, 2025

• Add unit test coverage to datastore pkg

  #391 closed Apr 28, 2025

• Implementations reference link in site introduction page is broken

  #749 closed Apr 27, 2025

6 Issues opened by 5 people

• Dashboard: Add total queue size metrics

  #778 opened May 2, 2025

• Support Semantic Processing using NLP models

  #770 opened May 1, 2025

• Docs: YAML Example with multiple inference pools

  #769 opened May 1, 2025

• API Discrepancy: `InferencePoolSpec` pod selector field name mismatch between API Proposal 002 and current Go type definition

  #766 opened Apr 30, 2025

• Enable Conformance Testing for Standalone (Non-Gateway API) EPP Implementations

  #753 opened Apr 28, 2025

• metrics dashboard should be documented for options other than Google Managed Prometheus

  #747 opened Apr 26, 2025

18 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Scheduler subsystem high level design proposal

  #603 commented on Apr 29, 2025 • 5 new comments

• Add prefix aware routing proposal

  #602 commented on May 2, 2025 • 2 new comments

• feat: merge two metric servers

  #728 commented on May 1, 2025 • 1 new comment

• Bump google.golang.org/grpc from 1.71.1 to 1.72.0

  #722 commented on May 2, 2025 • 0 new comments

• Proposal: EPP should support heterogeneous pods across the pool

  #715 commented on May 3, 2025 • 0 new comments

• v0.4 Release Tracker

  #681 commented on May 2, 2025 • 0 new comments

• Tools: Add Scheduler Plugin Metrics to Dashboards

  #705 commented on May 2, 2025 • 0 new comments

• Provide alerting best practices

  #694 commented on May 2, 2025 • 0 new comments

• e2e CI Job

  #259 commented on May 2, 2025 • 0 new comments

• EPP upgrade/downgrade guide

  #693 commented on May 1, 2025 • 0 new comments

• Docs: Create EPP Operations Guide

  #735 commented on May 1, 2025 • 0 new comments

• EPP HA deployment

  #692 commented on May 1, 2025 • 0 new comments

• Create InferenceModel Controller

  #409 commented on Apr 29, 2025 • 0 new comments

• replace InferenceModel uniquness check in code with admission validation webhook

  #716 commented on Apr 29, 2025 • 0 new comments

• Benchmark Test Harness

  #732 commented on Apr 28, 2025 • 0 new comments

• Add unit test coverage to the handlers pkg

  #392 commented on Apr 28, 2025 • 0 new comments

• support grayscale processes based on basic models

  #587 commented on Apr 27, 2025 • 0 new comments

• Expose baseline algorithm parameters as configurable

  #16 commented on Apr 27, 2025 • 0 new comments