Skip to content

Session affinity scorer #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 5, 2025

Conversation

dmitripikus
Copy link

  1. Session affinity scorer implementation
  2. Fix in request headers transfer

* dev:
  Added a simple unit test for the PostResponse plugin invocation
  Invoke the PostResponse handlers and send any added headers to the user
  Added code to scheduler to enable running the PostResponse plugins
  Use an init() function instead of modifying the scheduler code to inject our config
  Added PostResponse to scheduler config
  Fixed scorer tests
@dmitripikus dmitripikus requested review from mayabar, shmuelk and irar2 May 4, 2025 18:24
}

func (s *SessionAffinityScorer) Name() string {
return "session affinity scorer"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/session affinity scorer/session-affinity-scorer/ for alignment with other scorers.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

// session was sent to, by giving that pod the specified weight and assigning
// zero score to the rest of the targets
type SessionAffinityScorer struct {
sessions *sync.Map
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field is never used.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

@@ -74,3 +74,8 @@ type PostResponse interface {
Plugin
PostResponse(ctx *types.SchedulingContext, pod types.Pod)
}

type ScorerWithPostResponse struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this struct is redundant

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

sessions: &sync.Map{},
}
return plugins.ScorerWithPostResponse{
Scorer: s,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you remove the struct from prev comment, then here just return s

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

reqHeaders := ctx.Req.Headers

var sessionId = ""
for k, v := range reqHeaders {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this foor loop is very inefficient. headers field is a map (hash), means we can check if key exists in O(1) without iteration over all keys

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks


func (s *SessionAffinityScorer) PostResponse(ctx *types.SchedulingContext, pod types.Pod) {
ctx.MutatedHeaders[sessionIDHeader] = encode(pod.GetPod().NamespacedName.String())
}
Copy link
Collaborator

@nirrozenbaum nirrozenbaum May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a temp hack?
it puts as session id the namespaced name of the pod (encoded).
meaning different requests that are served by the same pod get the same value, which is not unique (not ID)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it's not a session Id, but a session token. I'll rename the header to Session-Token

Copy link
Collaborator

@nirrozenbaum nirrozenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Thanks Dima!

@dmitripikus dmitripikus merged commit f52caa2 into neuralmagic:dev May 5, 2025
1 check passed
@elevran elevran mentioned this pull request May 6, 2025
clubanderson pushed a commit that referenced this pull request May 7, 2025
* 'session affinity scorer' partial implementation (without headers in response)

* Fix in filling request headers

* Encoded value of namespaced pod name is sent in response to client

* Support of session affinity scorer configuration via environment variables, is added

* Go file for session affinity scorer is renamed

* Redundant 'sessions' field is removed

* Redundant 'ScorerWithPostResponse' struct is removed

* - SessionID is renamed to sessionToken
- Map fetch is done instead of loop

* Session token name is changed to 'x-session-token'

* Minor fixes are made in README

* Small fix after merge

---------

Co-authored-by: Shmuel Kallner <kallner@il.ibm.com>
clubanderson pushed a commit that referenced this pull request May 7, 2025
* 'session affinity scorer' partial implementation (without headers in response)

* Fix in filling request headers

* Encoded value of namespaced pod name is sent in response to client

* Support of session affinity scorer configuration via environment variables, is added

* Go file for session affinity scorer is renamed

* Redundant 'sessions' field is removed

* Redundant 'ScorerWithPostResponse' struct is removed

* - SessionID is renamed to sessionToken
- Map fetch is done instead of loop

* Session token name is changed to 'x-session-token'

* Minor fixes are made in README

* Small fix after merge

---------

Co-authored-by: Shmuel Kallner <kallner@il.ibm.com>
clubanderson pushed a commit that referenced this pull request May 7, 2025
* 'session affinity scorer' partial implementation (without headers in response)

* Fix in filling request headers

* Encoded value of namespaced pod name is sent in response to client

* Support of session affinity scorer configuration via environment variables, is added

* Go file for session affinity scorer is renamed

* Redundant 'sessions' field is removed

* Redundant 'ScorerWithPostResponse' struct is removed

* - SessionID is renamed to sessionToken
- Map fetch is done instead of loop

* Session token name is changed to 'x-session-token'

* Minor fixes are made in README

* Small fix after merge

---------

Co-authored-by: Shmuel Kallner <kallner@il.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants