Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Query Frontend: Expose samples_processed in Server-Timing header #10103

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Query Frontend: Expose samples_processed in Server-Timing header #10103

wants to merge 7 commits into from
+278 −30

Conversation

tinitiuset
Copy link
Contributor

@tinitiuset tinitiuset commented Dec 3, 2024 •
edited
Loading

What this PR does

This PR will make the Query frontend expose samples_processed in Server-Timing header. It is complementary to #9985 and this in the effort of measuring throughput on Mimir.

This was worked on by @krajorama before #7966, but I decided to open a different PR as the idea has evolved overtime.

Which issue(s) this PR fixes or relates to

Not related to an specific issue.

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

pracucci
pracucci reviewed Dec 3, 2024
View reviewed changes
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a comment about the naming. "total samples" may mean "total raw samples processed" or "total number of samples returned by the query". The cost of a query is a function of the actual raw samples processed, not returned. If that's what you're going to track (as I would expect) then we may consider renaming it to "samples processed" (to keep it consistent with "bytes processed" that we already have).

@tinitiuset tinitiuset force-pushed the tinitiuset/feat/exposeTotalSamples branch from c373bdd to cc1117d Compare December 3, 2024 19:57
@tinitiuset tinitiuset changed the title Query Frontend: Expose total_samples in Server-Timing header Query Frontend: Expose samples_processed in Server-Timing header Dec 4, 2024
@tinitiuset tinitiuset mentioned this pull request Dec 5, 2024
Open
@tinitiuset tinitiuset marked this pull request as ready for review December 5, 2024 10:10
@tinitiuset tinitiuset requested a review from a team as a code owner December 5, 2024 10:10
@tinitiuset
Copy link
Contributor Author

@pracucci 's feedback is acknowledged and MQE support will be added in another PR.

charleskorn
charleskorn reviewed Dec 10, 2024
View reviewed changes
Copy link
Contributor

@charleskorn charleskorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'd like to see the Stats tests expanded to cover the new SamplesProcessed field.

pkg/querier/stats/stats.go Outdated Show resolved Hide resolved
pkg/querier/stats_renderer_test.go Outdated Show resolved Hide resolved
dimitarvdimitrov
dimitarvdimitrov approved these changes Dec 10, 2024
View reviewed changes
Copy link
Contributor

@dimitarvdimitrov dimitarvdimitrov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, mod Charles and Marco's comments

@chencs
Copy link
Contributor

chencs commented Dec 10, 2024

The CHANGELOG has just been cut to prepare for the next release. Please rebase main and eventually move the CHANGELOG entry added / updated in this PR to the top of the CHANGELOG.md document. Thanks!

@tinitiuset
Copy link
Contributor Author

Ready for review again. Testing updated. CHANGELOG updated also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@charleskorn charleskorn charleskorn left review comments

@dimitarvdimitrov dimitarvdimitrov dimitarvdimitrov approved these changes

@pracucci pracucci Awaiting requested review from pracucci

Assignees
No one assigned
Labels
release/notified-changelog-cut
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@tinitiuset @chencs @pracucci @charleskorn @dimitarvdimitrov