-
Notifications
You must be signed in to change notification settings - Fork 706
feat: Implement Synchronous Metrics Reader #4542
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Implement Synchronous Metrics Reader #4542
Conversation
Thanks for submitting! What other benefits come with this new Reader, instead of using PeriodicExportingMetricReader with |
Hi @tammy-baylis-swi! Thanks for looking into this. You raise a fair point on using
I would be happy to look into merging the queue system into periodic reader if that makes more sense. If not I would be happy to contribute an example of this to the repo to make the feature more explicit. |
Yes correct.
That's great actually!!
Imo it'd be nice to keep this separate as an additional concrete Reader class like you have it, like with exporters/processors for logs and traces. Please could you have a check of the failing tests. There's a few failures like
|
Hi @tammy-baylis-swi, yes I shall look into the test. I think I had some weird logic which in the test which was passing locally but not here. Will get that fixed Monday. Documentation wise is there anything I need to do? I noticed that the documentation was being built but I couldn't find a specific page for the SDK. I assume part of it is being built from the inline class description? |
Hi @tammy-baylis-swi, |
Hi @Jayclifford345 , yes the docstrings are used to generate some of the docs with sphinx -- style guide links in the contributing.md. For the new example (much appreciated), the new class is part of the SDK so I don't think I think you need to do a local Thanks for the test updates! Those |
Hi @tammy-baylis-swi, can confirm the new reader is showing up in the docs as expected after building. The failed test seems to be a mismatch in the workflow, not being able to find a specific commit hash. Not quite sure how to resolve that one. Shall we try to run the test again and see if it clears up on its own? |
@Jayclifford345 Please merge Also: Please could you create an accompanying Issue (Feature Request) to link to this PR! https://github.com/open-telemetry/opentelemetry-python/issues |
…rd345/opentelemetry-python into metrics-batch-exporter
Hi @tammy-baylis-swi, |
Hmm @tammy-baylis-swi, I checked the error
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Jayclifford345 for creating that issue! I've also updated OP with a quick link to it.
Lgtm. I can't merge so I'm passing this on to @open-telemetry/python-maintainers to continue review. Feel free to also join the weekly Python SIG zoom meeting to request more eyes.
Yes I'm not sure what that markdown-link-check error is about 🤔 What you introduced to CHANGELOG
looks ok to me.
Thanks a lot @tammy-baylis-swi! I added a note to the agenda look forward to meeting you there :) |
I checked the otel specfication and sadly it looks like this would be out of current scope since we already implemented all of the required metrics readers: https://opentelemetry.io/docs/specs/otel/metrics/sdk/#metricreader Happy to pull the plug if this is more trouble than it`s worth. I can just open a new PR to give an example of the alternative method if that helps instead? Let me know what you think though after review |
Co-authored-by: Emídio Neto <9735060+emdneto@users.noreply.github.com>
…rd345/opentelemetry-python into metrics-batch-exporter
Apologies for the late review. It seems to me the only difference between this functionality and the current workaround in the periodic metric reader (with infinity) is the batching capability. While I do agree that our current components does not satisfy your specific situation, I have reservations in adding new components to the already stable api/sdk package, especially ones that are not in the spec. We will also have to support this metric reader forever (even past your use case) in case other people take a dependency on it. I'd be much more likely to approve this if we perhaps make this an experimental component (with an underscore) with guidance and documentation indicating to users to try at their own risk. wdyt? |
Hi @lzchen, I absolutely agree with your comments above. Sorry, I couldn't attend the Python sig meeting yesterday. I am happy to turn it into an experimental feature and see if it lands for the otel community. I want to make sure that I am not putting you all in a bad spot though where we are moving away from the specification outlined. I am happy to support this feature in the meantime. I ended up creating an example with the workaround here as an official method: #4559. I will work on the changes to get it moved into an experimental mode. Let me know though if there are any processes I need to follow in the meantime with regards to the specification :). Thanks again for the review. |
I think, based on the discussion in the issue, this is out of scope for now and with the given workaround now documented in #4559 I think this works as a good alternative. If this arises again in the future I am happy for this PR to be reopened and the code used in another implementation. |
Description
Addresses #4549
This PR introduces a new
SynchronousExportingMetricReader
that implements batched metric export without background threads. Unlike the existingPeriodicExportingMetricReader
which uses a scheduler, this reader collects and exports metrics on-demand when explicitly requested throughcollect()
calls, making it ideal for short-lived environments like serverless functions where predictable, explicit metric collection is preferred.The reader maintains metrics in a queue and exports them when:
force_flush()
is calledshutdown()
is calledI decided to implement this feature with serverless environments like AWS Lambda in mind, where you need to explicitly collect and export metrics before the function terminates, without relying on background threads that may not have time to execute.
Type of change
How Has This Been Tested?
Does This PR Require a Contrib Repo Change?
Checklist: