Skip to content

Conversation

outergod
Copy link
Contributor

@outergod outergod commented Jul 30, 2025

Description

Change the way the FastAPI instrumentor deals with the FastAPI middleware stack so that exception handling code doesn't get executed twice, but still has a valid OTEL context available. At the same time, make sure instrumentor hooks failures cannot crash the service itself.

Fixes #3642
Fixes #3637

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Using the MRE in the linked issue, and added unit tests.

Does This PR Require a Core Repo Change?

  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated (not needed)

@outergod outergod changed the title Fix/gh 3642 fastapi exceptions Rewrite FastAPI instrumentor middleware stack to be failsafe Jul 30, 2025
@outergod outergod requested a review from a team as a code owner July 30, 2025 11:00
Copy link
Contributor

@alexmojaki alexmojaki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid, thank you so much!

@outergod
Copy link
Contributor Author

Thank you for the thorough review and the persistence @alexmojaki!
Do you have any suggestions how to find a maintainer to sponsor the PR? Should I ask on the CNCF Slack?

@alexmojaki
Copy link
Contributor

@xrmx @emdneto @codefromthecrypt @lzchen please review? This relates to #3012 which you reviewed previously. Also cc @Kludex @adriangb

Copy link
Member

@emdneto emdneto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I gave it a try to run the repro and now I can see the recorded exception. Overall, it sounds good.

If it helps to others to review, before we had:
ServerErrorMiddleware (outermost) -> OpenTelemetryMiddleware -> ServerErrorMiddleware (innermost)

now we have:
ServerErrorMiddleware (outer -- same as before) -> OpenTelemetryMiddleware -> ServerErrorMiddleware (with original handler/debug) -> ExceptionHandlerMiddleware (with access to the span context)

But I'm afraid we are not seeing some issues in the current structure of tests for FastAPI. Noticed that while reviewing #3701

span.set_status(
Status(
status_code=StatusCode.ERROR,
description=f"{type(exc).__name__}: {exc}",
Copy link
Member

@emdneto emdneto Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to your PR I guess, but I can't see the description in the exported span. #3713

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you suggest I do?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Ready for review
Development

Successfully merging this pull request may close these issues.

FastAPI instrumentation: errors in hooks not handled properly FastAPI instrumentor stops recording exception event starting v0.55b0
3 participants