Misc: Add stage overlap script #19156
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a new Python script to compute stage overlap from DIL files, providing quick insights into the distribution of predicates across different stages.
- Added a new script at misc/scripts/stageoverlap.py that processes DIL files
- Implements basic stage caching and predicate recomputation logic
Tip: Leave feedback on Copilot's review comments with the 👎 and 👍 buttons to help improve review quality. Learn more
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
hvitved
approved these changes
Apr 3, 2025
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm adding this script, since it's been quite useful for me. The code is somewhat ad-hoc, but it does the job well enough. Possible future extensions: it would be nice to filter trivial predicates for which we don't care whether they're repeated across stages. That could be done by actually parsing some DIL or staying with the ad-hoc nature, we could also add an optional argument with an allow-list of predicate prefixes. Regardless, as-is it's quite useful to get an overview of the different stages in DIL and their overlap. The script also works on concatenated DIL from several queries (which is what you get when you dump DIL from compiling an entire suite).