Skip to content

Shared: Fix join in FileSystem.qll #19345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 23, 2025

Conversation

hvitved
Copy link
Contributor

@hvitved hvitved commented Apr 22, 2025

Before

Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@d2a7eeoa on iteration 2 running pipeline standard with tuple counts:
         120   ~2%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

           0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
           0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

          91   ~3%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
          81   ~3%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

         269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                      {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
          39   ~1%    {5}    | SCAN OUTPUT In.5, In.0, In.1, In.2, In.3
        1295   ~0%    {6}    | JOIN WITH containerparent ON FIRST 1 OUTPUT Rhs.1, Lhs.3, _, Lhs.1, Lhs.2, Lhs.4
        1295   ~0%    {6}    | REWRITE WITH Out.2 := 1
          34   ~1%    {4}    | JOIN WITH `cached_FileSystem::Container.splitAbsolutePath/2#dispred#dc97b0cc` ON FIRST 3 OUTPUT Lhs.5, Lhs.3, Lhs.4, Lhs.0

         115   ~0%    {4} r5 = r2 UNION r3 UNION r4
         115   ~0%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                      return r5

After

Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@4fb6e6v7 on iteration 2 running pipeline standard with tuple counts:
        120   ~0%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

          0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

         91   ~0%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
         81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
         81   ~5%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

        269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                     {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
         39   ~3%    {5}    | SCAN OUTPUT In.5, In.2, In.0, In.1, In.3
         34   ~0%    {4}    | JOIN WITH `FileSystem::Folder::Append<PathResolution::shouldAppend>::getAChildContainer/2#2e91feca` ON FIRST 2 OUTPUT Lhs.4, Lhs.2, Lhs.3, Rhs.2

        115   ~2%    {4} r5 = r2 UNION r3 UNION r4
        115   ~2%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                     return r5

Before
```
Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@d2a7eeoa on iteration 2 running pipeline standard with tuple counts:
         120   ~2%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

           0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
           0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

          91   ~3%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                      {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
          81   ~3%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

         269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                      {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
          39   ~1%    {5}    | SCAN OUTPUT In.5, In.0, In.1, In.2, In.3
        1295   ~0%    {6}    | JOIN WITH containerparent ON FIRST 1 OUTPUT Rhs.1, Lhs.3, _, Lhs.1, Lhs.2, Lhs.4
        1295   ~0%    {6}    | REWRITE WITH Out.2 := 1
          34   ~1%    {4}    | JOIN WITH `cached_FileSystem::Container.splitAbsolutePath/2#dispred#dc97b0cc` ON FIRST 3 OUTPUT Lhs.5, Lhs.3, Lhs.4, Lhs.0

         115   ~0%    {4} r5 = r2 UNION r3 UNION r4
         115   ~0%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                      return r5
```

After
```
Evaluated relational algebra for predicate FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d@4fb6e6v7 on iteration 2 running pipeline standard with tuple counts:
        120   ~0%    {4} r1 = SCAN `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev_delta` OUTPUT In.1, In.0, In.2, In.3

          0   ~0%    {6} r2 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#1` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
          0   ~0%    {4}    | SCAN OUTPUT In.2, In.0, In.1, In.4

         91   ~0%    {6} r3 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Lhs.1, Lhs.2, Lhs.3, _
                     {6}    | REWRITE WITH Tmp.5 := 1, Out.5 := (InOut.1 - Tmp.5), TEST Out.5 = InOut.3
         81   ~0%    {4}    | SCAN OUTPUT In.4, In.0, In.1, In.2
         81   ~5%    {4}    | JOIN WITH containerparent_10#join_rhs ON FIRST 1 OUTPUT Lhs.3, Lhs.1, Lhs.2, Rhs.1

        269   ~0%    {7} r4 = JOIN r1 WITH `_FileSystem::Folder::Append<PathResolution::shouldAppend>::getComponent/2#a6e36a04#join_rhs#2` ON FIRST 1 OUTPUT Lhs.0, Rhs.1, Rhs.2, Lhs.1, Lhs.2, Lhs.3, _
                     {7}    | REWRITE WITH Tmp.6 := 1, Out.6 := (InOut.1 - Tmp.6), TEST Out.6 = InOut.4
         39   ~3%    {5}    | SCAN OUTPUT In.5, In.2, In.0, In.1, In.3
         34   ~0%    {4}    | JOIN WITH `FileSystem::Folder::Append<PathResolution::shouldAppend>::getAChildContainer/2#2e91feca` ON FIRST 2 OUTPUT Lhs.4, Lhs.2, Lhs.3, Rhs.2

        115   ~2%    {4} r5 = r2 UNION r3 UNION r4
        115   ~2%    {4}    | AND NOT `FileSystem::Folder::Append<PathResolution::shouldAppend>::appendStep/3#bed54f6d#prev`(FIRST 4)
                     return r5
```
@hvitved hvitved marked this pull request as ready for review April 22, 2025 14:59
@Copilot Copilot AI review requested due to automatic review settings April 22, 2025 14:59
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)
  • shared/util/codeql/util/FileSystem.qll: Language not supported

@hvitved hvitved added the no-change-note-required This PR does not need a change note label Apr 22, 2025
Copy link
Contributor

@aschackmull aschackmull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hvitved hvitved merged commit 959a79f into github:main Apr 23, 2025
36 of 37 checks passed
@hvitved hvitved deleted the shared/file-system-append-join-fix branch April 23, 2025 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-change-note-required This PR does not need a change note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants