Skip to content

server: uploadpack, implement multi_ack capability #1204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 17, 2024

Conversation

manland
Copy link
Contributor

@manland manland commented Oct 8, 2024

I push to v6 (because I use it on my project) but I think it should work on v5.

Should close #647 (at least on my project).

I'm not sure why, but when I add ar.Capabilities.Add("multi_ack") in my project I can delete the Capabilities.Add("no-thin") capability.

@manland manland force-pushed the multi_ack branch 2 times, most recently from 9f04bee to cc22bde Compare October 8, 2024 14:06
@pjbgf
Copy link
Member

pjbgf commented Oct 9, 2024

@manland thanks for working on this, at first sight it is looking good. The CI errors are intermittent and orthogonal.

@prologic would you please be able to confirm this resolves the problem you were experiencing on the linked issue?

@pjbgf
Copy link
Member

pjbgf commented Oct 10, 2024

@manland as part of multi_ack implementation, please review the comments on these two tests:

And remove:

  • The capability.MultiACK from the UnsupportedCapabilities. This should trigger a few tests to fail, as they expect multiack to not be supported, so they would also need to be updated.
  • This comment and adhoc code to enable support for Azure DevOps.

This becomes an invalid state, its func needs updating, so that it can encode ServerResponse correctly (with multiple ACKs), while also returning an error if multiple ACKs are returned when isMultiACK is false.

@pjbgf pjbgf added this to the v6.0.0 milestone Oct 10, 2024
@pjbgf
Copy link
Member

pjbgf commented Oct 10, 2024

Side note: Once this is merged, the caveats called out in #64 for Azure DevOps are no longer required for users using the v6-exp branch.

The Azure DevOps example becomes redundant, as the other examples will just work:
go run _examples/memory/main.go <DEV_OPS_REPO>. This has been tested based on this PR + recommendations above, and worked as intented.

@manland
Copy link
Contributor Author

manland commented Oct 10, 2024

@pjbgf Thank you for review. I fixed what you found.

@manland
Copy link
Contributor Author

manland commented Oct 17, 2024

@pjbgf Here we are! An implem of single ack and multi_ack compatible with current API! I'm happy because I have put lot of efforts, try, fail and rework before all is ok (I have read c git code to understand what "acceptable commit" is 😄 ) I had added some tests too.

In my project (a simple git server) this is how I test it:

  • make a repo
  • clone it
  • push 300 commits
  • clone it elsewhere
  • return to the first clone
  • delete commits, and add 300 new commits, push force
  • in the second clone git pull --rebase
  • in single ack and multi_ack all is ok!!

Future tests could be to make same scenario but with 300 branches of 300 commits and to cut half commits of all branches, I think it's the worst case possible, but depend a lot of the client strategy to traverse the tree.

Please review but not merge, all is in place to add multi_ack_detailed... soon in a next commit!

Just tell me if you are OK with this implem, I'll squash bad commit and push multi_ack_detailed!

@manland manland requested a review from pjbgf October 17, 2024 15:20
@pjbgf
Copy link
Member

pjbgf commented Oct 25, 2024

@manland nice work so far, I will be taking a proper look over the weekend and provide some additional feedback.

@manland
Copy link
Contributor Author

manland commented Nov 4, 2024

@pjbgf Just to let you know I'm on it! I successfully implemented multi_ack_detailed but when adding more complex scenario (in my project) found a bug in the nak/ack algo when encoding response.

Unfortunately it's too late for me, I'm pretty sure after a big sleep all will become more simple 😅

@manland
Copy link
Contributor Author

manland commented Nov 6, 2024

@pjbgf All is good now!

I have squash previous commits and added a new one with multi_ack_detailed.

To test this dev I had:

  • build a little git server with go-git
  • init a repo with 50 branch containing 50 commits each (2500 commits)
  • clone this repo elsewhere
  • go to first clone and randomly recreate commit for each branch
  RESET=$(shuf -i 0-$NB_COMMIT -n 1)
  real_quiet_git reset --hard HEAD~$RESET
  count=$((NB_COMMIT-RESET))
  for (( j=0; j<=$count; j++ ))
  do
    echo "${i}_${j}_2" >> issues/bigCommits.txt
    git add .
    git commit -m "my ${i}_${j}_2 commit"
  done
  • then go to second clone and git pull all the world 😅 Yes this test make git crazy 🔨

I have succefully make:

  • single ack with no common hash (NAK) after more than 256 hash exchanged
  • single ack with at least one common hash
  • idem (nak and ack) for multi_ack and multi_ack_detailed

The client part should be done here with a reader for the server response. This is out of the scope of this PR which focus on server side.

For now I prefer to try to implement depth (aka shadow) and filter. If someone need help to do the client side, please ping me as I have read all specs and code from git about this subject ❤️

@pjbgf
Copy link
Member

pjbgf commented Nov 17, 2024

For now I prefer to try to implement depth (aka shadow) and filter. If someone need help to do the client side, please ping me as I have read all specs and code from git about this subject

@manland that sounds great. Feel free to create an issue with some links/additional information, so that whoever is keen on picking this up can ping you on it. 🙇

Copy link
Member

@pjbgf pjbgf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manland thanks for implementing multi_ack, this was a feature that go-git users were keen on for a very long time. 🥇 🙇

@pjbgf pjbgf merged commit 858d421 into go-git:v6-exp Nov 17, 2024
14 checks passed
pjbgf added a commit to pjbgf/go-git that referenced this pull request Nov 17, 2024
Since the multi_ack implementation (go-git#1204), Azure DevOps works
out of the box, no longer requiring code changes. Therefore,
the previous devops specific example is no longer needed.

Signed-off-by: Paulo Gomes <pjbgf@linux.com>
aymanbagabas added a commit to aymanbagabas/go-git that referenced this pull request Dec 31, 2024
…ities

This adds support for multi-ack and multi-ack-detailed capabilities in
the transport package. The support includes both the client and server.

Related: go-git@2ebd65f
Related: go-git#1204
aymanbagabas added a commit to aymanbagabas/go-git that referenced this pull request Dec 31, 2024
…ities

This adds support for multi-ack and multi-ack-detailed capabilities in
the transport package. The support includes both the client and server.

Related: go-git@2ebd65f
Related: go-git#1204
aymanbagabas added a commit to aymanbagabas/go-git that referenced this pull request Dec 31, 2024
…ities

This adds support for multi-ack and multi-ack-detailed capabilities in
the transport package. The support includes both the client and server.

Related: go-git@2ebd65f
Related: go-git#1204
aymanbagabas added a commit to aymanbagabas/go-git that referenced this pull request Jan 2, 2025
…ities

This adds support for multi-ack and multi-ack-detailed capabilities in
the transport package. The support includes both the client and server.

Related: go-git@2ebd65f
Related: go-git#1204
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants