Skip to content

[WIP] Shallow smart transport support #4747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 20 commits into from

Conversation

tiennou
Copy link
Contributor

@tiennou tiennou commented Aug 2, 2018

This is the Future ! protocol-level changes needed to support shallow cloning. It's a complete WIP, but I have a bunch of issues with related code and PR are nice for discussions.

Note that it's been a while since I've been chipping at it, so I might not remember all the details. Hopefully, I can still make sense of the problems I've encountered.

@tiennou
Copy link
Contributor Author

tiennou commented Aug 2, 2018

The first thing is that I've been coming uncomfortably close to the public/private API when I added b720e72 (fetch: pass negotiation data around as a struct), a preparatory API cleanup because shallow clones need to keep track of some state when they're done. I originally had a git_array_oid_t in 99d7265 (WIP smart: negotiate server capabilities), but since that's private I can't put it in a public struct, hence I'm publishing a typedef-ed version of it (dunno if that's fine). It's either that or packing the array in a git_oidarray (public) and unpacking it afterward (as my first draft was doing). I think that's fine, but since we're aiming for API stability maybe it'd be best to split up at least that first commit before 1.0.

@tiennou
Copy link
Contributor Author

tiennou commented Aug 2, 2018

The second issue is that state I alluded to above : I've grown to be somewhat dubious of the amount of state going on in git_remote (the (active_|passive_|)refspecs members are an example of it). Now that I'm going to add a list of "future OIDs to shallow after the download is done", it's only getting worse, especially since the functions to perform a fetch are all public (ie. git_remote_fetch vs. git_remote_connect + git_remote_download vs. git_remote_push, all having a tendency to stomp over previous callers' work). Add to that that git_push is interwoven (1, 2, 3) into all of this and it's (IMHO) really confusing to know what's used when, by who. Especially update_tips is confusing : am I supposed to have shallowed the necessary OIDs before, or after it's done ? (that's rhetorical, since I did some checking to see how fetches are performed in cgit, and yet I'm still not sure…).

So I'm envisioning an API where git_remote is simplified by removing/deprecating its connect/disconnect/stop, download/upload/update_tips functions, and a new git_fetch object is made to parallel git_push. That would simplify the internals of git_remote and makes it clear where an operation's state is.

@tiennou
Copy link
Contributor Author

tiennou commented Aug 10, 2018

Ref: #2516, which has @carlosmn arguing for the other thing to happen : git_push gets merged in git_remote.

@tiennou tiennou force-pushed the shallow/smart-transport branch 2 times, most recently from 4d157b1 to 47474c7 Compare October 25, 2018 23:07
This allows us to pick which data from a commit we're interested in. This will be used by the revwalk code, which is only interested in parents' and committer data.
This represents (old-style) grafted commits, a.k.a an array of overridden parents for a commit's OID.
This wires git_repository to open the .git/info/grafts file and load its contents as git_commit_grafts objects.
When doing the initial revwalking, we don't want to pollute the ODB with objects which will get grafted.
Alternatively, it might be better to manually prune the cache when reload grafts/shallow
We will need to pass some options to the negotiation step (like depth level for shallow clones)
We need to keep track of the OIDs the server told us to shallow, so we
can write those when we're done downloading.
@tiennou tiennou force-pushed the shallow/smart-transport branch from 47474c7 to bb5ef67 Compare February 3, 2019 09:22
Base automatically changed from master to main January 7, 2021 10:09
This was referenced Aug 30, 2022
@ethomson
Copy link
Member

ethomson commented May 9, 2023

Merged in #6393

@ethomson ethomson closed this May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants