Skip to content

Removed all Dir.chdir calls from private use #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from

Conversation

mbishop-fiksu
Copy link

When writing command-line tools that use ruby-git to interface with git, there are often ways to achieve better performance by running git query commands in parallel. This is unworkable with ruby-git given it's extensive use of the non-thread-safe Dir.chdir call. With all implicit calls to Dir.chdir removed, ruby-git is one step closer to being usable amongst threads.

From the Dir.chdir documentation:

The return value of chdir is the value of the block. chdir blocks can be nested, but in a multi-threaded program an error will be raised if a thread attempts to open a chdir block while another thread has one open.

@robertodecurnex
Copy link
Contributor

Ty @mbishop-fiksu ! I've started that over the thread_safety branch, just fixing the worst of these chdirs.
Let's see what's wrong with Travis and how can we make the gem even more safe (time will tell if there are still other spots for thread problems).

@robertodecurnex robertodecurnex added this to the 1.2.9 milestone Nov 28, 2014
@robertodecurnex robertodecurnex self-assigned this Nov 28, 2014
@robertodecurnex
Copy link
Contributor

Note that I took the --git-dir and --work-tree way to run the commands. I'll see how can we mege efforts.

@robertodecurnex
Copy link
Contributor

Hey @mbishop-fiksu can you please rebase your branch to master and push -f it ?

There was a problem with dates and tests.

Ty.

@mbishop-fiksu
Copy link
Author

I'd be happy to. I'll get to this after the thanksgiving break. Thank you for the heads up!

We know this is safe because a- there is a bug where `build_list` is being called, but that's obviously a bug copied from `config_list`.
Also, if `@git_dir` is non-nil, `#command` will already ensure the command is run within `@git_dir` as the working directory.
We know this is safe because if `@git_dir` is non-nil, `#command_lines` will already ensure the command is run within `@git_dir` as the working directory.
@mbishop-fiksu
Copy link
Author

Ok, rebased and pushed. There is still a Ruby 1.8 incompatibility (I'm using Open3.capture2) so I'll try and work around that.

@mbishop-fiksu
Copy link
Author

I've re-added back 1.8-compatibility, but at the cost of re-adding the Dir.chdir calls. popen doesn't accept a hash of options where we could add the chdir option.

@mbishop-fiksu
Copy link
Author

I'm not sure the failure in the test suite is the code. What do you think?

@robertodecurnex
Copy link
Contributor

It's a build problem, happens frequently for some ruby versions (not our fault).

Just sent a re-run, hope that will work. If not we should probably wait and try again later.

@mbishop-fiksu
Copy link
Author

Bah, same thing. Ok, we'll wait I guess...

@mbishop-fiksu
Copy link
Author

@robertodecurnex Think it's worth hitting that test up again or should we wait a little more? Also, I can't remove all the Dir.chdir statements for Ruby 1.8. Think that's a showstopper?

@robertodecurnex
Copy link
Contributor

Ruby 1.8 compatibility is definitely a show-stopper for the 1.x line.

Since there are a bunch of Ruby 1.8 issues that can be easily solved with Ruby >= 1.9 I will be removing 1.8 support on git 2.0 (the next release after 1.2.9).

I'll take your changes to the thread_safety branch and that would be probably the code base for 2.0.

@robertodecurnex robertodecurnex modified the milestones: 1.2.10, 1.2.9, 2.0.0 Dec 2, 2014
@mbishop-fiksu
Copy link
Author

@robertodecurnex I think this could still be considered for the 1.2.9 release and here's my reasoning. The change improves thread-safety for rubies from 1.9 on while the behavior remains the same for rubies < 1.9. Additionally, no public interface is changed. It doesn't fail on ruby 1.8. It remains the same. What do you think?

@jbohren
Copy link

jbohren commented Dec 16, 2014

@mbishop-fiksu I just came across this when trying to use ruby-git multithreaded. With the current state of 0f78b67, can you still fetch/init/checkout in parallel without chdir-related problems?

@robertodecurnex
Copy link
Contributor

@jbohren since your problems would be just on the way that command make the system calls you should be fine using this branch: https://github.com/schacon/ruby-git/tree/thread_safety

Mostly of the thread safety improvements will be also available under the next release (soon).

@jbohren
Copy link

jbohren commented Dec 16, 2014

@jbohren since your problems would be just on the way that command make the system calls you should be fine using this branch: https://github.com/schacon/ruby-git/tree/thread_safety

Cool, I'll check that out. Thanks!

@robertodecurnex robertodecurnex force-pushed the master branch 2 times, most recently from 3fc1c27 to b224195 Compare February 25, 2016 17:56
@stale
Copy link

stale bot commented Apr 1, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Apr 1, 2018
@stale stale bot closed this Apr 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants