Skip to content

Isolate Dir.chdir to a new process, or mutex #372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 15 commits into from
Closed

Isolate Dir.chdir to a new process, or mutex #372

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3d789f7
[WIP]Start replacing Dir.chdir with paths
Jun 25, 2018
3ad8851
Merge branch 'master' into 355
taquitos Jun 25, 2018
ba1df97
Merge branch 'master' into 355
taquitos Jun 25, 2018
f3f7518
fork() when using Dir.chdir, otherwise, use a mutex
Jun 25, 2018
f1cd6de
Merge pull request #1 from taquitos/355
taquitos Jun 25, 2018
df31b57
fork() when using Dir.chdir, otherwise, use a mutex
Jun 25, 2018
22ae609
wrap calls to in a semaphore
Jun 25, 2018
5449c01
Merge pull request #2 from taquitos/355
taquitos Jun 25, 2018
ae1baa3
Merge remote-tracking branch 'upstream/master'
Jul 17, 2018
e57f46f
Merge remote-tracking branch 'upstream/master'
Aug 13, 2018
0d32576
Revert "355"
taquitos Aug 13, 2018
a100a36
Merge pull request #3 from taquitos/revert-2-355
taquitos Aug 13, 2018
1cdc5fd
Revert "355"
taquitos Aug 13, 2018
cdcc6a5
Merge pull request #4 from taquitos/revert-1-355
taquitos Aug 13, 2018
0fb9798
Isolate Dir.chdir to a new process, or mutex
Aug 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fork() when using Dir.chdir, otherwise, use a mutex 
Signed-off-by: Joshua Liebowitz <taquitos@google.com>
  • Loading branch information
Joshua Liebowitz committed Jun 25, 2018
commit df31b573da5806b10d4fa0c243bf865d155b831c
35 changes: 30 additions & 5 deletions lib/git/base.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
module Git

class Base
# Adding a mutex to the class because each repo should be sharing the same mutex
# in case we need to Dir.chdir and we don't have fork() support to isolate that
class << self
attr_accessor :chdir_semaphore
end
Git::Base.chdir_semaphore = Mutex.new
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works perfectly fine, but maybe the approach below would be even better. It shouldn't really be an accessor but more like a reader since it's an immutable data structure that shouldn't be tampered with elsewhere, right?

class << self
  def chdir_semaphore
    @chdir_semaphore ||= Mutex.new
  end
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right, but are you sure that's threadsafe? If we have 2 threads that call chdir_semaphore before it is set, is it possible they both end up calling Mutex.new?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. So maybe something like this then:

@chdir_semaphore = Mutex.new

class << self
  attr_reader :chdir_semaphore
end

(if I remember the context right - the `@chdir_semaphore call in normal class context will assign it to the class-level variable. Please try this out before trusting me blindly on this.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perlun I tested with your suggestion, it works 👍


include Git::Base::Factory

Expand Down Expand Up @@ -92,18 +98,37 @@ def initialize(options = {})
@index = options[:index] ? Git::Index.new(options[:index], false) : nil
end

# changes current working directory for a block
# to the git working directory
# changes current working directory for a block to the git working directory.
#
# Note: If we can fork() or spawn(), Dir.chdir will happen in a new process
# otherwise, we will use a mutex to prevent threading errors
# See https://github.com/ruby-git/ruby-git/issues/355 for more info
#
# example
# @git.chdir do
# # write files
# @git.add
# @git.commit('message')
# end
def chdir # :yields: the Git::Path
Dir.chdir(dir.path) do
yield dir.path
def chdir(&block) # :yields: the Git::Path
chdir_block = Proc.new do
Dir.chdir(dir.path) do
block.call(dir.path)
end
end

if Process.respond_to?(:fork)
# Forking this process so that we can be threadsafe
pid = Process.fork do
chdir_block.call
end
Process.wait(pid)
else
# Windows and NetBSD 4 don't support fork()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting anecdote, I wasn't aware of NetBSD 4 here. But OTOH, it's past EOL already. Maybe we can drop that reference and instead mention JRuby (which is actively used by many), since it also has problems with fork() on the JVM.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True dat

# let's use a mutex to prevent race conditions with threads
Git::Base.chdir_semaphore.synchronize do
chdir_block.call
end
end
end

Expand Down