Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

subdatasets command doesn't report "safe directory" warnings from git #7651

Open
adswa opened this issue Sep 3, 2024 · 1 comment
Open

subdatasets command doesn't report "safe directory" warnings from git #7651

adswa opened this issue Sep 3, 2024 · 1 comment

Comments

@adswa
Copy link
Member

adswa commented Sep 3, 2024 •
edited
Loading

In a git repo I do not own, datalad subdatasets fails with

adina@juseless in /data/project/ukb_vbm/ukb_cat/code
❱ datalad subdatasets
CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false ls-files --stage -z -- code/pipeline' failed with exitcode 128
adina@juseless in /data/project/ukb_vbm/ukb_cat/code
❱ datalad --version                                                       128 !
datalad 1.1.0

git-annex, on the other hand, bubbles the underlying issue up.

adina@juseless in /data/project/ukb_vbm/ukb_cat/code
❱ git annex whereis pipeline/.datalad/environments/cat/image
git-annex: Git refuses to operate in this repository,
probably because it is owned by someone else.

To add an exception for this directory, call:
	git config --global --add safe.directory /data/project/ukb_vbm/ukb_cat

Once this is configured, datalad subdatasets is happy again.

Other commands handle it better, and report the underlying issue:

(hacking) adina@juseless in /data/project/ukb_vbm/ukb_cat
❱ datalad status                                                          128 !
CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false ls-files --stage -z --exclude-standard -o --directory --no-empty-directory' failed with exitcode 128
fatal: detected dubious ownership in repository at '/data/project/ukb_vbm/ukb_cat'
To add an exception for this directory, call:

	git config --global --add safe.directory /data/project/ukb_vbm/ukb_cat
@adswa adswa changed the title Datalad can't handle "safe directory" warnings from git subdatasets command doesn't report "safe directory" warnings from git Sep 4, 2024
@adswa
Copy link
Member Author

adswa commented Sep 5, 2024

I've rummaged in the internals frustratingly long, but I have no idea why subdatasets swallows the error. It is present:

<generator object GitWitlessRunner.run_on_filelist_chunks_items_ at 0x7f41ff7596c0>
(Pdb) l
354  	                cmd,
355  	                protocol=GeneratorStdOutErrCapture,
356  	                env=env)
357  	
358  	        line_splitter = {
359  ->	            STDOUT_FILENO: LineSplitter(sep, keep_ends=True),
360  	            STDERR_FILENO: LineSplitter(sep, keep_ends=True)
361  	        }
362  	
363  	        for file_no, content in generator:
364  	            if file_no in (STDOUT_FILENO, STDERR_FILENO):
(Pdb) n
> /home/adina/datalad/datalad/dataset/gitrepo.py(360)_generator_call_git()
-> STDERR_FILENO: LineSplitter(sep, keep_ends=True)
(Pdb) [print(i) for i in generator]
(2, "fatal: detected dubious ownership in repository at '/data/project/ukb_vbm/ukb_cat'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /data/project/ukb_vbm/ukb_cat\n")
*** datalad.runner.exception.CommandError: CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false ls-files --stage -z -- code/pipeline' failed with exitcode 128

but then it gets lost.

In status, it gets bubbled up, e.g.

❱ datalad status                                               128 !
> /home/adina/datalad/datalad/dataset/gitrepo.py(81)git_ignore_check()
-> e.stdout = "".join(stdout_buffer) if stdout_buffer else (e.stdout or "")
(Pdb) p e
CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false rev-parse --quiet --verify 'HEAD^{commit}'' failed with exitcode 128
(Pdb) n
> /home/adina/datalad/datalad/dataset/gitrepo.py(82)git_ignore_check()
-> e.stderr = "".join(stderr_buffer) if stderr_buffer else (e.stderr or "")
(Pdb) p stderr_buffer
["fatal: detected dubious ownership in repository at '/data/project/ukb_vbm/ukb_cat'\n", 'To add an exception for this directory, call:\n', '\n', '\tgit config --global --add safe.directory /data/project/ukb_vbm/ukb_cat\n']

The fundamental command call call_git_items_(['ls-files', '--stage', '-z'])) has it, it must be something specific that later disregards the crucial message.

❱ python3       127 !
Python 3.11.2 (main, Aug 26 2024, 07:20:54) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from datalad.support.gitrepo import GitRepo
>>> r=GitRepo('.')
>>> list(r.call_git_items_(['ls-files', '--stage', '-z']))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/datalad/dataset/gitrepo.py", line 525, in call_git_items_
    for file_no, line in self._generator_call_git(
  File "/usr/lib/python3/dist-packages/datalad/dataset/gitrepo.py", line 363, in _generator_call_git
    for file_no, content in generator:
  File "<frozen _collections_abc>", line 330, in __next__
  File "/usr/lib/python3/dist-packages/datalad/runner/nonasyncrunner.py", line 100, in send
    return self._locked_send(message)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/datalad/runner/nonasyncrunner.py", line 125, in _locked_send
    self._check_result()
  File "/usr/lib/python3/dist-packages/datalad/runner/nonasyncrunner.py", line 96, in _check_result
    self.runner._check_result()
  File "/usr/lib/python3/dist-packages/datalad/runner/nonasyncrunner.py", line 298, in _check_result
    raise CommandError(
datalad.runner.exception.CommandError: CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false ls-files --stage -z' failed with exitcode 128 [err: 'fatal: detected dubious ownership in repository at '/data/project/ukb_vbm/ukb_cat'
To add an exception for this directory, call:

	git config --global --add safe.directory /data/project/ukb_vbm/ukb_cat']
>>>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@adswa