Skip to content

FIX Remove temporary filenames from command list. #20220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

cmarmo
Copy link
Contributor

@cmarmo cmarmo commented Jun 7, 2021

Reference Issues/PRs

Fixes #20212

What does this implement/fix? Explain your changes.

#19650 did not exclude temporary filenames from the argument list.

Any other comments?

ping @NicolasHug ? Thanks!

Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cmarmo.

When we merge this we should open another issue so that we can test against this to avoid breaking the build instructions again.

setup.py Outdated
@@ -265,7 +265,8 @@ def setup_package():
package_data={'': ['*.pxd']},
**extra_setuptools_args)

commands = [arg for arg in sys.argv[1:] if not arg.startswith('-')]
commands = [arg for arg in sys.argv[1:] if not arg.startswith('-') and
not arg.startswith('/')]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the / come from a file-path or something like that? Will this also work on Windows where the file separator is \?

Eitherway it would be good to add a small comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows filenames would even typically start with r"C:\\", r"D:\\" to r"Z:\\" as often used for shared folders on virtual machines... (I guess r"A:\\" and r"B:\\" also would be possible). And maybe r"\\" as well but I am not 100% sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right sorry, I'm not very good at Windows fixes... This will definitely not work with Windows. I will try harder.

@ogrisel
Copy link
Member

ogrisel commented Jun 7, 2021

#19650 did not exclude temporary filenames from the argument list.

I don't understand how this PR fixes the problem described in #20212, could you explain a in a bit more details? In particular what temporary filenames are your referring to?

@cmarmo
Copy link
Contributor Author

cmarmo commented Jun 7, 2021

#19650 did not exclude temporary filenames from the argument list.

I don't understand how this PR fixes the problem described in #20212, could you explain a in a bit more details? In particular what temporary filenames are your referring to?

Before #19650 setup.py command line was parsed: if the argument was one in the list then the action was required to succeed without Numpy. After #19650 the list is built excluding arguments starting with -, the list variable commands then contains eg ['dist_info', '/tmp/pip-modern-metadata-rdgqd8q0'], where '/tmp/pip-modern-metadata-rdgqd8q0' is some temporary file needed in the build process, I guess. The filename is not in the list of commands tested here so the check on numpy is done even if is not necessary. To make it compatible with Windows it's probably more interesting to check if argument is a file?

@cmarmo
Copy link
Contributor Author

cmarmo commented Jun 7, 2021

After deeper reading of the traceback

Created temporary directory: /tmp/pip-modern-metadata-zdrbdtak

So I need to check if this is a directory...

commands = [arg for arg in sys.argv[1:] if not arg.startswith('-')]
# Exclude from the list of arguments options and temporary directories
commands = [arg for arg in sys.argv[1:] if not arg.startswith('-') and
not os.path.isdir(arg)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or even os.path.exists?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am worried about the extra system calls and the facts that we might have other kinds of command arguments that are not file that we should neither treat as commands themselves.

@ogrisel
Copy link
Member

ogrisel commented Jun 7, 2021

Maybe it would be more robust to do the opposite and explicitly list commands that require numpy.

I assume they are wheel, bdist_*, build, build_ext and install. @jeremiedbb do you see any other?

sdist might need numpy but I don't think so, we need to check.

@cmarmo
Copy link
Contributor Author

cmarmo commented Jun 7, 2021

Maybe it would be more robust to do the opposite and explicitly list commands that require numpy.

I assume they are wheel, bdist_*, build, build_ext and install. @jeremiedbb do you see any other?

sdist might need numpy but I don't think so, we need to check.

But then you just might want revert #19650?

@ogrisel
Copy link
Member

ogrisel commented Jun 7, 2021

But then you just might want revert #19650?

Well #19650 fixed another bug of its own.

@ogrisel
Copy link
Member

ogrisel commented Jun 8, 2021

I think it's worth trying to implement the strategy described in #20220 (comment) in a dedicated PR and run [cd build] to check that it does not break the build procedure.

@cmarmo cmarmo closed this Jun 8, 2021
@cmarmo cmarmo deleted the setup-isolation branch August 26, 2022 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build dependencies in developer guide seem incorrect
3 participants