Skip to content

Add Pathlib #1840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 18, 2020
Merged

Add Pathlib #1840

merged 11 commits into from
Apr 18, 2020

Conversation

palaviv
Copy link
Contributor

@palaviv palaviv commented Apr 4, 2020

This change add pathlib and changes many os method to support path like objects.
Many of the tests fail because Rust does not support Cstring with nul. Not sure how to bypass that. Would appreciate suggestion on how to solve this.

@coolreader18
Copy link
Member

I did implement TryFromObject for CString recently, but you could also just have a nulcheck function that makes sure a string doesn't have a nul char in it, and returns a ValueError if it does, then just call nulcheck(s, vm)?;. (Or maybe a vm method?)

@palaviv
Copy link
Contributor Author

palaviv commented Apr 4, 2020

I did implement TryFromObject for CString recently, but you could also just have a nulcheck function that makes sure a string doesn't have a nul char in it, and returns a ValueError if it does, then just call nulcheck(s, vm)?;. (Or maybe a vm method?)

The problem is that CPython support nul char while rust throws exception.

os.stat("/dev/null\x00")

Should not throw an error.

@coolreader18
Copy link
Member

coolreader18 commented Apr 4, 2020

Ah, I see; that is tricky.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems only trailing \x00 exists in tests. Trimming them before making PyPathLike would work?

This is not specifically related to this PR, but while reading, i found i have no idea why PyPathLike is not PyPath but Like

@palaviv
Copy link
Contributor Author

palaviv commented Apr 9, 2020

It seems only trailing \x00 exists in tests. Trimming them before making PyPathLike would work?

This will actually give the opposite result. As /dev/null should be different then /dev/null\x00

This is not specifically related to this PR, but while reading, i found i have no idea why PyPathLike is not PyPath but Like

Thats how they are called in CPython.

@youknowone
Copy link
Member

I try to test this problem a bit, but I can't make null-included path working. Could you describe your working environment?

macOS, python 3.8.1, APFS

>>> open('abc\x00', 'w')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: embedded null byte
>>> import os
>>> os.stat('/dev/null\x00')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: embedded null byte
>>>

Linux, Python 3.5.2, ext4

>>> open('abc\x00', 'w')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: embedded null byte
>>> import os
>>> os.stat('/dev/null\x00')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: embedded null byte

@palaviv
Copy link
Contributor Author

palaviv commented Apr 11, 2020

@palaviv
Copy link
Contributor Author

palaviv commented Apr 17, 2020

@coolreader18 @youknowone can this be pushed and we will solve the nul later?

@youknowone
Copy link
Member

It seems Vec<u8> is the reasonble internal data of path types. And they may be used to create ffistring in future when they are actually used. Is it possible to fix now? or do you want to merge this for right now as it is?

@palaviv
Copy link
Contributor Author

palaviv commented Apr 17, 2020

It seems Vec<u8> is the reasonble internal data of path types. And they may be used to create ffistring in future when they are actually used. Is it possible to fix now? or do you want to merge this for right now as it is?

I am not sure I understand... Would you like to change the OsString to Vec<u8>?

@youknowone
Copy link
Member

I guess so. I didn't check cpython implementation well yet, but it seems it is more close to Vec<u8> than OsString. Or enum { Bytes(Vec<u8>), Str(String) } if there are difference by str/bytes.

@coolreader18
Copy link
Member

@palaviv I think fixing it later would be fine, it's not too critical to pathlib, and I think it would rarely happen in practice.

@palaviv palaviv merged commit f60b07a into RustPython:master Apr 18, 2020
@palaviv
Copy link
Contributor Author

palaviv commented Apr 18, 2020

I have merged this and we can do the proposed changes in the future

@youknowone
Copy link
Member

I checked about PyPathLike impl but it revealed enum is not fit for the use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants