MinGW enhancements for datetime functionality. #156
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi Han, thanks for working on this. Unfortunately, it still ends the same way for me: |
|
Do I need to enable something in the build? I didn't look at what you did yet, just pulled and built. |
|
Ah, sorry, I had not tried it under wine yet.. I think it tries to fall back on the default |
|
It's not the most common setup I know, but am important one for me. The numpy Windows binaries are always built under Wine. |
|
Are you using an official Python executable? I just installed and built it under wine and it worked for me.. I think with a Python distro built with MinGW, it might be using a different compiler class. Hmm, I also need to test that. |
|
Does your build log indicate that |
|
(By the way, this is my command line: |
|
Yes, Python 2.6.4 binary from python.org. Those are built with Visual Studio. |
|
No, that doesn't occur in the build log. I do see 5 occurrences of |
|
I'll try with the exact same build command now. |
|
Ah, yeah, someone had encountered a similar problem before.. ;-) I'll look into the paver build, thanks! |
|
'Cannot build msvcr library: "msvcr90.dll" not found' --> this seems to be the problem.. |
|
Same segfault with your build command. |
|
|
Yeah, it cannot find the msvc runtime library.. It normally sits in the Python base directory, but the script also looks for it in PATH. |
|
So should I try adding that dir to the path with |
|
[snip] (does not show the one in the Python directory, though.) Well, it could be worth a try, but it should work without explicitly adding it to PATH. Does it exist in the Python directory? What does |
No dll in Python dir. |
|
Seem msvcr71.dll is the default for my wine: |
|
That is interesting, on my machine under wine as well as under windows, the Python installer puts msvcr90.dll in C:\Python27. The msvcr71.dll should have the |
|
I may have an older Wine, but installed Pythons recently. py3.2 looks the same, no dll. I could put it there by hand to test. But I'll have to do this tomorrow on my other machine, don't want to mess up this one too much and get problems right before a scipy release. |
|
Maybe you have to install vcrun2008 with winetricks, but for me that reports an error at the moment. I myself have wine-1.3.18. You have the dll in an SxS directory, though.. I will update the pull request in either case. |
|
Hi Ralf, I pushed a few changes. Now, the installer should be able to pick up the msvcr90.dll from the winsxs directory. Could you please try again? There is still the issue for the fallback-mechanism, but hopefully you do not need to make any more changes to the system. |
|
I had to add another preprocessor statement to reliably fall back on the localtime functionality in the newer versions of MSVC. Now the build should not fail whether or not msvc90.dll is found.. |
|
This works! |
|
Looking at your changes to mingw32ccompiler, that looks cleaner than I expected this solution to be. Learned something again, thanks. Mark should probably review your datetime changes. |
|
Some py3k related issue still: |
| defined(__GNUC__) && \ | ||
| (defined(NPY_MINGW_USE_CUSTOM_MSVCR) | \ | ||
| defined(NPY_MINGW_USE_64BIT_MSVCR)) | ||
| #define time_t __time64_t |
There was a problem hiding this comment.
This should be a typedef, not a #define. In general type aliases should never be done with the preprocessor.
There was a problem hiding this comment.
Ah, thanks, I didn't know that!
|
Oww.. I can reproduce them on WIN7.. I'll take a look at it.. sorry! |
|
The errors are from the issue that windows will not parse timestamps from before 1970. The See also: http://msdn.microsoft.com/en-us/library/a442x3ye.aspx. I am not sure about the best solution, because you might lose functionality when dates from before 1970 are not accepted. |
|
For an older version of MinGW, it might be possible to use the plain Does it give compile errors on MinGW 3.4.5? Or only in tests? Does it try to use the |
|
I've tested 2.5 and 2.7 with current master and MinGW 3.4.5 again just now. It builds, but tests give the following errors: Python 2.5 test errors: Python 2.7 test errors: |
|
And Python 3.1: |
|
The situation looks even worse for me, built in 64-bit windows with Visual Studio 2008: |
|
Doesn't look any worse. Note that I didn't post any failures other than the datetime ones, even though there are a few. The datetime ones are the only release blockers though, the other ones are easily fixed or skipped. |
|
Could it be possible that the 64-bits functions are called with a 32-bits time type? The There is also the issue with dates before 1970. (Which are the failures for me on Win7.) |
|
Ralf, could you, just for testing purposes, replace I don't have a windows machine at hand, at the moment.. |
|
I think the best way to deal with the pre-1970 issue is to throw a ValueError, like it is done here, when the time conversion fails. |
|
Doing that replacement on line 28 results in the following test failures for Python 2.7. There's less failures and not the same ones: |
|
Thanks! That was what I suspected. I think we need to solve two issues here:
What is the complete range for the numpy datetime64 specification/implementation? How are times handled that are out of the platform-supported time range? |
|
Not sure about out-of-range dates, @mwiebe should be able to answer that. |
|
I think it could be a simple solution to define |
|
Ah, I think we might not need the |
|
Would the user have to define |
|
Yeah, it is defined automatically in mingw32ccompiler. |
|
I've made a pull request for the test errors on Windows. Regarding the 1970 epoch, datetime64 and time_t are signed integers, so datetimes before that are just fine. |
|
That leaves just the MinGW-related issues. What I think is important is that the default builds of NumPy not include a Y2038 bug, which is what using the 32-bit time_t will give. Hopefully the issues can be resolved reasonably! |
|
Hi Mark, are you sure that the localtime functions on Windows will accept a negative integer? According to their documentation, they do not. The 64bit time range on windows is defined from the year 1970 to 3000: http://msdn.microsoft.com/en-us/library/a442x3ye.aspx. Have you checked the validity of the datetime64 functionality on linux? Because the linux localtime might behave differently. (Although I still think there is a boundary on linux as well.) |
|
Ah, I see what you mean. For the NumPy implementation, negative integers work perfectly fine. |
|
I think if we force a 64-bits time_t for MinGW, the only thing left that could go wrong is when the 64-bits functionality is not available (e.g. on older platforms, like msvcr < 7). |
|
I made a pull request that should fix the time type on (older versions of) MinGW. Together with the pull request from Mark, this will hopefully fix all the issues with datetime64.. (when available.) |
This was referenced Oct 16, 2012
luyahan
pushed a commit
to plctlab/numpy
that referenced
this pull request
Apr 25, 2024
feat: Add vabdl_[s16|s32|u8|u16|u32]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, as a solution for ticket 1909 and the issues in pull request 118, I made some enhancements (or hacks) to the MinGW compiler class in the NumPy distutils.
Like for the Python runtime library, it will try to generate a custom version of the MSVC runtime library, which exposes all functionality, for versions of VS2005 and up.
I had to hack in a preprocessor statement (NPY_MINGW_USE_CUSTOM_MSVCR), to be able to use this functionality safely with a fallback on the conventional (non-threadsafe) functions. It is a bit messy, but hopefully it will fix the issues with MinGW.