-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
ENH: Improve support for POWER7 and POWER8 machines #8134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Avoid IBM extended double as machar algorithm doesn't converge. Configured it as opt-in as POWER9 will have native floats of 128 bit length.
Now `./runtests.py` on PPC64 outputs: Ran 6273 tests in 96.508s OK (KNOWNFAIL=6, SKIP=14)
This is linked to the following discussion on the mailing list: |
Do you have any other tests than the ones found on numpy source? Maybe from a major framework that uses numpy which is also supposed to work? Tell if you have any please so I can also verify. |
what happens if one just fixes the precision determined by finfo? |
Fixing the precision? Well, I can take a look at that. If you can send me these patches I'd appreciate. Just to be clear about the current behavior: This PR outputs: >>> np.finfo(np.longdouble)
finfo(resolution=1e-15, min=-1.7976931348623157e+308, max=1.7976931348623157e+308, dtype=float64) And the master outputs: >>> np.finfo(np.longdouble)
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:127: RuntimeWarning: overflow encountered in add
a = a + a
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:129: RuntimeWarning: invalid value encountered in subtract
temp1 = temp - a
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:138: RuntimeWarning: invalid value encountered in subtract
itemp = int_conv(temp-a)
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:162: RuntimeWarning: overflow encountered in add
a = a + a
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:164: RuntimeWarning: invalid value encountered in subtract
temp1 = temp - a
/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py:171: RuntimeWarning: invalid value encountered in subtract
if any(temp-a != zero):
finfo(resolution=1e-75, min=--9.22337203471e+18, max=-9.22337203471e+18, dtype=float128) |
so you are aware of the ibm extended double implementation which uses 2 doubles to represent a long double. Exponent is the same as double but it has doubled the mantissa size, so the precision is better. Do you have a requirement to use his IBM Extended Double on PPC64? It could be hardcoded as #2669 suggested. |
yes we are aware of the different representation, there is specific code to support it. It would be good if we can keep it by just fixing the wrong np.finfo output and maybe adapting some tests boundaries. |
I note that the long double format is detected during build, to include IBM both little endian and big endian, so we could expose that information. I'd like to expose the various macros generated during build in any case. |
@juliantaylor : But then I guess that the algorithm in machar would need to be fix, right? The little I understood from that algorithm, it looked like it'd be a major update in order to be able to detect the peculiar precision of IBM extended double, right? Only by fixing |
How should we then proceed? |
@charris @juliantaylor ping? |
@juliantaylor If the Debian patches fix the problem, I'd like to get hold of them. In general, I'd like to fix finfo if that does the trick. |
@charris finfo output relies on a machar call, right? Or do you intend in fixing finfo somewhere else? |
I actually think that we should dump machar and hard code the values. Enough info is gathered during the build that we know what types we have, which are basically IEEE and IBM double_double long doubles. We only claim to support IEEE in any case but can probably make the IBM type work. However, we need to expose a bit more of the build information, so that needs to be the first order of business. |
Should be closed by #8504. |
I confirm there is no issue when running numpy tests on python 3.5.2 and 2.7.12. |
I noticed that those PowerPC machines (as ppc64le architecture) were not passing
all tests of numpy. Mainly because of how
long double
type is implemented:https://gcc.gnu.org/wiki/Ieee128PowerPC#A1.3_Sizes
It uses IBM Extended Double and machar algorithm doesn't converge to determine
the long double precision. For that reason I chose to change the
long double
size to be 8 bytes (just as
double
). The consequence of doing that is thatonly the precision (mantissa size) is decreased as the exponent remains the
same.
Tests are all passing after I removed
long double
type of one of them.