Skip to content

Improve subnormal expectations #339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
honno opened this issue Nov 17, 2021 · 4 comments · Fixed by #341
Closed

Improve subnormal expectations #339

honno opened this issue Nov 17, 2021 · 4 comments · Fixed by #341
Labels
Narrative Content Narrative documentation content.
Milestone
v2021

Comments

@honno
Copy link
Member

honno commented Nov 17, 2021 •
edited
Loading

Currently the spec doesn't mention subnormal floats, so assumedly the IEEE 754 behaviour stands.

For typical CuPy builds subnormals are flushed to zero, as explained by @leofang in numpy/numpy#18536 (comment).

Just FYI. In CuPy we turn flush-to-zero on by default (via setting a compiler flag -ftz=true), see floating.py#L56-L58. The reason is there could be significant computational cost on accelerators like GPU. So, I suspect that CuPy is not the only array library that does this.

It does seem bad if IEEE 754 subnormal behaviour is expected but will be seemingly forever violated by one of its adopters (when compiled with default settings). So I wonder if subnormal behaviour could just be specified as out-of-scope. What would be the ramifications?

Notably in #131 and numpy/numpy#18536 there was discussion of a smallest_subnormal property for finfo, but it was seen as a pretty awkward fit. It might be useful if some kind of information could tell the user that subnormals are not supported, e.g. smallest_subnormal == smallest_normal.
@honno honno mentioned this issue Nov 17, 2021
Closed
@rgommers rgommers added this to the v2021 milestone Nov 18, 2021
@rgommers rgommers added the Narrative Content Narrative documentation content. label Nov 18, 2021
@rgommers
Copy link
Member

+1 for making behavior on subnormals implementation-defined.

Not sure we should add any feature to determine this behavior. It may indeed make sense to add to finfo, but then I'd rather have something more direct than smallest_subnormal == smallest_normal.

Are there more options than "handle subnormals correctly" or "flush them to zero"?

@leofang
Copy link
Contributor

leofang commented Nov 18, 2021

Are there more options than "handle subnormals correctly" or "flush them to zero"?

Quick drive-by comment: On NVIDIA GPUs, no. See this section: https://docs.nvidia.com/cuda/floating-point/index.html#compiler-flags

@leofang
Copy link
Contributor

leofang commented Nov 18, 2021

I support leaving the behavior on subnormals implementation-defined too. Perhaps, in finfo we could add a new field "is_subnormal_zero"?

Are there more options than "handle subnormals correctly" or "flush them to zero"?

For HIP/OpenCL the answer is also no, at least for LLVM-based compilers: https://clang.llvm.org/docs/ClangCommandLineReference.html

@leofang
Copy link
Contributor

leofang commented Nov 18, 2021

@oleksandr-pavlyk What's the subnormal behavior of oneAPI on Intel CPUs/GPUs?

@kgryte kgryte mentioned this issue Dec 6, 2021
Merged
@kgryte kgryte mentioned this issue Mar 21, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Narrative Content Narrative documentation content.
Projects
None yet
Milestone
v2021
Development

Successfully merging a pull request may close this issue.

Add note concerning subnormal numbers data-apis/array-api
3 participants
@rgommers @leofang @honno