Skip to content

The Python implementation of Decimal does not support the "N" format #89902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
serhiy-storchaka opened this issue Nov 6, 2021 · 11 comments
Closed
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Nov 6, 2021

BPO 45739
Nosy @rhettinger, @facundobatista, @mdickinson, @ericvsmith, @serhiy-storchaka

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-11-06.13:51:18.396>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'The Python implementation of Decimal does not support the "N" format'
updated_at = <Date 2021-11-29.09:28:44.922>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-11-29.09:28:44.922>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-11-06.13:51:18.396>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 45739
keywords = []
message_count = 10.0
messages = ['405861', '405898', '405901', '407195', '407196', '407210', '407212', '407229', '407234', '407254']
nosy_count = 5.0
nosy_names = ['rhettinger', 'facundobatista', 'mark.dickinson', 'eric.smith', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45739'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

@serhiy-storchaka
Copy link
Member Author

The C implementation supports both formats "n" and "N". The Python implementation only supports format "n".

>>> from decimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
'1E+100'
>>> from _pydecimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 3766, in __format__
    spec = _parse_format_specifier(specifier, _localeconv=_localeconv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 6194, in _parse_format_specifier
    raise ValueError("Invalid format specifier: " + format_spec)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Invalid format specifier: N

@serhiy-storchaka serhiy-storchaka added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Nov 6, 2021
@mdickinson
Copy link
Member

Interesting. I think the behaviour of the Python implementation behaviour is actually more correct here: neither int nor float supports 'N', and I'm not seeing any indication in tests or documentation that 'N' should be supported. So is this a bug in libmpdec, or a missing feature in the Python implementation? (Either way, it's definitely a bug that the two aren't aligned.)

>>> format(123, 'n')
'123'
>>> format(123, 'N')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'N' for object of type 'int'

@ericvsmith
Copy link
Member

I don’t think “N” should be supported. It was never the intention to have it work.

@mdickinson
Copy link
Member

Eric, Serhiy: do you have opinions on the right way forward?

Here are 6 options, on a spectrum of increasing level of acceptance of "N".

-2. Remove "N" support for cdecimal right now (i.e., for Python 3.11), on the basis that there's no need for deprecation warnings, because it was never officially a feature.
-1. Deprecate "N" support for cdecimal, remove it in Python 3.13.
0. Do nothing (the default), leaving _pydecimal and cdecimal inconsistent.

  1. Add "N" support to the Python implementation for parity with cdecimal, but don't document it - leave it as an undocumented feature.
  2. Officially add "N" support to decimal formatting - add documentation, tests, and fix the Python implementation.
  3. Officially add "N" support to all numeric formatting ...

@mdickinson
Copy link
Member

I could be persuaded for any of options -1, 1 and 2. I don't much like option 0.

@rhettinger
Copy link
Contributor

I support deprecating "N".

@ericvsmith
Copy link
Member

I'd support -1.

@serhiy-storchaka
Copy link
Member Author

I have no strong opinion. I found the discrepancy when reviewed one of Mark's PRs. I was surprised because I did not know that "N" is supported.

On other hand, there are reasons for supporting upper-case variant of "n" if we support upper-case variants of "e", "g" and "x" (formats which produce output containing letters).

There is an alternative solution of the problem solved by upper-case variations of formats: add a special converter for converting a string to upper case, so for example f'{x:12.8E}' could be written as f'{x:12.8e!u}'.

I prefer -1 or 2. If we choose deprecation I prefer adding an upper-case converter.

@ericvsmith
Copy link
Member

While I'd prefer -1 overall, I would also prefer 3 over 2. If we were adding the feature from scratch, we wouldn't have decimal differ from int, float, and complex.

And I'm not in favor of an uppercase converter, no matter what we do here. The other converters work like:

f'{obj!s}' -> format(str(obj))
f'{obj!r}' -> format(repr(obj))

but the proposed !u would be

f'{obj!u}' -> format(obj).upper()

That is, it operates on the result of __format__, not its input.

@serhiy-storchaka
Copy link
Member Author

You are right about an uppercase converter. I forget this. I withdraw this proposition.

It seems that even if we add support of "N" to all numeric formatting it will not cover all use cases. "n" is a locale specific variant of "g", but other formats could have locale specific variants too. In C all numeric formatting is locale specific. Maybe add a flag which makes any numeric formatting locale specific? Then "n" will became obsolete.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Oct 7, 2023
It was not documented and only supported in the C implementation.
serhiy-storchaka added a commit that referenced this issue Oct 8, 2023
…-110508)

It was not documented and only supported in the C implementation.
@serhiy-storchaka
Copy link
Member Author

We missed terms for 3.11 and 3.12, but finally it has been deprecated in 3.13. It can be removed in 3.15 or like.

Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
…al (pythonGH-110508)

It was not documented and only supported in the C implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants