Skip to content

Add support for IEEE 754 contexts to decimal module. #53032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mdickinson opened this issue May 22, 2010 · 12 comments
Closed

Add support for IEEE 754 contexts to decimal module. #53032

mdickinson opened this issue May 22, 2010 · 12 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@mdickinson
Copy link
Member

mdickinson commented May 22, 2010

BPO 8786
Nosy @rhettinger, @terryjreedy, @facundobatista, @mdickinson, @tiran, @ezio-melotti, @skrah, @mmaker
Files
  • issue8786.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2010-05-22.10:28:31.524>
    labels = ['type-feature', 'library']
    title = 'Add support for IEEE 754 contexts to decimal module.'
    updated_at = <Date 2013-08-30.16:08:54.755>
    user = 'https://github.com/mdickinson'

    bugs.python.org fields:

    activity = <Date 2013-08-30.16:08:54.755>
    actor = 'skrah'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2010-05-22.10:28:31.524>
    creator = 'mark.dickinson'
    dependencies = []
    files = ['27214']
    hgrepos = []
    issue_num = 8786
    keywords = ['patch']
    message_count = 8.0
    messages = ['106294', '106296', '106297', '106303', '106325', '170637', '172903', '196559']
    nosy_count = 8.0
    nosy_names = ['rhettinger', 'terry.reedy', 'facundobatista', 'mark.dickinson', 'christian.heimes', 'ezio.melotti', 'skrah', 'maker']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'needs patch'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue8786'
    versions = ['Python 3.4']

    Linked PRs

    @mdickinson
    Copy link
    Member Author

    Discussion migrated from bpo-8540 into its own issue.

    For ease of communication with other libraries, it would be good to be able to easily create contexts corresponding to the IEEE 754 (2008) decimal interchange formats.

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented May 22, 2010

    Some context from bpo-8540:

    [Stefan Krah]

    I'm busy implementing the IEEE754 contexts for cdecimal. To keep things
    in sync, it would be nice to agree how they should be created.

    Suggestions:

    1. c = Decimal64Context

    2. c = Context(Decimal64)

    [Mark Dickinson]

    Rather that complicating the Context constructor, I'd prefer a separate factory
    function. I was thinking of something like:

    def IEEEContext(n):
    """Return the decimal<n> IEEE 754 context. n should be a multiple
    of 32."""
    ...

    Again, it's clear with this that you get a new context object (I agree that
    there are problems with (1) and the mutability of Contexts).

    I like that, too. Where do you find the "multiple of 32" in the standard?
    In a draft of IEEE 754, I only see:

    Table 2—Interchange format parameters defining floating-point numbers:

    storage format: decimal32

    basic format: decimal64 and decimal128

    This is what Java and decNumber offer.

    @mdickinson
    Copy link
    Member Author

    It's Table 3.6 ("Decimal interchange format parameters") in the final version of IEEE 754; I'm not sure what that corresponds to in the various drafts. It has column headings: "decimal32", "decimal64", "decimal128" and "decimal{k} (k >= 32)".

    Parameters for decimal{k}: k must be a multiple of 32. precision is 9*k/32-2. emax is 3*2**(k/16+3). I think these formulas all work for the specific cases k in {32, 64, 128} too, so it should be easy to check that they make sense.

    They give an example below the table, too:

    "For example, decimal256 would have p = 70 and emax = 1572864."

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented May 22, 2010

    Mark Dickinson <report@bugs.python.org> wrote:

    It's Table 3.6 ("Decimal interchange format parameters") in the final version of IEEE 754;

    Thanks! I think this is not in the draft I have.

    +1 for IEEEContext(n). Could we have module constants Decimal32, Decimal64 and
    Decimal128 so that people coming from Java or DecNumber can write:

    c = IEEEContext(Decimal64)

    It is somewhat redundant, but 99% of people will use only those.

    @mdickinson mdickinson added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels May 22, 2010
    @mdickinson
    Copy link
    Member Author

    Thinking ahead a bit: at some point we might well also want functions to pack and unpack these IEEE formats into byte sequences, using the bit representations described in the standard.

    A natural place for those functions would be as methods on a Context object; perhaps IEEEContext should be a subclass of Context? (OTOH, the struct module is another possible place for such functionality.)

    I'll think about this.

    @mmaker
    Copy link
    Mannequin

    mmaker mannequin commented Sep 18, 2012

    Something like this? That's a pretty trivial draft for the patch.

    About "byte sequences", those features should be available using builtins bin(), oct() and hex(), hacking on __index__, or with internal methods?

    I am lacking imagination, what else there should be to test?

    @terryjreedy
    Copy link
    Member

    I do not think IEEEContext should not be made a class rather than a function until it really *is* a subclass with altered or added behavior. This means at least its own __str__ method and in this case, a __setattr__ that enforces the invariants. Here that means refusing to modify the context, or at least not prec and Emax (assuming that changes to any other context attributes (are there any?) are harmless. Something like

      def __setattr__(self, key, value):
        if key in ('prec', 'Emax'):
          raise AttributeError("cannot change {} of IEEEContext".format(key))
        else:
          super(IEEEContext, self).__setattr__(key, name)

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Aug 30, 2013

    BTW, in _decimal the feature can already be enabled with:

    ./configure CFLAGS=-DEXTRA_FUNCTIONALITY

    >>> IEEEContext(DECIMAL128)
    Context(prec=34, rounding=ROUND_HALF_EVEN, Emin=-6143, Emax=6144, capitals=1, clamp=1, flags=[], traps=[])
    >>> IEEEContext(DECIMAL64)
    Context(prec=16, rounding=ROUND_HALF_EVEN, Emin=-383, Emax=384, capitals=1, clamp=1, flags=[], traps=[])
    >>> IEEEContext(DECIMAL32)
    Context(prec=7, rounding=ROUND_HALF_EVEN, Emin=-95, Emax=96, capitals=1, clamp=1, flags=[], traps=[])
    >>>
    
    
    >>> IEEEContext(512)
    Context(prec=142, rounding=ROUND_HALF_EVEN, Emin=-103079215103, Emax=103079215104, capitals=1, clamp=1, flags=[], traps=[])
    >>> IEEEContext(1024)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: argument must be a multiple of 32, with a maximum of 512
    >>>

    Of course this isn't the official API yet, but I think it's
    reasonable.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @skirpichev
    Copy link
    Member

    It seems, this doesn't work anymore:

    $ ./configure CFLAGS=-DEXTRA_FUNCTIONALITY -q && make -s
    configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)
    In function ‘word_to_string’,
        inlined from ‘coeff_to_string’ at ./Modules/_decimal/libmpdec/io.c:411:13,
        inlined from ‘_mpd_to_string’ at ./Modules/_decimal/libmpdec/io.c:612:18:
    ./Modules/_decimal/libmpdec/io.c:349:40: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
      349 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
          |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
    ./Modules/_decimal/libmpdec/io.c:360:14: note: in expansion of macro ‘EXTRACT_DIGIT’
      360 |     case 15: EXTRACT_DIGIT(s, x, 100000000000000ULL, dot);
          |              ^~~~~~~~~~~~~
    [... and similar warnings from issue #108562, hardly relevant]
    Checked 112 modules (34 built-in, 77 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)
    $ ./python 
    Python 3.14.0a0 (heads/main:cecaceea31, Jul 19 2024, 05:34:00) [GCC 12.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from decimal import *
    >>> IEEEContext(11)
    Traceback (most recent call last):
      File "<python-input-1>", line 1, in <module>
        IEEEContext(11)
        ~~~~~~~~~~~^^^^
    ValueError: argument must be a multiple of 32, with a maximum of 512
    >>> IEEEContext(1024)
    Traceback (most recent call last):
      File "<python-input-2>", line 1, in <module>
        IEEEContext(1024)
        ~~~~~~~~~~~^^^^^^
    ValueError: argument must be a multiple of 32, with a maximum of 512
    >>> IEEEContext(512)  # oops
    Segmentation fault
    

    I'm working on this.

    skirpichev added a commit to skirpichev/cpython that referenced this issue Jul 19, 2024
    This was in C version from beginning, but available only
    on conditional compilation (EXTRA_FUNCTIONALITY).  Current
    patch adds function to create IEEE contexts to the
    pure-python module as well.
    skirpichev added a commit to skirpichev/cpython that referenced this issue Jul 19, 2024
    This was in C version from beginning, but available only
    on conditional compilation (EXTRA_FUNCTIONALITY).  Current
    patch adds function to create IEEE contexts to the
    pure-python module as well.
    @skirpichev
    Copy link
    Member

    PR is ready for review: #122003

    vstinner pushed a commit that referenced this issue Apr 28, 2025
    This was in C version from beginning, but available only
    on conditional compilation (EXTRA_FUNCTIONALITY).  Current
    patch adds function to create IEEE contexts to the
    pure-python module as well.
    
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    @vstinner
    Copy link
    Member

    Implemented by the change 5bf0f36.

    @vstinner
    Copy link
    Member

    It only took 15 years to add this feature :-)

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants