Skip to content

How to run stubtest on ujson (all compiled code)? #19261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
MarcoGorelli opened this issue Jun 9, 2025 · 1 comment
Open

How to run stubtest on ujson (all compiled code)? #19261

MarcoGorelli opened this issue Jun 9, 2025 · 1 comment
Labels
question topic-stubtest

Comments

@MarcoGorelli
Copy link
Contributor

I'm trying to run stubtest on ujson stubs, but am running into issues

Here are the exact steps I've followed:

~$ git clone git@github.com:ultrajson/ultrajson.git ujson-dev
Cloning into 'ujson-dev'...
remote: Enumerating objects: 4386, done.
remote: Counting objects: 100% (252/252), done.
remote: Compressing objects: 100% (120/120), done.
remote: Total 4386 (delta 208), reused 137 (delta 132), pack-reused 4134 (from 2)
Receiving objects: 100% (4386/4386), 8.48 MiB | 12.28 MiB/s, done.
Resolving deltas: 100% (2791/2791), done.
~$ cd ujson-dev/
~/ujson-dev$ python3.12 -m venv .venv
~/ujson-dev$ . .venv/bin/activate
(.venv) ~/ujson-dev$ pip install -U pip
Requirement already satisfied: pip in ./.venv/lib/python3.12/site-packages (24.3.1)
Collecting pip
  Using cached pip-25.1.1-py3-none-any.whl.metadata (3.6 kB)
Using cached pip-25.1.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.3.1
    Uninstalling pip-24.3.1:
      Successfully uninstalled pip-24.3.1
Successfully installed pip-25.1.1
(.venv) ~/ujson-dev$ pip install -e .
Obtaining file:///home/marcogorelli/ujson-dev
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: ujson
  Building editable for ujson (pyproject.toml) ... done
  Created wheel for ujson: filename=ujson-5.10.1.dev52-0.editable-cp312-cp312-linux_x86_64.whl size=8080 sha256=2f9fdc740c1389f41a8894444567d32af7bdd898000038e0df9864498452b70b
  Stored in directory: /tmp/pip-ephem-wheel-cache-w_65nxa1/wheels/90/a7/bf/92e454c2116377fe91ef96e498917290150aa7c3d025d25ac4
Successfully built ujson
Installing collected packages: ujson
Successfully installed ujson-5.10.1.dev52

I then added ujson.pyi with the following contents (note: the signature are all intentionally completely wrong):

from collections.abc import Callable
from typing import Any, Final, Protocol


__version__: Final[str]

def encode(
    foo: int
) -> str: ...
def dumps(
    bar: str
) -> str: ...
def dump(
    foo: str
) -> None: ...
def decode(bar: str) -> Any: ...
def loads(foo: int) -> Any: ...
def load(bar: float) -> Any: ...

class JSONDecodeError(ValueError): ...

And then ran

(.venv) ~/ujson-dev$ pip install mypy
Collecting mypy
  Downloading mypy-1.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl.metadata (2.1 kB)
Collecting typing_extensions>=4.6.0 (from mypy)
  Downloading typing_extensions-4.14.0-py3-none-any.whl.metadata (3.0 kB)
Collecting mypy_extensions>=1.0.0 (from mypy)
  Using cached mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pathspec>=0.9.0 (from mypy)
  Using cached pathspec-0.12.1-py3-none-any.whl.metadata (21 kB)
Downloading mypy-1.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (12.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.7/12.7 MB 30.9 MB/s eta 0:00:00
Using cached mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB)
Using cached pathspec-0.12.1-py3-none-any.whl (31 kB)
Downloading typing_extensions-4.14.0-py3-none-any.whl (43 kB)
Installing collected packages: typing_extensions, pathspec, mypy_extensions, mypy
Successfully installed mypy-1.16.0 mypy_extensions-1.1.0 pathspec-0.12.1 typing_extensions-4.14.0
(.venv) ~/ujson-dev$ python -m mypy.stubtest ujson
Success: no issues found in 1 module

So, despite having added completely wrong signatures, mypy.stubtest isn't reporting any issues.

Is there anything I should do differently in order to get stubtest to report issues with the wrong signatures I added?
@brianschubert
Copy link
Collaborator

So, despite having added completely wrong signatures, mypy.stubtest isn't reporting any issues.

The issue you're running into is that ujson has no runtime signature information that stubtest can compare against:

>>> import inspect, ujson
>>> inspect.signature(ujson.encode)
Traceback (most recent call last):
  <snip>
ValueError: no signature found for builtin <built-in function encode>

AFAIK there's currently no way for third-party1 C extension modules to supply runtime signature information. You should still get errors for omitted / extraneous symbols with extension modules, though.

There's been some discussion about standardizing this in the past (e.g. https://discuss.python.org/t/type-signatures-for-extension-modules-pep-draft/43914). Though I'm not sure what the current state of those proposals are.

Footnotes

  1. the standard library uses __text_signature__ for this purpose, but it's an undocumented implementation detail ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question topic-stubtest
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brianschubert @MarcoGorelli