gh-116021: Deprecate support for instantiating abstract AST nodes #137865

brianschubert · 2025-08-16T17:39:34Z

Demonstration:

>>> ast.stmt()
<python-input-1>:1: DeprecationWarning: Instantiating abstract AST node class ast.stmt is deprecated. This will become an error in Python 3.20
stmt()

>>> class MyStmt(ast.stmt): pass
... 
>>> MyStmt()
MyStmt()

This tracks which AST nodes are abstract using an internal set that gets built during node type initialization. I tried a few different approaches, and this seemed the simplest. A nice thing about this approach is that the check for abstract-ness depends on the exact node type, so this avoids most inheritance subtleties that could interfere with existing user-defined subclasses.

Issue: Deprecate creation of abstract AST nodes #116021

📚 Documentation preview 📚: https://cpython-previews--137865.org.readthedocs.build/

brianschubert · 2025-08-16T18:08:55Z

Lib/test/test_sys.py

-        check(_ast.AST(), size('P'))
+        with self.assertWarns(DeprecationWarning):
+            check(_ast.AST(), size('P'))


I think this should probably be rewritten, but I'm not sure I understand the goal of this test well enough to do that. Suggestions welcome!

It's to check that objects in Python/ have the correct types. I don't know why it's actually in test_sys though. I think we should split this test and move each check to the module it belongs (ast.AST's size should be checked in test_ast).

For now, leave it here. A separate issue should be opened.

picnixz

Would it be easier to have a module function checking if a node is abstract? say _ast.is_abstract(node) (in the C module, not the Python module). That way we don't need an additional method on AST objects (which could be shadowed).

picnixz · 2025-08-18T12:59:27Z

Lib/test/test_sys.py

-        check(_ast.AST(), size('P'))
+        with self.assertWarns(DeprecationWarning):
+            check(_ast.AST(), size('P'))


It's to check that objects in Python/ have the correct types. I don't know why it's actually in test_sys though. I think we should split this test and move each check to the module it belongs (ast.AST's size should be checked in test_ast).

For now, leave it here. A separate issue should be opened.

picnixz · 2025-08-18T13:00:26Z

Parser/asdl_c.py

+        if (PyErr_WarnFormat(
+            PyExc_DeprecationWarning, 1,
+            "Instantiating abstract AST node class %T is deprecated. "
+            "This will become an error in Python 3.20",
+            self
+        ) < 0) {
+            return -1;
+        }


Suggested change

if (PyErr_WarnFormat(

PyExc_DeprecationWarning, 1,

"Instantiating abstract AST node class %T is deprecated. "

"This will become an error in Python 3.20",

self

) < 0) {

return -1;

}

if (PyErr_WarnFormat(

PyExc_DeprecationWarning, 1,

"Instantiating abstract AST node class %T is deprecated. "

"This will become an error in Python 3.20", self) < 0)

{

return -1;

}

for PEP-8

picnixz · 2025-08-18T13:04:51Z

Parser/asdl_c.py

@@ -1443,6 +1458,23 @@ def visitModule(self, mod):
    return result;
 }

+/* Helper for checking if a node class is abstract in the tests. */


I think we should use clinic here, but this would create a new file as we don't use it at all. Maybe we can do it in a follow-up PR though.

picnixz · 2025-08-18T13:08:25Z

Parser/asdl_c.py

@@ -1454,6 +1486,7 @@ def visitModule(self, mod):
     PyDoc_STR("__replace__($self, /, **fields)\\n--\\n\\n"
               "Return a copy of the AST node with new values "
               "for the specified fields.")},
+    {"_is_abstract", _PyCFunction_CAST(ast_is_abstract), METH_CLASS | METH_NOARGS, NULL},


Change the signature of the function from PyObject (*)(PyObject *, void *) to PyObject (*)(PyObject *, PyObject *) and remove the cast. The second argument is always NULL so you can name it dummy.

picnixz · 2025-08-18T13:15:20Z

Lib/test/test_ast/test_ast.py

@@ -109,15 +111,15 @@ def cleanup():

        msg = "type object 'ast.AST' has no attribute '_fields'"
        # Both examples used to crash:
-        with self.assertRaisesRegex(AttributeError, msg):
+        with self.assertRaisesRegex(AttributeError, msg), self.assertWarns(DeprecationWarning):


Suggested change

with self.assertRaisesRegex(AttributeError, msg), self.assertWarns(DeprecationWarning):

with (

self.assertRaisesRegex(AttributeError, msg),

self.assertWarns(DeprecationWarning),

):

and ditto below, just to warp at 80 chars.

picnixz · 2025-08-18T13:17:34Z

Lib/test/test_ast/test_ast.py

+                if typ._is_abstract():
+                    # Use an arbitrary concrete subclass
+                    concrete = next(sub for sub in typ.__subclasses__() if not sub._is_abstract())
+                    kwargs[name] = self._construct_ast_class(concrete)
+                else:
+                    kwargs[name] = self._construct_ast_class(typ)


Maybe ensure that "concrete" exists? also you could do concrete = typ and remove the else: and only call self._construct_ast_class once.

picnixz · 2025-08-18T13:18:21Z

Lib/test/test_ast/test_ast.py

@@ -573,14 +580,20 @@ def test_nodeclasses(self):
            x = ast.BinOp(1, 2, 3, foobarbaz=42)
        self.assertEqual(x.foobarbaz, 42)

+        # Directly instantiating abstract node types is allowed (but deprecated)
+        with self.assertWarns(DeprecationWarning):


Unless we need a variable, and unless we need to fit on 80 chars, I would suggest using self.assertWarns(DeprecationWarning, ast.stmt). In 3.20, we can just do self.assertRaises(..., ast.stmt). This reduces the overall file length without necessarily reducing readability IMO.

picnixz · 2025-08-18T13:19:53Z

Lib/test/test_ast/test_ast.py

-        m = ast.Module([ast.Expr(ast.expr(**pos), **pos)], [])
+        with self.assertWarns(DeprecationWarning):
+            # Creating instances of ast.expr is deprecated
+            m = ast.Module([ast.Expr(ast.expr(**pos), **pos)], [])


Can we just wrap ast.expr() as being the deprecated one? just to be sure we're not catching another deprecation warning

picnixz · 2025-08-18T13:20:39Z

Lib/test/test_ast/test_ast.py

@@ -1140,6 +1153,9 @@ def do(cls):
                return
            if cls is ast.Index:
                return
+            # Don't attempt to create instances of abstract AST nodes


You can also update the docstring saying that we also skip abstract classes.

Deprecate support for instantiating abstract AST nodes

1b767fc

brianschubert requested review from AA-Turner, Eclips4, tomasr8, isidentical, JelleZijlstra and ericsnowcurrently as code owners August 16, 2025 17:39

bedevere-app bot added the awaiting review label Aug 16, 2025

bedevere-app bot mentioned this pull request Aug 16, 2025

Deprecate creation of abstract AST nodes #116021

Open

brianschubert added 2 commits August 16, 2025 13:52

Missing static

b72a362

Capture warning in test_sys

c8dda97

brianschubert commented Aug 16, 2025

View reviewed changes

picnixz reviewed Aug 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-116021: Deprecate support for instantiating abstract AST nodes #137865

gh-116021: Deprecate support for instantiating abstract AST nodes #137865

brianschubert commented Aug 16, 2025 •

edited

Loading

Uh oh!

brianschubert Aug 16, 2025 •

edited

Loading

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz left a comment

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

picnixz Aug 18, 2025

Uh oh!

Uh oh!

Uh oh!

gh-116021: Deprecate support for instantiating abstract AST nodes #137865

Are you sure you want to change the base?

gh-116021: Deprecate support for instantiating abstract AST nodes #137865

Conversation

brianschubert commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brianschubert Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brianschubert commented Aug 16, 2025 •

edited

Loading

brianschubert Aug 16, 2025 •

edited

Loading