gh-127604: Add C stack dumps to `faulthandler` #128159

ZeroIntensity · 2024-12-21T18:30:33Z

This only adds it for glibc right now. I think it's possible to implement it for Windows with StackWalk64, but I'd prefer to get a solid working implementation for other systems before I dive into that.

Issue: Add a way of printing a C backtrace to faulthandler #127604

📚 Documentation preview 📚: https://cpython-previews--128159.org.readthedocs.build/en/128159/library/faulthandler.html#dumping-the-c-stack

picnixz

There are some PEP-7 nits but I don't know if this was done to be aligned with the rest of the code so you can freely ignore those nits.

Doc/library/faulthandler.rst

Doc/whatsnew/3.14.rst

Doc/library/faulthandler.rst

Modules/faulthandler.c

Doc/whatsnew/3.14.rst

Modules/faulthandler.c

picnixz · 2024-12-21T18:59:51Z

@ZeroIntensity If you ever use the web UI, don't forget to replace my tabs with spaces... hopefully your IDE does it but I see that I have tabs instead of spaces on my suggestions (I've corrected them but just double-check)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

…thon into c-faulthandler

Python/traceback.c

Doc/whatsnew/3.14.rst

Modules/faulthandler.c

configure.ac

pablogsal · 2025-04-14T12:15:04Z

Modules/faulthandler.c

+
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs,
+        "|O:dump_c_stack", kwlist,
+        &file))


PEP 7: add braces

I think this got brought up before. The rest of the file avoids the braces, it's probably better to just stay consistent.

I respectfully disagree, we are adding new functions so it's better to go closer to the standard that deviate more from it

As the author of the original code, I would prefer that new code respects PEP 7 (add braces) :-)

Will do. cc @picnixz if you want to bicker about styling (I'm pretty sure you were the one that originally brought this up).

vstinner · 2025-04-14T14:54:53Z

I tested with:

import faulthandler
faulthandler.dump_c_stack()

gdb is able to retrieve the function name of all frames, whereas faulthandler.dump_c_stack() only displays a few function names.

I built Python on Fedora 41 with ./configure --with-pydebug: using gcc -g -Og.

gdb stack output:

(gdb) where -frame-arguments none
#0  faulthandler_dump_c_stack_py (self=..., args=..., kwargs=...) at ./Modules/faulthandler.c:285
#1  0x00000000004da76b in cfunction_call (func=..., args=..., kwargs=...) at Objects/methodobject.c:569
#2  0x000000000048b43a in _PyObject_MakeTpCall (tstate=..., callable=..., args=..., nargs=..., keywords=...) at Objects/call.c:242
#3  0x000000000048b640 in _PyObject_VectorcallTstate (tstate=..., callable=..., args=..., nargsf=..., kwnames=...)
    at ./Include/internal/pycore_call.h:167
#4  0x000000000048b68e in PyObject_Vectorcall (callable=..., args=..., nargsf=..., kwnames=...) at Objects/call.c:327
#5  0x00000000005a9ec2 in _PyEval_EvalFrameDefault (tstate=..., frame=..., throwflag=...) at Python/generated_cases.c.h:1443
#6  0x00000000005c50e9 in _PyEval_EvalFrame (tstate=..., frame=..., throwflag=...) at ./Include/internal/pycore_ceval.h:119
#7  0x00000000005c5287 in _PyEval_Vector (tstate=..., func=..., locals=..., args=..., argcount=..., kwnames=...) at Python/ceval.c:1913
#8  0x00000000005c535a in PyEval_EvalCode (co=..., globals=..., locals=...) at Python/ceval.c:829
#9  0x00000000006321dc in run_eval_code_obj (tstate=..., co=..., globals=..., locals=...) at Python/pythonrun.c:1365
#10 0x0000000000632372 in run_mod (mod=..., filename=..., globals=..., locals=..., flags=..., arena=..., interactive_src=..., 
    generate_new_source=...) at Python/pythonrun.c:1436
#11 0x0000000000632b2e in pyrun_file (fp=..., filename=..., start=..., globals=..., locals=..., closeit=..., flags=...)
    at Python/pythonrun.c:1293
#12 0x0000000000634177 in _PyRun_SimpleFileObject (fp=..., filename=..., closeit=..., flags=...) at Python/pythonrun.c:521
#13 0x0000000000634367 in _PyRun_AnyFileObject (fp=..., filename=..., closeit=..., flags=...) at Python/pythonrun.c:81
#14 0x000000000065a092 in pymain_run_file_obj (program_name=..., filename=..., skip_source_first_line=...) at Modules/main.c:396
#15 0x000000000065a19e in pymain_run_file (config=...) at Modules/main.c:415
#16 0x000000000065ab5a in pymain_run_python (exitcode=...) at Modules/main.c:680
#17 0x000000000065ad92 in Py_RunMain () at Modules/main.c:761
#18 0x000000000065ade6 in pymain_main (args=...) at Modules/main.c:791
#19 0x000000000065ae66 in Py_BytesMain (argc=..., argv=...) at Modules/main.c:815
#20 0x000000000040292f in main (argc=..., argv=...) at ./Programs/python.c:15

faulthandler.dump_c_stack() output:

Current thread's C stack trace (most recent call first):
  Binary file "/home/vstinner/python/main/python", at _Py_DumpStack+0x31 [0x64b02f]
  Binary file "/home/vstinner/python/main/python" [0x65d775]
  Binary file "/home/vstinner/python/main/python" [0x4da76b]
  Binary file "/home/vstinner/python/main/python", at _PyObject_MakeTpCall+0xcc [0x48b43a]
  Binary file "/home/vstinner/python/main/python" [0x48b640]
  Binary file "/home/vstinner/python/main/python", at PyObject_Vectorcall+0x23 [0x48b68e]
  Binary file "/home/vstinner/python/main/python", at _PyEval_EvalFrameDefault+0x3657 [0x5a9ec2]
  Binary file "/home/vstinner/python/main/python" [0x5c50e9]
  Binary file "/home/vstinner/python/main/python" [0x5c5287]
  Binary file "/home/vstinner/python/main/python", at PyEval_EvalCode+0xb4 [0x5c535a]
  Binary file "/home/vstinner/python/main/python" [0x6321dc]
  Binary file "/home/vstinner/python/main/python" [0x632372]
  Binary file "/home/vstinner/python/main/python" [0x632b2e]
  Binary file "/home/vstinner/python/main/python" [0x634177]
  Binary file "/home/vstinner/python/main/python" [0x634367]
  Binary file "/home/vstinner/python/main/python" [0x65a092]
  Binary file "/home/vstinner/python/main/python" [0x65a19e]
  Binary file "/home/vstinner/python/main/python" [0x65ab5a]
  Binary file "/home/vstinner/python/main/python", at Py_RunMain+0x22 [0x65ad92]
  Binary file "/home/vstinner/python/main/python" [0x65ade6]
  Binary file "/home/vstinner/python/main/python", at Py_BytesMain+0x2b [0x65ae66]
  Binary file "/home/vstinner/python/main/python" [0x40292f]
  Binary file "/lib64/libc.so.6", at +0x3248 [0x7ffff7ccb248]
  Binary file "/lib64/libc.so.6", at __libc_start_main+0x8b [0x7ffff7ccb30b]
  Binary file "/home/vstinner/python/main/python", at _start+0x25 [0x402865]

ZeroIntensity · 2025-04-14T16:43:31Z

I'm pretty sure that's because GDB is using extra debugging information that we can't access from a signal handler.

pablogsal · 2025-04-14T16:54:07Z

I'm pretty sure that's because GDB is using extra debugging information that we can't access from a signal handler.

Yes but the functions you call should be using the GNU libbacktrace code and that can parse DWARF 4 so not sure what's going on. I guess the only thing to do here is to check if your custom functions are not missing anything here by comparing the output with the vanilla ones

ZeroIntensity · 2025-04-14T17:01:43Z

Maybe it's some sort of compiler flag? If you checkout 533b1db (before I started the homebrewed version), you see similar results in the output.

pablogsal · 2025-04-14T18:35:39Z

Maybe it's some sort of compiler flag? If you checkout 533b1db (before I started the homebrewed version), you see similar results in the output.

Can you check if compiling with -g3 makes any difference?

vstinner · 2025-04-15T09:42:41Z

Replacing -g with -g3 doesn't change anything to faulthandler.dump_c_stack() output in my #128159 (comment) test.

ZeroIntensity · 2025-04-19T16:52:38Z

I'm pretty sure this is as good as we're going to get.

gpshead

as much as I believe there can be better ways to do this, this is a great start to the feature. we can improve upon the capabilities in the future.

pablogsal

Agreed! LGTM

great job @ZeroIntensity 👌

vstinner · 2025-04-22T07:00:57Z

Congrats @ZeroIntensity, nice enhancement.

ZeroIntensity added 9 commits December 21, 2024 12:27

Initial implementation for glibc

0ebdc69

Make it cross-platform buildable.

0ccf9fb

Call it on Windows.

52945e5

Add a whatsnew entry

9bdd802

Add dump_c_stack

66f2641

Reverse the loop.

0591013

Add documentation entries.

64707a2

Make the whatsnew entry more specific.

8a5b784

Add tests.

4f09358

ZeroIntensity requested review from gpshead and picnixz December 21, 2024 18:30

ZeroIntensity requested review from erlend-aasland and corona10 as code owners December 21, 2024 18:30

bedevere-app bot mentioned this pull request Dec 21, 2024

Add a way of printing a C backtrace to faulthandler #127604

Open

bedevere-app bot added the awaiting review label Dec 21, 2024

ZeroIntensity added 7 commits December 21, 2024 13:32

Add blurb from whatsnew.

e1aa619

Fix failing Sphinx build.

6b515d3

Fix PyArg_ParseTupleAndKeywords format string.

c69ee9d

Fix Sphinx warnings.

f08b1dd

Add ignore to test_inspect

dbb6d25

Fix name.

faf1a3e

Ignore the reentrant variable in the C analyzer.

ec832aa

picnixz reviewed Dec 21, 2024

View reviewed changes

ZeroIntensity requested a review from ericsnowcurrently as a code owner December 21, 2024 18:58

ZeroIntensity and others added 5 commits December 21, 2024 14:03

Apply suggestions from code review

524f167

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Use manpage references.

dd08bcb

Add issue number to whatsnew.

bda3dcd

Merge branch 'c-faulthandler' of https://github.com/ZeroIntensity/cpy…

a3564a5

…thon into c-faulthandler

Rename macros.

f3fcea1

sergey-miryanov reviewed Apr 13, 2025

View reviewed changes

Python/traceback.c Outdated Show resolved Hide resolved

sergey-miryanov reviewed Apr 13, 2025

View reviewed changes

Python/traceback.c Outdated Show resolved Hide resolved

sergey-miryanov reviewed Apr 13, 2025

View reviewed changes

Python/traceback.c Outdated Show resolved Hide resolved

ZeroIntensity added 4 commits April 13, 2025 18:30

Don't use GNU extension.

ce9c39f

Handle NULL.

c344ad7

Fix newlines in the whatsnew.

e899792

Fix stray newline change.

b136f71

pablogsal reviewed Apr 14, 2025

View reviewed changes

Doc/whatsnew/3.14.rst Outdated Show resolved Hide resolved

pablogsal reviewed Apr 14, 2025

View reviewed changes

Modules/faulthandler.c Show resolved Hide resolved

pablogsal reviewed Apr 14, 2025

View reviewed changes

configure.ac Outdated Show resolved Hide resolved

pablogsal reviewed Apr 14, 2025

View reviewed changes

ZeroIntensity added 3 commits April 14, 2025 08:26

Add configure guard for dladdr1()

2c381b9

Add documentation about compatibility.

52c0748

Fix sphinx role.

bd47026

Fix styling.

07a20d0

gpshead approved these changes Apr 21, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Apr 21, 2025

pablogsal approved these changes Apr 21, 2025

View reviewed changes

pablogsal merged commit 8dfa840 into python:main Apr 21, 2025
46 checks passed

bedevere-app bot removed the awaiting merge label Apr 21, 2025

ZeroIntensity deleted the c-faulthandler branch April 21, 2025 19:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-127604: Add C stack dumps to `faulthandler` #128159

gh-127604: Add C stack dumps to `faulthandler` #128159

ZeroIntensity commented Dec 21, 2024 •

edited

Loading

picnixz left a comment

picnixz commented Dec 21, 2024 •

edited

Loading

pablogsal Apr 14, 2025

ZeroIntensity Apr 14, 2025

pablogsal Apr 14, 2025

vstinner Apr 14, 2025

ZeroIntensity Apr 14, 2025

vstinner commented Apr 14, 2025

ZeroIntensity commented Apr 14, 2025

pablogsal commented Apr 14, 2025

ZeroIntensity commented Apr 14, 2025

pablogsal commented Apr 14, 2025

vstinner commented Apr 15, 2025

ZeroIntensity commented Apr 19, 2025

gpshead left a comment

pablogsal left a comment

vstinner commented Apr 22, 2025

gh-127604: Add C stack dumps to faulthandler #128159

gh-127604: Add C stack dumps to faulthandler #128159

Conversation

ZeroIntensity commented Dec 21, 2024 • edited Loading

picnixz left a comment

Choose a reason for hiding this comment

picnixz commented Dec 21, 2024 • edited Loading

pablogsal Apr 14, 2025

Choose a reason for hiding this comment

ZeroIntensity Apr 14, 2025

Choose a reason for hiding this comment

pablogsal Apr 14, 2025

Choose a reason for hiding this comment

vstinner Apr 14, 2025

Choose a reason for hiding this comment

ZeroIntensity Apr 14, 2025

Choose a reason for hiding this comment

vstinner commented Apr 14, 2025

ZeroIntensity commented Apr 14, 2025

pablogsal commented Apr 14, 2025

ZeroIntensity commented Apr 14, 2025

pablogsal commented Apr 14, 2025

vstinner commented Apr 15, 2025

ZeroIntensity commented Apr 19, 2025

gpshead left a comment

Choose a reason for hiding this comment

pablogsal left a comment

Choose a reason for hiding this comment

vstinner commented Apr 22, 2025

gh-127604: Add C stack dumps to `faulthandler` #128159

gh-127604: Add C stack dumps to `faulthandler` #128159

ZeroIntensity commented Dec 21, 2024 •

edited

Loading

picnixz commented Dec 21, 2024 •

edited

Loading