Skip to content

JIT build crashes on x86_64-linux with LLVM 18 #130673

Closed
@rennsax

Description

@rennsax

Bug report

Bug description:

I'm trying to build CPython 3.13.2 with JIT supported (--enable-experimental-jit=yes-off), but the build process crashes after reporting like this:

python3>     | Traceback (most recent call last):
python3>     |   File "/build/Python-3.13.2/Tools/jit/_targets.py", line 181, in _compile
python3>     |     return await self._parse(o)
python3>     |            ^^^^^^^^^^^^^^^^^^^^
python3>     |   File "/build/Python-3.13.2/Tools/jit/_targets.py", line 89, in _parse
python3>     |     self._handle_section(wrapped_section["Section"], group)
python3>     |   File "/build/Python-3.13.2/Tools/jit/_targets.py", line 330, in _handle_section
python3>     |     value, base = group.symbols[section["Info"]]
python3>     |                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
python3>     | KeyError: 5

I've tracked the build process and found that the _parse routine in Tools/_targets.py almost fails for all the object files produced by the _compile process. When handling a ELF section of type SHT_PROGBITS, if SHF_ALLOC is not included in its flags, then the symbol tables of the stencil group will not be updated. Then if a later section refers to the symbol, a KeyError occurs. For example, an object file (_NOP.o) like this:

[
  {
    "Section": {
      "Index": 5,
      "Name": { "Name": ".debug_info", "Value": 108 },
      "Type": { "Name": "SHT_PROGBITS", "Value": 1 },
      "Flags": {
        "Value": 2048,
        "Flags": [{ "Name": "SHF_COMPRESSED", "Value": 2048 }]
      },
      "Address": 0,
      "Offset": 463,
      "Size": 29486,
      "Link": 0,
      "Info": 0
      // ...
  },
  {
    "Section": {
      "Index": 6,
      "Name": { "Name": ".rela.debug_info", "Value": 103 },
      "Type": { "Name": "SHT_RELA", "Value": 4 },
      "Flags": {
        "Value": 64,
        "Flags": [{ "Name": "SHF_INFO_LINK", "Value": 64 }]
      },
      "Address": 0,
      "Offset": 44968,
      "Size": 96,
      "Link": 20,
      "Info": 5
      // ...
  }

When handling the 5th section, L349 is not executed:

elif section_type == "SHT_PROGBITS":
if "SHF_ALLOC" not in flags:
return
if "SHF_EXECINSTR" in flags:
value = _stencils.HoleValue.CODE
stencil = group.code
else:
value = _stencils.HoleValue.DATA
stencil = group.data
group.symbols[section["Index"]] = value, len(stencil.body)
for wrapped_symbol in section["Symbols"]:
symbol = wrapped_symbol["Symbol"]
offset = len(stencil.body) + symbol["Value"]
name = symbol["Name"]["Name"]
name = name.removeprefix(self.prefix)
group.symbols[name] = value, offset
stencil.body.extend(section["SectionData"]["Bytes"])
assert not section["Relocations"]

Then when handling the 6th section, L330 will try to index group.symbols[5]:

if section_type == "SHT_RELA":
assert "SHF_INFO_LINK" in flags, flags
assert not section["Symbols"]
value, base = group.symbols[section["Info"]]
if value is _stencils.HoleValue.CODE:
stencil = group.code
else:
assert value is _stencils.HoleValue.DATA
stencil = group.data
for wrapped_relocation in section["Relocations"]:
relocation = wrapped_relocation["Relocation"]
hole = self._handle_relocation(base, relocation, stencil.body)
stencil.holes.append(hole)

where the error occurs.

I'm not sure whether it's because of the version of LLVM (18.1.8) I'm using.

CPython versions tested on:

3.13.2

Operating systems tested on:

GNU/Linux

Build Toolchains

  • Python 3.12.4
  • LLVM 18.1.8

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildtopic-JITtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions