1822416 - ./mach lint fails on Python 3.11 with typed_ast compilation errors

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, defect)

Description

[az [:az]]

Attempting to run ./mach lint on a Debian install running Python 3.11 fails with the following error message:

creating build/temp.linux-x86_64-3.11/ast27/Python
   x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iast27/Include -I/home/az/git/mozilla-unified/obj-x86_64-pc-linux-gnu/_virtualenvs/lint/include -I/usr/include/python3.11 -c ast27/Custom/typed_ast.c -o build/temp.linux-x86_64-3.11/ast27/Custom/typed_ast.o
    In file included from ast27/Custom/typed_ast.c:3:
    ast27/Custom/../Include/compile.h:5:10: fatal error: code.h: No such file or directory
        5 | #include "code.h"
          |          ^~~~~~~~
    compilation terminated.
    error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
    ----------------------------------------

The reason for this failure is that the code.h header file was removed from Python, so typed_ast is no longer able to compile. Newer versions of ast no longer include the removed header file and ./mach lint works again.

I'm not familiar with typed_ast and there may be other ramifications to this upgrade. I have a patch I'll attach shortly for review by those more knowledgeable in this area.

Comment 1

[az [:az]]

Attachment: Bug 1822416 - Upgrade typed_ast package to 1.5.4 to fix compilation errors running ./mach lint with Python 3.11

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:andi, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 3

[Adam Brouwers-Harries [:aabh] [he/him] ⌚GMT]

Note that this also affects MacOS, specifically when installing python using homebrew. This is due to python 3.11 being a (homebrew) dependency for mercurial, which then overrides the system default python.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5b8f515dab24
Upgrade typed_ast package to 1.5.4 to fix compilation errors running ./mach lint with Python 3.11 r=sylvestre

Comment 5

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/5b8f515dab24