1332758 - PyAsn1 error prevents building mozilla-central (Component value is tag-incompatible: OctetString(hexValue='30030101ff') vs Any())

Status: NEW

(Firefox Build System :: General, defect)

Description

[Josh Matthews [:jdm]]

Attachment: Build error for collection_signing_root.pem

When I build mozilla-central, I now receive this error on macOS X 10.9.5 using Python 2.7.11.

Comment 1

[Josh Matthews [:jdm]]

Attachment: Workaround to force use of in-tree PyAsn1

When I discovered that we vendor PyAsn1, I suspected that my site-packages version was only being used by accident. The build completed when I used this hacky workaround to force the imports to resolve to the in-tree modules.

Comment 2

[Gregory Szorc [:gps]]

build/test_templates.mozbuild is where the build config defines how to execute pycert.py.

The symptoms make it seem like the underlying problem is that pycert.py is being executed as a standalone executable, which means the shebang will be used to find the `python` binary. This can be fixed by using the Python interpreter installed in the build system's virtualenv.

However, my reading of https://hg.mozilla.org/mozilla-central/file/5a4412474c63/python/mozbuild/mozbuild/backend/recursivemake.py#l506 and https://hg.mozilla.org/mozilla-central/file/5a4412474c63/python/mozbuild/mozbuild/action/file_generate.py say this should "just work" since all generated files use Python and are executed with the build system's virtualenv. So I'm scratching my head.

Maybe try changing build/test_templates.mozbuild's "script" attribute to the "filename:method" form. e.g. '/security/manager/ssl/tests/unit/pycert.py:__main__'. This /should/ be equivalent. But it's worth a try.

Comment 3

[Josh Matthews [:jdm]]

 0:44.31 Error: script "/Users/jdm/src/mozilla-central/security/manager/ssl/tests/unit/pycert.py" is missing a __main__ method

Comment 4

[Gregory Szorc [:gps]]

Oh, right. Try adding a named function to pycert.py that does what the 'if __name__ == "__main__"' check is doing and use that function name after the colon.

Comment 5

[Josh Matthews [:jdm]]

It appears to hang indefinitely waiting on input from stdin?

Comment 6

[Gregory Szorc [:gps]]

¯\_(ツ)_/¯

Is this a high priority or can you work around it?

Comment 7

[Josh Matthews [:jdm]]

I have a patch in my mq that I apply whenever I need to build right now.

Comment 8

[Philipp Kewisch [:Fallen][☁️📆]]

For the record, I had this too and the reason was that my shell has a PYTHONPATH set that included /usr/local/lib/python2.7/site-packages/. Using unset PYTHONPATH fixed it.

pycert.py conveniently has a main method already that has the correct format for file_generate.py and therefore doesn't hang waiting for stdin, but without removing the PYTHONPATH it of course still preferred PyAsn1 from /usr/local.

Maybe it makes sense to warn/error if the virtualenv python has a PYTHONPATH set, or make sure the virtualenv site-packages is first in order?