1356860 - Mac builds broken as of 2017-04-15 - AttributeError: 'NoneType' object has no attribute 'startswith' - Python problem

Status: RESOLVED WORKSFORME

(Thunderbird :: Build Config, defect)

Description

[Jorg K (CEST = GMT+2)]

builder: comm-central-macosx64-debug
slave: bld-lion-r5-081
starttime: 1492294089.89
results: failure (2)
buildid: Please use the *system* python to run this script
Traceback (most recent call last):
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/virtualenv/virtualenv.py", line 2325, in <module>
    main()
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/virtualenv/virtualenv.py", line 711, in main
    symlink=options.symlink and hasattr(os, 'symlink')) # MOZ: Make sure we don't use symlink when we don't have it
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/virtualenv/virtualenv.py", line 924, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/objdir-tb/_virtualenv/lib/python2.7/posixpath.py", line 343, in abspath
    if not isabs(path):
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/objdir-tb/_virtualenv/lib/python2.7/posixpath.py", line 53, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'
Error running mach:

    ['python', '/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/config/printconfigsetting.py', '/builds/slave/tb-c-cen-m64-d-000000000000000/build/objdir-tb/dist/bin/application.ini', 'App', 'BuildID']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Failed to create virtualenv: /builds/slave/tb-c-cen-m64-d-000000000000000/build/objdir-tb/_virtualenv

  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/mach_commands.py", line 42, in python
    self._activate_virtualenv()
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/mozbuild/mozbuild/base.py", line 623, in _activate_virtualenv
    self.virtualenv_manager.ensure()
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/mozbuild/mozbuild/virtualenv.py", line 166, in ensure
    return self.build(python)
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/mozbuild/mozbuild/virtualenv.py", line 439, in build
    self.create(python)
  File "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/mozbuild/mozbuild/virtualenv.py", line 203, in create
    'Failed to create virtualenv: %s' % self.virtualenv_root)
builduid: 362af7bbdb034e528f3b877f7e24e051
revision: e2a93f33a9826ae2039589d25ed0ea6bde13dbf9

There are more errors in the log further down:
https://archive.mozilla.org/pub/thunderbird/tinderbox-builds/comm-central-macosx64/1492287270/comm-central-macosx64-bm84-build1-build111.txt.gz

Anything our build experts can spot easily? Maybe a buildbot issue?

===

Last good:
M-C: 9379831bb9c3d9abfea7dbf8dd06dbdab1

First bad:
M-C: ce69b6e1773e9e0d0a190ce899f34b1658

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=9379831bb9c3d9abfea7dbf8dd06dbdab1&tochange=ce69b6e1773e9e0d0a190ce899f34b1658

Comment 1

[Richard Marti (:Paenglab)]

Philipp, maybe you can help here?

Comment 2

[Frank-Rainer Grahl (:frg)]

Maybe caused by bug 1356101. Don't think its bug 1356541.

Don't own a mac. Is local building still possible?

Comment 3

[Jorg K (CEST = GMT+2)]

Gregory and Mike, could you please cast an eye on this failure and give us a hint.

Comment 4

[Mike Hommey [:glandium]]

(In reply to Jorg K (GMT+2) from comment #0)
> builder: comm-central-macosx64-debug
> slave: bld-lion-r5-081
> starttime: 1492294089.89
> results: failure (2)
> buildid: Please use the *system* python to run this script
> Traceback (most recent call last):
>   File
> "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/
> virtualenv/virtualenv.py", line 2325, in <module>
>     main()
>   File
> "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/
> virtualenv/virtualenv.py", line 711, in main
>     symlink=options.symlink and hasattr(os, 'symlink')) # MOZ: Make sure we
> don't use symlink when we don't have it
>   File
> "/builds/slave/tb-c-cen-m64-d-000000000000000/build/mozilla/python/
> virtualenv/virtualenv.py", line 924, in create_environment
>     site_packages=site_packages, clear=clear, symlink=symlink))

That code reads:

    py_executable = os.path.abspath(install_python(
        home_dir, lib_dir, inc_dir, bin_dir,
        site_packages=site_packages, clear=clear, symlink=symlink))

You need to figure out why install_python is returning None in your case.

Comment 5

[Gregory Szorc [:gps]]

What glandium said.

Comment 6

[Jorg K (CEST = GMT+2)]

This is intermittent, would you believe.

Failed: Mon Apr 17, 0:15:06, both opt and debug
Worked: Mon Apr 17, 18:04:06, both opt and debug
Failed: Mon Apr 17, 22:39:06, in fact, opt failed, debug worked.

Comment 7

[Gregory Szorc [:gps]]

I'm guessing it is intermittent due to file reuse from a previous job. There's probably a symlink pointing to a now-removed directory (perhaps a per-job temporary directory) or something like that.

Comment 8

[Jorg K (CEST = GMT+2)]

Not seen any more, we can reopen if it happens again.