Closed Bug 1334093 Opened 3 years ago Closed 3 years ago

Test jobs should always extract the mach script and mozinfo.json (for manifestparser) from


(Release Engineering :: Applications: MozharnessCore, defect)

Not set


(firefox52 fixed, firefox-esr52 fixed, firefox53 fixed, firefox54 fixed)

Tracking Status
firefox52 --- fixed
firefox-esr52 --- fixed
firefox53 --- fixed
firefox54 --- fixed


(Reporter: whimboo, Assigned: whimboo)




(1 file)

As seen on bug 1330253 at least the firefox-ui-functional jobs are not extracting the mozinfo.json file from the package. As result the test harness might not be able to filter tests for skipping based on values inside the mozinfo.json file, like:

> skip-if = debug

A quick look showed me that at least all Marionette based test jobs are affected, but not the marionette script itself.
Given that this is somewhat urgent for being able to skip the correct tests and especially for bug 1330253, I'm going to take it.
Assignee: nobody → hskupin
So the two files we are talking about here are actually the mach script and the mozinfo.json file. Both are in the root directory of the common.tests archive and should be unpacked everytime IMHO. So special-casing it for specific test suites doesn't make sense. It means I think it's better to put some code into download_and_extract() for, which adds those by default, even if not specified by extract_dirs.
Currently it's not that easy to always white-list top-level files for extraction of The method uses fnmatch.filter and this only supports '*'. In our own code-base we also have '**' which means recursively.

So we would have to get a copy of fnmatch.filter() and make the modifications. Aki, do you have some feedback for me, or another idea? Thanks.
Flags: needinfo?(aki)
It appears that adding 'mozinfo.json' to the extract_dirs Just Works.

>>> import fnmatch
>>> import functools
>>> import itertools
>>> import zipfile
>>> bundle = zipfile.ZipFile("/Users/asasaki/Downloads/")
>>> 'mozinfo.json' in bundle.namelist()
>>> filter_partial = functools.partial(fnmatch.filter, bundle.namelist())
>>> entries = itertools.chain(*map(filter_partial, ['mozinfo.json', 'reftest/*']))
>>> for entry in entries:
...     print entry

So after we could possibly

    if isintance(unpack_dirs, list) and 'mozinfo.json' not in unpack_dirs:

It would be good to push that to try to verify it doesn't break anything :)
Flags: needinfo?(aki)
Adding mozinfo.json to the list of files to unpack from there is a permanent failure for a single browser chrome test with the name browser_verify_content_about_newtab.js.

Interestingly the same test gets run in bc2, bc4, bc5, and bc6 chunks, and shows failures like:

TEST-UNEXPECTED-FAIL | dom/security/test/contentverifier/browser_verify_content_about_newtab.js | Valid remote newtab page must have built-in CSP. - Got {}, expected {"csp-policies":[{"report-only":false,"script-src":["","'unsafe-inline'"],"style-src":[""]}]}

The content of mozinfo.json looks like the following for Windows:

    "addon_signing": true, 
    "appname": "firefox", 
    "artifact": false, 
    "asan": false, 
    "bin_suffix": ".exe", 
    "bits": 32, 
    "buildapp": "browser", 
    "buildtype_guess": "debug", 
    "crashreporter": true, 
    "datareporting": true, 
    "debug": true, 
    "healthreport": true, 
    "mozconfig": "c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/.mozconfig", 
    "nightly_build": true, 
    "official": true, 
    "os": "win", 
    "pgo": false, 
    "platform_guess": "win32", 
    "processor": "x86", 
    "release_or_beta": false, 
    "require_signing": false, 
    "stylo": false, 
    "sync": true, 
    "telemetry": false, 
    "tests_enabled": true, 
    "toolkit": "windows", 
    "topsrcdir": "c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src", 
    "tsan": false, 
    "updater": true

Franziskus or Mike, do you have an idea what this could be?
Flags: needinfo?(franziskuskiefer)
Flags: needinfo?(mconley)
Oh, wait. The browser.ini manifest file shows that this test started perma failing lately and got skipped 3 days ago via bug 1336654. I was clearly working on an old changeset when pushing it to try.

So the changes introduced with my patch should not have caused any regression.
Flags: needinfo?(mconley)
Flags: needinfo?(franziskuskiefer)
Summary: test jobs should always extract mozinfo.json from → Test jobs should always extract the mach script and mozinfo.json (for manifestparser) from
For safety I rebased my patch against current central, and pushed a new try build for browser-chrome tests only:
Comment on attachment 8834016 [details]
Bug 1334093 - Test jobs should always extract the mach script and mozinfo.json from

I don't love the hardcode, but we're already hardcoding jsshell, and we're only affecting tests. r=me
Attachment #8834016 - Flags: review?(aki) → review+
Pushed by
Test jobs should always extract the mach script and mozinfo.json from r=aki
Closed: 3 years ago
Resolution: --- → FIXED
Test-only change which we would like to see uplifted to aurora and beta. Thanks.
Whiteboard: [checkin-needed-aurora][checkin-needed-beta]
Whiteboard: [checkin-needed-aurora][checkin-needed-beta] → [checkin-needed-beta]
You need to log in before you can comment on or make changes to this bug.