Open Bug 1162955 Opened 7 years ago Updated 2 years ago

REFTEST_MANIFESTS property does nothing locally


(Firefox Build System :: General, defect)

Not set


(Not tracked)


(Reporter: kats, Unassigned)



+++ This bug was initially created as a clone of Bug #1162954 +++

- Create some new reftests, list them in a reftest.list, and add that list file to REFTEST_MANIFESTS in a file (see e.g.
- Run reftests locally. This works fine
- Now, remove the reftest.list from the file (i.e. don't add it to REFTEST_MANIFESTS) and try to run the reftests

Expected: reftests don't run
Actual: Reftests DO get run.

Now, rename your reftest.list file to or something else, and put it back in your file in REFTEST_MANIFESTS. Try to run reftests.

Expected: reftests run
Actual: reftests do not run, because mach cannot find the "reftest.list" file.

Apparently it doesn't bother using REFTEST_MANIFESTS at all, and just has reftest.list hard-coded somewhere. It dumps something like this out:

Exception: No manifest file was found at gfx/layers/apz/reftests/reftest.list.

  File "/home/kats/zspace/B2G-flame-kk/gecko/layout/tools/reftest/", line 422, in run_reftest
    return self._run_reftest(test_file, suite='reftest', **kwargs)
  File "/home/kats/zspace/B2G-flame-kk/gecko/layout/tools/reftest/", line 450, in _run_reftest
    return reftest.run_desktop_test(test_file, suite=suite, **kwargs)
  File "/home/kats/zspace/B2G-flame-kk/gecko/layout/tools/reftest/", line 265, in run_desktop_test
    raise Exception('No manifest file was found at %s.' % path)
The goal was always to follow up and do something useful with the metadata. When you don't have full-time staffing on the build system, things like this fall through the cracks.

It's been a year and AFAIK you are the only one to fall through this crack. So this bug will likely linger.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.