1285618 - Running browser_CTP_plugins.js locally fails consistently

Status: RESOLVED FIXED

(Toolkit :: Add-ons Manager, defect)

Description

[Benjamin Smedberg]

STR:
* Build Firefox from a known-good version of mozilla-central. `mach clobber && mach build`
** In my particular case, I used https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=94cce4e79310&selectedJob=4270049
** Win10, --enable-debug, MSVC2015 release 2
* Run `mach test toolkit/mozapps/extensions/test/browser/browser_CTP_plugins.js`

EXPECTED:
The test runs and passes. Perhaps in multiple configurations, because I see that this test is run multiple times on treeherder.

ACTUAL:
The test runs, loads http://127.0.0.1:8888/browser/toolkit/mozapps/extensions/test/browser//plugin_test.html which is a 404 and then it hangs and eventually times out/dies. It then repeats this another time.

I don't know whether that file is *supposed* to be at that URL. This could well be a test packaging problem. There is no copy of `plugin_test.html` anywhere in my objdir. But I don't know how tests get packaged up any more: it used to be that everything was shoved into objdir/testing.

Comment 1

[Gregory Szorc [:gps]]

I think the problem here is support-files foo in the test .ini files under toolkiz/mozapps/extensions/test/browser/.

browser.ini and browser-window.ini are defined in moz.build files.

browser-window.ini includes browser-common.ini, which defines browser_CTP_plugins.js. However, plugin_test.html is listed as a support-files in browser.ini, which isn't included by browser-window.ini.

As of a few months ago, the build system lazily installs only the required tests and support-files at test invocation time (because installing a few hundred files is faster than checking thousands of them on every test invocation). Since plugin_test.html isn't known by browser-window.ini, it doesn't get installed and the test fails. It works in automation because all files are available in automation.

The fix is likely to move support-files from browser.ini to browser-common.ini.

Comment 2

[Benjamin Smedberg]

Attachment: Bug 1285618 - Move support files to browser-common.ini so that running individual addon manager tests installs them properly.

Review commit: https://reviewboard.mozilla.org/r/63280/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63280/

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8769330 [details]
Bug 1285618 - Move support files to browser-common.ini so that running individual addon manager tests installs them properly.

https://reviewboard.mozilla.org/r/63280/#review60122

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/09595a7eabf2
Move support files to browser-common.ini so that running individual addon manager tests installs them properly. r=gps

Comment 5

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/09595a7eabf2

Comment 6

[Andrew Swan [:aswan]]

This changes seems to have broken local runs of tests defined only in browser.ini (and perhaps browser-windows.ini too?)
In particular, I'm trying to run browser_webapi_addon_listener.js which is defined in browser.ini.  browser.ini includes browser-common.ini, but the support files that just got moved there do not get copied when I run that test.
gps, is that intended or is it a known bug?

Comment 7

[Gregory Szorc [:gps]]

Looks like you found a bug in test installation. Filed bug 1286016.

You can work around the issue by running a test from browser-common.ini in the same invocation of something defined in browser.ini. If you are on Linux or OS X, you only need to do this once to get the symlinks in place. If Windows, you only need to run an extra test every time a support-files changes.

Hopefully we'll fix the underlying issue soon so a workaround isn't needed.