Figure out how to make reftests run in automation when legacy extensions are disabled

RESOLVED FIXED in Firefox 55


2 years ago
2 years ago


(Reporter: aswan, Assigned: aswan)


Version 3
Dependency tree / graph

Firefox Tracking Flags

(firefox55 fixed)



(1 attachment)



2 years ago
Reftests use some browser extensions that are not webextensions and hence will not load in release and beta beginning with version 57.  These extensions currently circumvent the signing requirements by using temporary installation but that won't work for legacy extensions.

Comment 1

2 years ago
We will still allow legacy extensions to be loaded if Cu.isInAutomation is true and the extensions.legacy.enabled preference is set to true.  If we can arrange for that to happen for reftests, I think that would be the simplest solution.

The test for Cu.isInAutomation is here:

It looks like reftests already set the preference referenced there:

But apparently MOZ_DISABLE_NONLOCAL_CONNECTIONS is not set in reftests, if we can set it that would be great.
The needinfo here is kind of a shot in the dark, please redirect as necessary...
Flags: needinfo?(dbaron)
What info do you need?
Flags: needinfo?(dbaron) → needinfo?(aswan)

Comment 3

2 years ago
Sorry, from comment 1, is it safe to set MOZ_DISABLE_NONLOCAL_CONNECTIONS in reftests?  If so, where is the right place to do it?
Flags: needinfo?(aswan) → needinfo?(dbaron)
(In reply to Andrew Swan [:aswan] from comment #3)
> Sorry, from comment 1, is it safe to set MOZ_DISABLE_NONLOCAL_CONNECTIONS in
> reftests?  If so, where is the right place to do it?

We should probably do so, though there might be some failures you'll have to sort through.

Probably the best place is in somewhere in and possibly also  jmaher probably knows the best place(s) that will catch as many reftest variants as possible.
Flags: needinfo?(dbaron) → needinfo?(jmaher)
Comment hidden (mozreview-request)

Comment 6

2 years ago
Well, I had it backward in comment 1.  Turns out the NONLOCAL_CONNECTIONS environment variable is set during reftests, but the security.turn_of_all_security... preference is generally not.  The attached patch sets that preference unconditionally, and also sets extensions.legacy.enabled.  This try run shows reftests passing on linux64 opt with the attached patch and the build-time flag for legacy extensions disabled:


2 years ago
Attachment #8873707 - Flags: review?(jmaher)
I would be concerned about autophone, could you push to try with:
try: -b do -p android-api-15 -u plain-reftest-1,jsreftest-1,crashtest-1,autophone-crashtest-dom-media,autophone-reftest-ogg-video -t none

notice there is jsreftest and crashtest to sort out as well, I would like to see that for linux (I have it in the above syntax)
Flags: needinfo?(jmaher)
Comment on attachment 8873707 [details]
Bug 1369525: Make sure reftests still run when legacy extensions are disabled

assuming things work on autophone and sanity checked on crashtest and jsreftest.
Attachment #8873707 - Flags: review?(jmaher) → review+

Comment 9

2 years ago
Since reviewboard only shows the most recent try run, here is the first one that covers reftests across all platforms:
And here is the new one requested in comment 7:

Comment 10

2 years ago
There's on failure in the first try run from comment 9 but it matches a known intermittent.  A few jobs in the second try run aren't finished but they're all tier 2.  Everything else is green, so I'm going to go ahead and land this.


2 years ago
Assignee: nobody → aswan

Comment 11

2 years ago
Pushed by
Make sure reftests still run when legacy extensions are disabled r=jmaher

Comment 12

2 years ago
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
You need to log in before you can comment on or make changes to this bug.