1577912 - test harness should understand static prefs

Status: NEW

(Testing :: General, defect, P3)

Description

[Perry Jiang [:perry] [no longer employee, use ni?]]

I'm trying to make a static pref true by default. There's tests that are supposed to be skipped if that pref is true, as annotated in its expectation file via skip-if = serviceworker_e10s (for mochitest and xpcshell) and if sw-e10s (for WPTs), but the tests aren't actually being skipped because I believe the test harnesses still think that the pref value is false.

Comment 1

[Geoff Brown [:gbrown]]

It looks like the serviceworker_e10s support was added in bug 1470266. If I understand that correctly, I'd only expect the manifest condition serviceworker_e10s to be true for the serviceworker tests:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=264344048&searchStr=xpcshell%2Cserviceworker

and it looks like test_observer_remoting.js is skipped there:

https://taskcluster-artifacts.net/f6aDmaYcSuSyQln-d8U-uw/0/public/logs/live_backing.log
[task 2019-08-30T17:05:04.051Z] 17:05:04 INFO - TEST-SKIP | dom/push/test/xpcshell/test_observer_remoting.js | took 0ms

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Ah, I think I may have confused things in my analysis when discussing this with Perry. Although if Perry tested with this and it worked, I may be experiencing nested levels of confusion.

The "serviceworker_e10s" gets set in https://searchfox.org/mozilla-central/rev/c8dba8c886609d0fdb8fae68bab865bf03f1165c/testing/xpcshell/runxpcshelltests.py#1218 based on the "dom.serviceWorkers.parent_intercept" pref there.

I had thought this was based on pulling the preferences out of the profile and executable itself via calls like https://searchfox.org/mozilla-central/rev/c8dba8c886609d0fdb8fae68bab865bf03f1165c/testing/mozbase/mozprofile/mozprofile/prefs.py#156. But it appears it instead only gets its preferences from buildPrefsFile's call to parse_preferences at https://searchfox.org/mozilla-central/rev/c8dba8c886609d0fdb8fae68bab865bf03f1165c/testing/xpcshell/runxpcshelltests.py#991 which is only influenced by explicit calls to "--setpref" or similar.

So I guess the right course of action for Perry's motivating scenario is to have preference-based checks only happen using the built-in test condition mechanisms to the framework, or just force the test to use the pref it wants, or to bypass most of the logic if the preference is wrong.

To help others being similarly confused in the future, it might be good if updateMozInfo changed its prefs arg to something like explicit_commandline_prefs_to_set, as prefs is a bit ambiguous on its own in terms of where they come from and where they're going. (For example, the fact that there's a fission pref there is likely to be super confusing to people as well.)

Comment 3

[Perry Jiang [:perry] [no longer employee, use ni?]]

My local testing is consistent with Andrew's explanation