Closed Bug 1723198 Opened 2 months ago Closed 2 months ago

prefs option in xpcshell manifest causes Android tests to run without testing/profiles/xpcshell/user.js

Categories

(Testing :: XPCShell Harness, defect)

Default
defect

Tracking

(firefox-esr78 wontfix, firefox-esr91 wontfix, firefox91 wontfix, firefox92 wontfix, firefox93 fixed)

RESOLVED FIXED
93 Branch
Tracking Status
firefox-esr78 --- wontfix
firefox-esr91 --- wontfix
firefox91 --- wontfix
firefox92 --- wontfix
firefox93 --- fixed

People

(Reporter: robwu, Assigned: robwu)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

In bug 1722966 the patch was backed out because of an unexpected failure on Android. It turns out that a pref that is supposedly set on all platforms and all test types wasn't set, because somehow the prefs set in testing/profiles/xpcshell/user.js were missing.

The prefs were missing because since bug 1635227 the presence of the prefs option in a xpcshell test manifest triggers logic to generate a new user.js file in updateTestPrefsFile. The logic assumes that the profile directory contains a user.js file. In the default case, this file is generated at testing/profiles/xpcshell/user.js generated in buildPrefsFile. In the remote case (e.g. when Android xpcshell tests are run), the subclass in remotexpcshelltests.py overrides the buildPrefsFile method with one that deletes the local user.js file.
Consequently, when updateTestPrefsFile attempts to generate a new user.js file, it will derive the file from an empty set of preferences (due to the missing user.js).

This can be fixed by not deleting the local user.js file.

And once fixed, we could revisit the changes from bug 1614966 (which added prefs = in some xpcshell manifests (example of xpcshell.ini)), because services.settings.server is already present in testing/profiles/xpcshell/user.js. The prefs added in bug 1614966 didn't do anything until bug 1635227 landed, so the base set of preferences in the user.js file is probably sufficient. This will also make extension tests run faster on Android, because there is no longer a need for pushing a new user.js file for each test to the device.

Depends on: 1723189
No longer depends on: 1723189
See Also: → 1723189
See Also: → 1723536

The new tests in testing/xpcshell/example/ cover the following scenarios:

Try run is green where we care: https://treeherder.mozilla.org/jobs?repo=try&revision=028b837991ba975f1864ef3047e01605754142c4
The new tests pass locally, and I can confirm that they pass on try too (e.g. the X1 job of android-em-7-0-x86_64-lite-qr opt).

I classified the orange failures as pre-existing issues. The x-fis tests are a non-standard configuration (with pre-existing failures) and can be ignored according to nika in the Fission channel on Matrix.

Pushed by rob@robwu.nl:
https://hg.mozilla.org/integration/autoland/rev/43a0e54578b1
Don't delete user.js in remotexpcshelltests.py + tests r=jmaher

Backed out 2 changesets (Bug 1723198, Bug 1723536) for causing xpcshell failures in test_SocketScalars.js and test_ext_cookieBehaviors.js
Backout link: https://hg.mozilla.org/integration/autoland/rev/5d3b6ace0a3b0a38f16f92688998e1d0f55ea3cb
Push with failures, failure log:

Flags: needinfo?(rob)

The try push was good as seen in comment 3.

The test failure in test_SocketScalars.js is bug 1716307, and unrelated to the change here.

Flags: needinfo?(rob)

Soft freeze is over, I'm going to reland.

The failure in test_SocketScalars.js is independent of the work here, and bug 1716307 has a patch, so if there are any newly reported failures, associate them with bug 1716307 rather than backing out my patches again.

Pushed by rob@robwu.nl:
https://hg.mozilla.org/integration/autoland/rev/16c398716c60
Don't delete user.js in remotexpcshelltests.py + tests r=jmaher

Backed out for causing xpcshell failures on test_SocketScalars.js.

Push with failures

Failure log

Backout link

Flags: needinfo?(rob)
Flags: needinfo?(rob)
Pushed by rob@robwu.nl:
https://hg.mozilla.org/integration/autoland/rev/127d855f7cf9
Don't delete user.js in remotexpcshelltests.py + tests r=jmaher
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch
You need to log in before you can comment on or make changes to this bug.