1723536 - objdir/temp/user.js generated by XPCShellTests' buildPrefsFile expands on every test run

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Rob Wu [:robwu]]

The xpcshell test runner creates a user.js file at https://searchfox.org/mozilla-central/rev/4b88e0b8cca115009e82fdd65e5bf5812ff99128/testing/xpcshell/runxpcshelltests.py#1122-1159. The tempDir referenced there is not a temporary directory, but a fixed one at objdir/temp/ (set by the mach command runner).

This buildPrefsFile logic relies on profile.merge, which appends prefs to an existing user.js file if any. At every test run (./mach test path/to/xpcshell/test.js), this file becomes increasingly larger (the directory is not cleaned up in this case).

Within the same checkout of the tree, any existing prefs are effectively overwritten because the last value takes precedence. But if a pref is removed from the testing/profiles/xpcshell/user.js file (e.g. after updating the tree), then a test run with the non-clobbered object directory can result in an unexpected test failure if the pref did anything significant.

To avoid that issue, we should empty that user.js file before re-generating it.
A side effect of that change is that devs cannot manually add prefs to objdir/temp/user.js. But for them the alternative would be to use the --pref flag of mach (or prefs = in test manifests), so I think that such a change would be fine.

Comment 1

[Rob Wu [:robwu]]

Working on it as part of bug 1723198.

In remote xpcshell tests, the file is currently removed at the end of a test run. The fix for bug 1723198 causes the file to be kept, which triggers this bug.

Comment 2

[Rob Wu [:robwu]]

Attachment: Bug 1723536 - Clear generated user.js before start of xpcshell test

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/15c67fd83308
Clear generated user.js before start of xpcshell test r=jmaher

Comment 4

[Alexandru Michis [:malexandru]]

Backed out 2 changesets (Bug 1723198, Bug 1723536) for causing xpcshell failures in test_SocketScalars.js and test_ext_cookieBehaviors.js
Backout link: https://hg.mozilla.org/integration/autoland/rev/5d3b6ace0a3b0a38f16f92688998e1d0f55ea3cb
Push with failures, failure log:

• test_SocketScalars.js
  (Update): The cause for the failures in test_ext_cookieBehaviors.js was actually Bug 1650089.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/47c08d2c33f4
Clear generated user.js before start of xpcshell test r=jmaher

Comment 6

[Iulian Moraru]

Backed out for causing xpcshell failures on test_SocketScalars.js.

Push with failures

Failure log

Backout link

Comment 7

[Rob Wu [:robwu]]

Handled in bug 1723198

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11c7bc6d58ee
Clear generated user.js before start of xpcshell test r=jmaher

Comment 9

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/11c7bc6d58ee