Closed Bug 1599756 Opened 3 months ago Closed 3 months ago

Perma Beta TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out & test_privacy_other_prefs - [test_privacy_other_prefs : 380] when Gecko 72 merges to Beta on 2019-12-02

Categories

(WebExtensions :: General, defect, P1)

defect

Tracking

(firefox-esr68 unaffected, firefox70 unaffected, firefox71 unaffected, firefox72+ verified)

VERIFIED FIXED
mozilla72
Tracking Status
firefox-esr68 --- unaffected
firefox70 --- unaffected
firefox71 --- unaffected
firefox72 + verified

People

(Reporter: malexandru, Assigned: rpl)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

Central as Beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=278402922&revision=0057e6e44b1898fa3e4ec6324b970396fb9bad47&searchStr=%28x&group_state=expanded

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278402922&repo=try&lineNumber=3282

[task 2019-11-27T13:43:42.464Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal.js | took 0ms
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal_url.js
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal_url.js | took 0ms
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_proxy_socks.js
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_proxy_socks.js | took 1ms
[task 2019-11-27T13:43:42.467Z] 13:43:42 INFO - Retrying tests that failed when run in parallel.
[task 2019-11-27T13:43:42.470Z] 13:43:42 INFO - TEST-START | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js
[task 2019-11-27T13:48:42.484Z] 13:48:42 WARNING - TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out
[task 2019-11-27T13:48:42.484Z] 13:48:42 INFO - TEST-INFO took 300001ms
[task 2019-11-27T13:48:42.485Z] 13:48:42 INFO - >>>>>>>
[task 2019-11-27T13:48:42.486Z] 13:48:42 INFO - (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2019-11-27T13:48:42.486Z] 13:48:42 INFO - (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2019-11-27T13:48:42.487Z] 13:48:42 INFO - (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2019-11-27T13:48:42.488Z] 13:48:42 INFO - running event loop
[task 2019-11-27T13:48:42.488Z] 13:48:42 INFO - xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Starting check_remote
[task 2019-11-27T13:48:42.489Z] 13:48:42 INFO - (xpcshell/head.js) | test check_remote pending (2)
[task 2019-11-27T13:48:42.489Z] 13:48:42 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | check_remote - [check_remote : 1] useRemoteWebExtensions matches - true == true
[task 2019-11-27T13:48:42.490Z] 13:48:42 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | check_remote - [check_remote : 1] testing from extension process - false == false

Andrea, I believe this is caused by the recent changes pushed in Bug 1593635, could you please take a look?

Flags: needinfo?(amarchesini)

Another failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278621494&repo=try&lineNumber=3125

[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 382] security.tls.version.max set correctly for [object Object] - 4 == 4
[task 2019-11-28T12:51:24.369Z] 12:51:24 WARNING - TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 380] {"minimum":"TLSv1","maximum":"TLSv1.1"} deepEqual {"minimum":"TLSv1.2","maximum":"TLSv1.1"}
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/extensions/test/xpcshell/test_ext_privacy.js:testSetting:380
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/head.js:_do_main:246
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test:573
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - -e:null:1
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - exiting test
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - Unexpected exception NS_ERROR_ABORT:
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _abort_failed_test@/builds/worker/workspace/build/tests/xpcshell/head.js:791:20
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - do_report_result@/builds/worker/workspace/build/tests/xpcshell/head.js:892:5
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - Assert<@/builds/worker/workspace/build/tests/xpcshell/head.js:67:21
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - proto.report@resource://testing-common/Assert.jsm:233:10
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - deepEqual@resource://testing-common/Assert.jsm:310:8
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - testSetting@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/extensions/test/xpcshell/test_ext_privacy.js:380:14
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _do_main@/builds/worker/workspace/build/tests/xpcshell/head.js:246:6
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:573:5
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - @-e:1:1
[task 2019-11-28T12:51:24.371Z] 12:51:24 INFO - exiting test

Summary: Perma Beta TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out when Gecko 72 merges to Beta on 2019-12-02 → Perma Beta TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out & test_privacy_other_prefs - [test_privacy_other_prefs : 380] when Gecko 72 merges to Beta on 2019-12-02

Hey Andrea,
the reason for the error mentioned in comment 0 is unfortunately one that is happening very often during beta simulations :-(
it is due to the different behavior between nightly and beta (in nightly the extensions are always considered as privileged by default, which is not the case on beta builds).

That failure should be prevented by calling AddonTestUtils.overrideCertDB(), like other xpcshell tests are doing for the same reason.

The other failure from comment 1 seems to be instead related to a different value that the test got from that setting while running in the dev edition:

xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 380] {"minimum":"TLSv1","maximum":"TLSv1.1"} deepEqual {"minimum":"TLSv1.2","maximum":"TLSv1.1"}

As a side note, we should have included in Bug 1593635 also a test that verify the opposite behavior (verify as part of the automated tests that changing the setting from an unprivileged extension should reject), it would be still ok to add that test in a separate followup.

Assignee: nobody → lgreco

I've just attached a small patch to fix both the failures described in comment 2.

Flags: needinfo?(amarchesini)
Status: NEW → ASSIGNED
Priority: -- → P1

(In reply to Cosmin Sabou [:CosminS] from comment #5)

The patch works in today's beta-sims: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278877944&repo=try&lineNumber=2128

Thanks for confirming it.

If landing this patch sooner is going to be necessary to unblock the merge to beta, feel free to land it even if baku has not been able to reach the patch yet
(this revision has been already accepted by another WebExtension peer, besides myself, and it is a test-only change and so we could even uplift further changes to the test case later if necessary).

Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e429ab021364
Fix test_ext_privacy.js permafailure on 72 Beta and DevEdition builds. r=zombie
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72

Verified fixed with yesterday's beta sim and on beta.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.