1599756 - Perma Beta TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out & test_privacy_other_prefs - [test_privacy_other_prefs : 380] when Gecko 72 merges to Beta on 2019-12-02

Status: VERIFIED FIXED

(WebExtensions :: General, defect, P1)

Description

[Alexandru Michis [:malexandru]]

Central as Beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=278402922&revision=0057e6e44b1898fa3e4ec6324b970396fb9bad47&searchStr=%28x&group_state=expanded

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278402922&repo=try&lineNumber=3282

[task 2019-11-27T13:43:42.464Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal.js | took 0ms
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal_url.js
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal_url.js | took 0ms
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_proxy_socks.js
[task 2019-11-27T13:43:42.465Z] 13:43:42 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_proxy_socks.js | took 1ms
[task 2019-11-27T13:43:42.467Z] 13:43:42 INFO - Retrying tests that failed when run in parallel.
[task 2019-11-27T13:43:42.470Z] 13:43:42 INFO - TEST-START | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js
[task 2019-11-27T13:48:42.484Z] 13:48:42 WARNING - TEST-UNEXPECTED-TIMEOUT | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Test timed out
[task 2019-11-27T13:48:42.484Z] 13:48:42 INFO - TEST-INFO took 300001ms
[task 2019-11-27T13:48:42.485Z] 13:48:42 INFO - >>>>>>>
[task 2019-11-27T13:48:42.486Z] 13:48:42 INFO - (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2019-11-27T13:48:42.486Z] 13:48:42 INFO - (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2019-11-27T13:48:42.487Z] 13:48:42 INFO - (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2019-11-27T13:48:42.488Z] 13:48:42 INFO - running event loop
[task 2019-11-27T13:48:42.488Z] 13:48:42 INFO - xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | Starting check_remote
[task 2019-11-27T13:48:42.489Z] 13:48:42 INFO - (xpcshell/head.js) | test check_remote pending (2)
[task 2019-11-27T13:48:42.489Z] 13:48:42 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | check_remote - [check_remote : 1] useRemoteWebExtensions matches - true == true
[task 2019-11-27T13:48:42.490Z] 13:48:42 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | check_remote - [check_remote : 1] testing from extension process - false == false

Andrea, I believe this is caused by the recent changes pushed in Bug 1593635, could you please take a look?

Comment 1

[Cristina Coroiu [:ccoroiu]]

Another failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278621494&repo=try&lineNumber=3125

[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - TEST-PASS | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 382] security.tls.version.max set correctly for [object Object] - 4 == 4
[task 2019-11-28T12:51:24.369Z] 12:51:24 WARNING - TEST-UNEXPECTED-FAIL | xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 380] {"minimum":"TLSv1","maximum":"TLSv1.1"} deepEqual {"minimum":"TLSv1.2","maximum":"TLSv1.1"}
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/extensions/test/xpcshell/test_ext_privacy.js:testSetting:380
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/head.js:_do_main:246
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test:573
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - -e:null:1
[task 2019-11-28T12:51:24.369Z] 12:51:24 INFO - exiting test
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - Unexpected exception NS_ERROR_ABORT:
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _abort_failed_test@/builds/worker/workspace/build/tests/xpcshell/head.js:791:20
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - do_report_result@/builds/worker/workspace/build/tests/xpcshell/head.js:892:5
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - Assert<@/builds/worker/workspace/build/tests/xpcshell/head.js:67:21
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - proto.report@resource://testing-common/Assert.jsm:233:10
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - deepEqual@resource://testing-common/Assert.jsm:310:8
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - testSetting@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/extensions/test/xpcshell/test_ext_privacy.js:380:14
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _do_main@/builds/worker/workspace/build/tests/xpcshell/head.js:246:6
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - _execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:573:5
[task 2019-11-28T12:51:24.370Z] 12:51:24 INFO - @-e:1:1
[task 2019-11-28T12:51:24.371Z] 12:51:24 INFO - exiting test

Comment 2

[Luca Greco [:rpl] [:luca] [:lgreco]]

Hey Andrea,
the reason for the error mentioned in comment 0 is unfortunately one that is happening very often during beta simulations :-(
it is due to the different behavior between nightly and beta (in nightly the extensions are always considered as privileged by default, which is not the case on beta builds).

That failure should be prevented by calling AddonTestUtils.overrideCertDB(), like other xpcshell tests are doing for the same reason.

The other failure from comment 1 seems to be instead related to a different value that the test got from that setting while running in the dev edition:

xpcshell-remote.ini:toolkit/components/extensions/test/xpcshell/test_ext_privacy.js | test_privacy_other_prefs - [test_privacy_other_prefs : 380] {"minimum":"TLSv1","maximum":"TLSv1.1"} deepEqual {"minimum":"TLSv1.2","maximum":"TLSv1.1"}

---

As a side note, we should have included in Bug 1593635 also a test that verify the opposite behavior (verify as part of the automated tests that changing the setting from an unprivileged extension should reject), it would be still ok to add that test in a separate followup.

Comment 3

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1599756 - Fix test_ext_privacy.js permafailure on 72 Beta and DevEdition builds. r?baku!,zombie!

Comment 4

[Luca Greco [:rpl] [:luca] [:lgreco]]

I've just attached a small patch to fix both the failures described in comment 2.

Comment 5

[Cosmin Sabou [:CosminS]]

The patch works in today's beta-sims: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278877944&repo=try&lineNumber=2128

Comment 6

[Luca Greco [:rpl] [:luca] [:lgreco]]

(In reply to Cosmin Sabou [:CosminS] from comment #5)

The patch works in today's beta-sims: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=278877944&repo=try&lineNumber=2128

Thanks for confirming it.

If landing this patch sooner is going to be necessary to unblock the merge to beta, feel free to land it even if baku has not been able to reach the patch yet
(this revision has been already accepted by another WebExtension peer, besides myself, and it is a test-only change and so we could even uplift further changes to the test case later if necessary).

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e429ab021364
Fix test_ext_privacy.js permafailure on 72 Beta and DevEdition builds. r=zombie

Comment 8

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/e429ab021364

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed with yesterday's beta sim and on beta.