1543696 - Perma toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Color property should be unset - Got rgb(255, 0, 255), expected when Gecko 68 merges to Beta on 2019-05-06

Status: VERIFIED FIXED

(WebExtensions :: General, defect, P2)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: aciure [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=239665536&repo=try

https://queue.taskcluster.net/v1/task/SSN-T6OWTWiPpBcxbTRBbg/runs/0/artifacts/public/logs/live_backing.log

13:30:21 INFO - TEST-PASS | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Generic Property should be unset. -
13:30:21 INFO - Testing that current window updated with the experiment applied
13:30:21 INFO - Buffered messages finished
13:30:21 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Color property should be unset - Got rgb(255, 0, 255), expected
13:30:21 INFO - Stack trace:
13:30:21 INFO - chrome://mochikit/content/browser-test.js:test_is:1325
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:testExperimentApplied:61
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:test_experiment_static_theme:71
13:30:21 INFO - Not taking screenshot here: see the one that was previously logged
13:30:21 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Image property should be unset - Got url(moz-extension://156a68f5-0ca5-485e-b23e-6b139f02946e/background.jpg), expected
13:30:21 INFO - Stack trace:
13:30:21 INFO - chrome://mochikit/content/browser-test.js:test_is:1325
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:testExperimentApplied:63
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:test_experiment_static_theme:71
13:30:21 INFO - Not taking screenshot here: see the one that was previously logged
13:30:21 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Generic Property should be unset. - Got no-repeat, expected
13:30:21 INFO - Stack trace:
13:30:21 INFO - chrome://mochikit/content/browser-test.js:test_is:1325
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:testExperimentApplied:65
13:30:21 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:test_experiment_static_theme:71
13:30:21 INFO - Testing that new window initialized with the experiment applied
13:30:21 INFO - GECKO(9260) | JavaScript warning: chrome://global/content/bindings/tabbox.xml, line 162: Array.forEach is deprecated; use Array.prototype.forEach instead
13:30:21 INFO - Console message: [JavaScript Warning: "Array.forEach is deprecated; use Array.prototype.forEach instead" {file: "chrome://global/content/bindings/tabbox.xml" line: 162}]
13:30:21 INFO - GECKO(9260) | JavaScript warning: chrome://browser/content/browser-ctrlTab.js, line 568: Array.filter is deprecated; use Array.prototype.filter instead
13:30:22 INFO - Not taking screenshot here: see the one that was previously logged
13:30:22 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Color property should be unset - Got rgb(255, 0, 255), expected
13:30:22 INFO - Stack trace:
13:30:22 INFO - chrome://mochikit/content/browser-test.js:test_is:1325
13:30:22 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:testExperimentApplied:61
13:30:22 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:test_experiment_static_theme:76
13:30:22 INFO - Not taking screenshot here: see the one that was previously logged
13:30:22 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js | Image property should be unset - Got url(moz-extension://156a68f5-0ca5-485e-b23e-6b139f02946e/background.jpg), expected
13:30:22 INFO - Stack trace:
13:30:22 INFO - chrome://mochikit/content/browser-test.js:test_is:1325
13:30:22 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:testExperimentApplied:63
13:30:22 INFO - chrome://mochitests/content/browser/toolkit/components/extensions/test/browser/browser_ext_themes_experiment.js:test_experiment_static_theme:76
13:30:22 INFO - Not taking screenshot here: see the one that was previously logged

Comment 1

[Tim Nguyen :ntim]

Attachment: Bug 1543696 - Don't set extensions.legacy.enabled on beta in browser_ext_themes_experiment.js. r=kmag

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Changing the priority to p2 as the bug is tracked by a release manager for the current nightly.
See How Do You Triage for more information

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7f0e02401370
Don't set extensions.legacy.enabled on beta in browser_ext_themes_experiment.js. r=kmag

Comment 4

[Cristina Coroiu [:ccoroiu]]

Backed out changeset 7f0e02401370 (bug 1543696) for browser-chrome failures at toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js

Backout: https://hg.mozilla.org/integration/autoland/rev/7dad55e9bd2bd60ea5462c2cfa1281a82aa0a4b6

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=7f0e024013704d89e01b6f8103fbfd6ef5f0107b

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=240123700&repo=autoland&lineNumber=6496

09:57:19 INFO - TEST-PASS | toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js | monitorConsole | number of messages [{message:/Unrecognized theme property found: properties.unknown_property/}] -
09:57:19 INFO - Testing that experiments are handled correctly when legacy pref is disabled
09:57:19 INFO - Extension loaded
09:57:19 INFO - Console message: Warning: attempting to write 7479 bytes to preference extensions.webextensions.uuids. This is bad for general performance and memory usage. Such an amount of data should rather be written to an external file. This preference will not be sent to any content processes.
09:57:19 INFO - Console message: SENTINEL
09:57:19 INFO - Buffered messages finished
09:57:19 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js | monitorConsole | number of messages [{message:/This extension is not allowed to run theme experiments/}] - got +0, expected 1
09:57:19 INFO - Stack trace:
09:57:19 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.is:320
09:57:19 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:listener:1323
09:57:19 INFO - chrome://mochikit/content/tests/SimpleTest/specialpowersAPI.js:observe/<:377
09:57:19 INFO - Leaving test bound test_experiment
09:57:19 INFO - GECKO(2373) | MEMORY STAT | vsize 5549MB | residentFast 413MB | heapAllocated 183MB

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1ee98a3cd8fb
Don't set extensions.legacy.enabled on beta in browser_ext_themes_experiment.js. r=kmag

Comment 6

[Cristina Coroiu [:ccoroiu]]

Backout: https://hg.mozilla.org/integration/autoland/rev/5f989e2d735902a87a300c8791050926ce2993d3

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cade5cf8a0ed
Don't set extensions.legacy.enabled on beta in browser_ext_themes_experiment.js. r=kmag

Comment 8

[Cristina Coroiu [:ccoroiu]]

Backed out changeset cade5cf8a0ed (bug 1543696) for browser-chrome at toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js

Backout: https://hg.mozilla.org/integration/autoland/rev/b60534ec20b8bc628b691d72f4c3100b4f31b97f

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=240129224&revision=cade5cf8a0ed653fae2b81f9de8219df426a847c

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=240133944&repo=autoland&lineNumber=6493

11:46:36 INFO - TEST-PASS | toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js | monitorConsole | number of messages [{message:/Unrecognized theme property found: properties.unknown_property/}] -
11:46:36 INFO - Testing that experiments are handled correctly when legacy pref is disabled
11:46:36 INFO - Extension loaded
11:46:36 INFO - Console message: Warning: attempting to write 7479 bytes to preference extensions.webextensions.uuids. This is bad for general performance and memory usage. Such an amount of data should rather be written to an external file. This preference will not be sent to any content processes.
11:46:36 INFO - Console message: SENTINEL
11:46:36 INFO - Buffered messages finished
11:46:36 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js | monitorConsole | number of messages [{message:/This extension is not allowed to run theme experiments/}] - got +0, expected 1
11:46:36 INFO - Stack trace:
11:46:36 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.is:320
11:46:36 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:listener:1323
11:46:36 INFO - chrome://mochikit/content/tests/SimpleTest/specialpowersAPI.js:observe/<:377
11:46:36 INFO - Leaving test bound test_experiment
11:46:36 INFO - GECKO(1227) | MEMORY STAT | vsize 5549MB | residentFast 388MB | heapAllocated 135MB
11:46:36 INFO - TEST-OK | toolkit/components/extensions/test/browser/browser_ext_themes_warnings.js | took 1052ms
11:46:36 INFO - checking window state
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1232
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1229
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1233
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1235
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1234
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1236
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1230
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1237
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1228
11:46:38 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1231
11:46:39 INFO - GECKO(1227) | Completed ShutdownLeaks collections in process 1227

Comment 9

[Intermittent Failures Robot]

18 failures in 3025 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 18

Platform breakdown:

• windows10-64-shippable: 2
• windows7-32: 2
• macosx64-devedition: 1
• windows7-32-devedition: 1
• linux32-devedition: 1
• windows10-64-devedition: 1
• linux64: 4
• linux32: 2
• osx-10-10: 2
• osx-10-10-shippable: 1
• linux32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bdd3825511f3
Don't set extensions.legacy.enabled on beta in browser_ext_themes_experiment.js. r=kmag

Comment 11

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/bdd3825511f3

Comment 12

[Intermittent Failures Robot]

21 failures in 543 pushes (0.039 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 21

Platform breakdown:

• linux64: 4
• osx-10-10: 1
• linux32: 2
• windows7-32: 2
• osx-10-10-shippable: 2
• windows10-64: 2
• linux64-shippable: 2
• windows7-32-shippable: 2
• linux32-shippable: 2
• windows10-64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-17&endday=2019-04-17&tree=all

Comment 13

[Cristina Coroiu [:ccoroiu]]

This is still failing on beta sims: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=9064a77fe72c606c2e0e1d7562ceb8d27da12e39&selectedJob=241146479

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241146479&repo=try&lineNumber=29755

Comment 14

[Tim Nguyen :ntim]

Attachment: Bug 1543696 - Fix browser_ext_themes_experiment.js and browser_ext_themes_warnings.js failures due to extensions.legacy.enabled behaving differently on automation. r=kmag

Comment 15

[Noemi Erli[:noemi_erli]]

With this bug imported in beta-sim, the initial fail seems to pass, but it causes other fail:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241427403&repo=try&lineNumber=18648

Initial fail that passes after importing patch: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241427403&repo=try&lineNumber=17893

Beta-sim push: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=dde31a9413adc353a14745f3aeca2bf460ad3065&selectedJob=241427403

Comment 16

[Tim Nguyen :ntim]

How is the latest version of the patch ?

Comment 17

[Noemi Erli[:noemi_erli]]

I tried the latest patch here: https://hg.mozilla.org/try/rev/008019ed9eb7a436b65e04bba91a5d42c35fd11b

Comment 18

[Tim Nguyen :ntim]

(In reply to Noemi Erli[:noemi_erli] from comment #17)

I tried the latest patch here: https://hg.mozilla.org/try/rev/008019ed9eb7a436b65e04bba91a5d42c35fd11b

Note that this is not the latest patch, the latest patch has a cleanup() function.

Comment 19

[Tim Nguyen :ntim]

Looks like this was done here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=badfe285e2a1047dab68909ba39683ffd0f180ab

Comment 20

[Cosmin Sabou [:CosminS]]

Tim, we were just waiting for the test results before posting it.

Comment 21

[Tim Nguyen :ntim]

(In reply to Tim Nguyen :ntim from comment #19)

Looks like this was done here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=badfe285e2a1047dab68909ba39683ffd0f180ab

Hmm, looks like the failure is still there :/

Comment 23

[Intermittent Failures Robot]

25 failures in 2673 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 25

Platform breakdown:

• linux64: 5
• osx-10-10-shippable: 4
• osx-10-10: 1
• linux32: 2
• windows7-32: 3
• windows10-64: 2
• linux64-shippable: 2
• windows7-32-shippable: 2
• linux32-shippable: 2
• windows10-64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-15&endday=2019-04-21&tree=all

Comment 24

[Intermittent Failures Robot]

22 failures in 307 pushes (0.072 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 22

Platform breakdown:

• linux64: 6
• osx-10-10: 2
• linux32: 2
• windows10-64: 1
• osx-10-10-shippable: 2
• windows7-32: 1
• windows7-32-shippable: 2
• linux64-shippable: 2
• linux32-shippable: 2
• windows10-64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-22&endday=2019-04-22&tree=all

Comment 25

[Intermittent Failures Robot]

21 failures in 499 pushes (0.042 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 21

Platform breakdown:

• linux64: 6
• linux32: 2
• osx-10-10: 2
• windows10-64: 2
• osx-10-10-shippable: 1
• windows7-32: 2
• linux64-shippable: 2
• linux32-shippable: 2
• windows7-32-shippable: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-23&endday=2019-04-23&tree=all

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/352f3ef0d7df
Fix browser_ext_themes_experiment.js and browser_ext_themes_warnings.js failures due to extensions.legacy.enabled behaving differently on automation. r=kmag

Comment 27

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/352f3ef0d7df

Comment 28

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed with yesterday's beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=689c102a69b154eac666dbd70df3a1d7beb60345

Comment 29

[Intermittent Failures Robot]

43 failures in 3031 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• try: 43

Platform breakdown:

• linux64: 12
• linux32: 4
• osx-10-10: 4
• windows10-64: 3
• osx-10-10-shippable: 3
• windows7-32: 3
• windows7-32-shippable: 3
• linux64-shippable: 4
• windows10-64-shippable: 3
• linux32-shippable: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1543696&startday=2019-04-22&endday=2019-04-28&tree=all