1277669 - browser_BrowserUITelemetry_defaults.js is going to permafail when Gecko 49 merges to Aurora

Status: RESOLVED FIXED

(Firefox :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Attachment: test screenshot

[Tracking Requested - why for this release]: Merge day permafail when Gecko 49 hits Aurora.

Looks like this only affects Aurora, so I guess the DevEdition branding throws the test expectations off? Merge day is next Monday, so please investigate ASAP.

https://treeherder.mozilla.org/logviewer.html#?job_id=21773946&repo=try#L11556

20:57:10     INFO -  303 INFO TEST-START | browser/modules/test/browser_BrowserUITelemetry_defaults.js
20:57:10     INFO -  TEST-INFO | started process screentopng
20:57:12     INFO -  TEST-INFO | screentopng: exit 0
20:57:12     INFO -  304 INFO checking window state
20:57:12     INFO -  305 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser_BrowserUITelemetry_defaults.js | No other test should have left CUI in a dirty state. - false == true - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser_BrowserUITelemetry_defaults.js :: test :: line 13
20:57:12     INFO -  Stack trace:
20:57:12     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser_BrowserUITelemetry_defaults.js:test:13
20:57:12     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:810
20:57:12     INFO -  chrome://mochikit/content/browser-test.js:Tester.prototype.nextTest</<:711
20:57:12     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:741
20:57:12     INFO -  Not taking screenshot here: see the one that was previously logged
20:57:12     INFO -  306 INFO TEST-UNEXPECTED-FAIL | browser/modules/test/browser_BrowserUITelemetry_defaults.js | ["developer-button"] deepEqual [] - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser_BrowserUITelemetry_defaults.js :: test :: line 18
20:57:12     INFO -  Stack trace:
20:57:12     INFO -  chrome://mochitests/content/browser/browser/modules/test/browser_BrowserUITelemetry_defaults.js:test:18
20:57:12     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:810
20:57:12     INFO -  chrome://mochikit/content/browser-test.js:Tester.prototype.nextTest</<:711
20:57:12     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:741
20:57:12     INFO -  307 INFO TEST-PASS | browser/modules/test/browser_BrowserUITelemetry_defaults.js | [] deepEqual [] -
20:57:12     INFO -  308 INFO TEST-PASS | browser/modules/test/browser_BrowserUITelemetry_defaults.js | ["social-share-button"] deepEqual ["social-share-button"] -
20:57:12     INFO -  309 INFO TEST-PASS | browser/modules/test/browser_BrowserUITelemetry_defaults.js | [1] deepEqual [1] -
20:57:12     INFO -  MEMORY STAT | vsize 645MB | residentFast 245MB | heapAllocated 78MB
20:57:12     INFO -  310 INFO TEST-OK | browser/modules/test/browser_BrowserUITelemetry_defaults.js | took 846ms

Comment 1

[Marcia Knous [:marcia]]

Tracking this permafail for 49+ - Mark can you please take this bug and try to resolve prior to the merge? Thanks.

Comment 2

[Mark Hammond [:markh] [:mhammond]]

I'll do my best. FTR though, a sub-optimial-but-ok fix would be to disable it in Aurora only - it will pass again in beta.

Comment 3

[Intermittent Failures Robot]

74 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 74

Platform breakdown:
* windows7-32: 24
* linux64: 18
* osx-10-10: 15
* linux32: 13
* windowsxp: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1277669&startday=2016-05-30&endday=2016-06-05&tree=all

Comment 4

[Mark Hammond [:markh] [:mhammond]]

Attachment: Bug 1277669 - skip a couple of browserUITelemetry tests in dev edition.

Review commit: https://reviewboard.mozilla.org/r/57854/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/57854/

Comment 5

[Mark Hammond [:markh] [:mhammond]]

Gijs has marked himself as not taking review requests until June 6, which is today - hopefully I'll be able to flag him in a couple of hours ;)

Comment 6

[:Gijs (he/him)]

Comment on attachment 8760144 [details]
Bug 1277669 - skip a couple of browserUITelemetry tests in dev edition.

https://reviewboard.mozilla.org/r/57854/#review54682

I think bug 1278176 is likely a dupe of bug 1262447, but we can unify that when we get there. For now, this will make sure we don't bust aurora.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d1626e5f33af
skip a couple of browserUITelemetry tests in dev edition. r=Gijs

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d1626e5f33af

Comment 9

[Liz Henry (:lizzard) (relman/hg->git project)]

Does this no longer affect 49? In other words is the test just running on aurora but not on beta?  Just cleaning up tracked bugs and wanted to make sure. Thanks.

Comment 10

[Mark Hammond [:markh] [:mhammond]]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #9)
> Does this no longer affect 49? In other words is the test just running on
> aurora but not on beta?  Just cleaning up tracked bugs and wanted to make
> sure. Thanks.

This push skipped some tests on Aurora, but they work correctly in Nightly, Beta and Release. I don't think there's any further action needed here.