1493148 - Perma multiple failures in e.g. extensions/cookie/test/unit/test_cookies_thirdparty.js when Gecko 64 merges to Beta on 2018-10-15

Status: VERIFIED FIXED

(Firefox :: Protections UI, defect, P1)

Description

[Eliza Balazs [:ebalazs_]]

[Tracking Requested - why for this release]:

Central as Beta: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed,busted,exception,usercancel,runnable&revision=8599c085c31bc9dc5201acaf9ec822429e5890a8&selectedJob=200723985

Failure logs:
https://treeherder.mozilla.org/logviewer.html#?job_id=200723985&repo=try&lineNumber=1846
https://treeherder.mozilla.org/logviewer.html#?job_id=200728186&repo=try&lineNumber=6543
https://treeherder.mozilla.org/logviewer.html#?job_id=200723969&repo=try&lineNumber=13923
https://treeherder.mozilla.org/logviewer.html#?job_id=200724972&repo=try&lineNumber=23991
https://treeherder.mozilla.org/logviewer.html#?job_id=200725178&repo=try&lineNumber=14781
https://treeherder.mozilla.org/logviewer.html#?job_id=200723969&repo=try&lineNumber=13923

This is a regression from https://bugzilla.mozilla.org/show_bug.cgi?id=1491061

Comment 1

[(no longer active)]

I can reproduce.

Comment 2

[(no longer active)]

Well, this is embarrassing!

Now that I look at this again, it's so obvious that it's a bit funny!  Part 3 of bug 1491061 was actually only necessary before part 4, since in part 4 I updated the Gecko checks to match the default values of the content blocking preferences, whatever they may be.  So with part 4, part 3 actually makes all of the tests that it touches now be wrong, since the prefs that we're setting in those tests will be inconsistent e.g. on beta!  This test failure, plus tons of others in the push in comment 0 is a result of that.

The fix is simple: back out part 3 of bug 1491061.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/13c9130747de
Backout changeset 7a1ee0e4b6d7 (bug 1491061 part 3)

Comment 4

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/13c9130747de

Comment 5

[Andreea Pavel [:apavel]]

There is a failure missing from the first comment: 

https://treeherder.mozilla.org/logviewer.html#?job_id=200985482&repo=try&lineNumber=4223

13:29:08     INFO - TEST-START | toolkit/components/antitracking/test/browser/browser_blockingCookies.js
13:29:08     INFO - TEST-INFO | started process screenshot
13:29:08     INFO - TEST-INFO | screenshot: exit 0
13:29:08     INFO - Buffered messages logged at 13:29:08
13:29:08     INFO - Entering test bound 
13:29:08     INFO - Starting blocking cookieBehavior (4) and blocking contentBlocking and  contentBlocking UI and  contentBlocking third-party cookies UI without allow list test Set/Get Cookies running in a normal window  with iframe sandbox set to null
13:29:08     INFO - Creating a new tab
13:29:08     INFO - Creating a 3rd party content
13:29:08     INFO - Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "http://example.net/browser/toolkit/components/antitracking/test/browser/page.html" line: 0}]
13:29:08     INFO - Sending code to the 3rd party content
13:29:08     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | No cookies for me - true == true - 
13:29:08     INFO - Buffered messages finished
13:29:08     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | No cookies for me - false == true - 
13:29:08     INFO - Stack trace:
13:29:08     INFO - resource://testing-common/content-task.js line 59 > eval:msg:22
13:29:08     INFO - Not taking screenshot here: see the one that was previously logged
13:29:08     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | We should not have cookies - false == true - 
13:29:08     INFO - Stack trace:
13:29:08     INFO - resource://testing-common/content-task.js line 59 > eval:msg:22
13:29:08     INFO - Not taking screenshot here: see the one that was previously logged
13:29:08     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | We should not have cookies - false == true - 
13:29:08     INFO - Stack trace:
13:29:08     INFO - resource://testing-common/content-task.js line 59 > eval:msg:22
13:29:08     INFO - Not taking screenshot here: see the one that was previously logged
13:29:08     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Still no cookies for me - false == true - 
13:29:08     INFO - Stack trace:
13:29:08     INFO - resource://testing-common/content-task.js line 59 > eval:msg:22
13:29:08     INFO - Not taking screenshot here: see the one that was previously logged
13:29:08     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Checking cookie blocking notifications - Got false, expected true
13:29:08     INFO - Stack trace:
13:29:08     INFO - chrome://mochikit/content/browser-test.js:test_is:1304
13:29:08     INFO - chrome://mochitests/content/browser/toolkit/components/antitracking/test/browser/head.js:_createTask/<:519
13:29:08     INFO - Removing the tab
13:29:08     INFO - Leaving test bound 
13:29:08     INFO - Entering test bound 
13:29:08     INFO - Cleaning up.
13:29:08     INFO - Leaving test bound 
13:29:08     INFO - Entering test bound 
13:29:08     INFO - Starting non-blocking cookieBehavior (0) and blocking contentBlocking and  contentBlocking UI and  contentBlocking third-party cookies UI without allow list test Set/Get Cookies running in a normal window  with iframe sandbox set to null
13:29:09     INFO - GECKO(1700) | JavaScript error: resource://gre/modules/WebProgressChild.jsm, line 58: TypeError: this.mm.content is null; can't access its "document" property
13:29:09     INFO - Console message: [JavaScript Error: "TypeError: this.mm.content is null; can't access its "document" property" {file: "resource://gre/modules/WebProgressChild.jsm" line: 58}]
13:29:09     INFO - Creating a new tab
13:29:09     INFO - Creating a 3rd party content
13:29:09     INFO - Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "http://example.net/browser/toolkit/components/antitracking/test/browser/page.html" line: 0}]
13:29:09     INFO - Sending code to the 3rd party content
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | No cookies for me - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | We should not have cookies - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Some cookies for me - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Some cookies for me - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | We should have cookies - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Some Cookies for me - true == true - 
13:29:09     INFO - TEST-PASS | toolkit/components/antitracking/test/browser/browser_blockingCookies.js | Checking cookie blocking notifications - 
13:29:09     INFO - Removing the tab

Ehsan: this is still occurring on today's beta simulations which means it's not fixed. Can you please take a look?

Comment 6

[Andreea Pavel [:apavel]]

Note: the failures mentioned by Eliza in the first comment no longer appear, just the ones in Comment 5.

Comment 7

[Intermittent Failures Robot]

2 failures in 2902 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* try: 2

Platform breakdown:
* osx-10-10: 1
* windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1493148&startday=2018-09-17&endday=2018-09-23&tree=all

Comment 8

[(no longer active)]

I would always appreciate filing new bugs for remaining issues instead of commenting on old bugs with patches landed.  Thanks!

Comment 9

[(no longer active)]

Filed bug 1493682 for comment 5.

Comment 10

[Eliza Balazs [:ebalazs_]]

Ehsan: Noted and thanks for looking into this.

Comment 11

[Eliza Balazs [:ebalazs_]]

According to Comment 9, a new bug was created for the remaining issue so I am closing this as Resolved Fixed.

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Verified fixed with yesterday's beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed,busted,exception,retry,runnable&revision=fffeabd4033a77e2fcf57c309aac91f63e1ba9f2&selectedJob=203419728