Open Bug 1411118 Opened 2 years ago Updated 10 months ago

Intermittent browser/base/content/test/alerts/browser_notification_remove_permission.js | Test timed out -

Categories

(Toolkit :: Notifications and Alerts, defect, P1)

defect

Tracking

()

Tracking Status
firefox58 --- disabled
firefox59 --- disabled

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, leave-open, Whiteboard: [stockwell disabled])

In the last 4 days there have been 33 failures. There are many failures on windows10-64 and also a significant number of failures on windows10-64-stylo-disabled. I've noticed 2 recent failures on Windows 7, too.
The most failures are occurring on the opt build type and the rest on debug and pgo.

Here is an example of a recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=139923497&lineNumber=1894


And a relevant snippet of the log: 

17:01:27     INFO -  3 INFO Entering test bound test_settingsOpen_observer
1881
17:01:27     INFO -  4 INFO Opening a dummy tab so openPreferences=>switchToTabHavingURI doesn't use the blank tab.
1882
17:01:27     INFO -  Buffered messages logged at 16:59:58
1883
17:01:27     INFO -  5 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 174}]
1884
17:01:27     INFO -  6 INFO simulate a notifications-open-settings notification
1885
17:01:27     INFO -  7 INFO TEST-PASS | browser/base/content/test/alerts/browser_notification_open_settings.js | The notification settings tab opened -
1886
17:01:27     INFO -  Buffered messages logged at 17:00:00
1887
17:01:27     INFO -  8 INFO Leaving test bound test_settingsOpen_observer
1888
17:01:27     INFO -  9 INFO Entering test bound test_settingsOpen_button
1889
17:01:27     INFO -  10 INFO Adding notification permission
1890
17:01:27     INFO -  11 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 174}]
1891
17:01:27     INFO -  12 INFO Waiting for notification
1892
17:01:27     INFO -  13 INFO openNotification: showNotification2
1893
17:01:27     INFO -  Buffered messages finished
1894
17:01:27    ERROR -  14 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/alerts/browser_notification_open_settings.js | Test timed out -
1895
17:01:27     INFO -  GECKO(6624) | MEMORY STAT | vsize 1822MB | vsizeMaxContiguous 130444582MB | residentFast 258MB | heapAllocated 79MB
1896
17:01:27     INFO -  15 INFO TEST-OK | browser/base/content/test/alerts/browser_notification_open_settings.js | took 90213ms
1897
17:01:27     INFO -  Not taking screenshot here: see the one that was previously logged
1898
17:01:27    ERROR -  16 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/alerts/browser_notification_open_settings.js | Found a tab after previous test timed out: http://example.org/browser/browser/base/content/test/alerts/file_dom_notifications.html -
1899
17:01:28     INFO -  GECKO(6624) | ++DOCSHELL 000001F54F543000 == 1 [pid = 876] [id = {ab0966b1-4dd3-44e8-aff3-125b0a3392f8}]
1900
17:01:28     INFO -  GECKO(6624) | ++DOMWINDOW == 1 (000001F54F544000) [pid = 876] [serial = 1] [outer = 0000000000000000]

:MattN, can you please take a look?
Flags: needinfo?(MattN+bmo)
Whiteboard: [stockwell needswork]
Currently, this has 153 failures in the last week. Since 29 October, the spike of failures went down, so we're not in danger to disable this.
we are still getting 18 failures/day, that will cross the 200 failures/month rate, hopefully the rate goes down, only time will tell.
Assignee: nobody → MattN+bmo
Status: NEW → ASSIGNED
Flags: needinfo?(MattN+bmo)
There is something really weird going on here… it seems like the first ContentTask.spawn call doesn't work and it seemed like the content-task.js script wasn't being loaded by loadFrameScript. This problem isn't related to the alert service and was caused by some outside change. It's a perma-failure for me locally. The other tests in this directory were probably hitting the same problem. I think this test started failing more when the test before was skipped and the same for that test. :(
This is a Sandboxing issue, not a problem with the test. The content process Sandbox on Windows is denying[1] access to content-task.js so this should be a perma-failure, if it's not then the sandbox is broken.

My mochitest profile only has
> obj-fx-dbg\_tests\testing\mochitest\
on the whitelist but resource://testing-common/content-task.js resolves to
> obj-fx-dbg\_tests\modules\content-task.js

[1] https://dxr.mozilla.org/mozilla-central/rev/2535bad09d720e71a982f3f70dd6925f66ab8ec7/dom/base/nsFrameMessageManager.cpp#1628
This is disabled-recommended, but I'm not disabling since there is recent activity in bug 1415379.
There are fewer failures now - bug 1415379 may have helped - but failures continue.

:MattN -- Any updates?
Flags: needinfo?(MattN+bmo)
30 failures in the last 4 days, a much lower rate, but still high.
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/93df67e671bd
Disable browser/base/content/test/alerts/browser_notification_open_settings.js on windows for frequent failures. r=me, a=testonly
please remember to enable this test when working on it.
Keywords: leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
ok, disabling this test yields the next test failing, basically all of these tests are failing.  :mattn, I am looking to disable all these tests, is there work that can be done to make these tests work?  right now all the tests are failing :(
Summary: Intermittent browser/base/content/test/alerts/browser_notification_open_settings.js | Test timed out - → Intermittent browser/base/content/test/alerts/browser_notification_remove_permission.js | Test timed out -
This bug has failed 83 times in the past 7 days. Failing mostly on Windows 10-64 with a few Windows 7 fails, build types are opt, debug and PGO.

Here is a recent log of the failure:
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=148450190&lineNumber=1996

And here is a part of the log: 
 TEST-INFO | started process screenshot
11:46:23     INFO -  TEST-INFO | screenshot: exit 0
11:46:23     INFO -  Buffered messages logged at 11:44:53
11:46:23     INFO -  3 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 174}]
11:46:23     INFO -  Buffered messages logged at 11:44:54
11:46:23     INFO -  4 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 174}]
11:46:23     INFO -  5 INFO openNotification: showNotification2
11:46:23     INFO -  Buffered messages finished
11:46:23    ERROR -  6 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/alerts/browser_notification_remove_permission.js | Test timed out -
11:46:23     INFO -  GECKO(3604) | MEMORY STAT | vsize 1799MB | vsizeMaxContiguous 131464221MB | residentFast 238MB | heapAllocated 70MB
11:46:23     INFO -  7 INFO TEST-OK | browser/base/content/test/alerts/browser_notification_remove_permission.js | took 90178ms
11:46:23     INFO -  GECKO(3604) | ++DOCSHELL 000001B41CD1E000 == 1 [pid = 8520] [id = {fef5d457-8436-497e-9cbc-7fb9c79087e9}]
11:46:23     INFO -  GECKO(3604) | ++DOMWINDOW == 1 (000001B41AEA5470) [pid = 8520] [serial = 1] [outer = 0000000000000000]
11:46:23     INFO -  8 INFO checking window state
11:46:23     INFO -  9 INFO TEST-START | browser/base/content/test/alerts/browser_notification_replace.js

:MattN: Can you please take a look at this bug? It had quite a spike in the last 7 days even after disabling tests on windows.
Whiteboard: [stockwell disabled] → [stockwell needswork]
See Also: → 1352791, 1227785
(In reply to Matthew N. [:MattN] (PM if requests are blocking you) from comment #15)
> This is a Sandboxing issue, not a problem with the test. The content process
> Sandbox on Windows is denying[1] access to content-task.js so this should be
> a perma-failure, if it's not then the sandbox is broken.
> 
> My mochitest profile only has
> > obj-fx-dbg\_tests\testing\mochitest\
> on the whitelist but resource://testing-common/content-task.js resolves to
> > obj-fx-dbg\_tests\modules\content-task.js
> 
> [1]
> https://dxr.mozilla.org/mozilla-central/rev/
> 2535bad09d720e71a982f3f70dd6925f66ab8ec7/dom/base/nsFrameMessageManager.
> cpp#1628

The test is failing very often in the Windows coverage build too, where sandboxing is disabled.
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c65ae7eb788a
Disable browser/base/content/test/alerts/browser_notification_remove_permission.js on windows for frequent failures. r=me, a=testonly
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
I've informed my manager that all of the alert service b-c tests have been disabled due to intermittent failures but there are currently higher priorities projects.
Assignee: MattN+bmo → nobody
Status: ASSIGNED → NEW
Flags: needinfo?(MattN+bmo)
Moving to p3 because no activity for at least 24 weeks.
Priority: P1 → P3
Mike, dolske told me your team was going to look into alert service issues a while ago. Is that still the case?
Flags: needinfo?(mdeboer)
Priority: P3 → P1
I'm afraid not, we've since moved on with higher priority projects in the world of search. We'll need to find a new owner for this piece.
Flags: needinfo?(mdeboer)
You need to log in before you can comment on or make changes to this bug.