1244936 - (e10s bc1 permanent) TEST-UNEXPECTED-FAIL | browser/base/content/test/alerts/browser_notification_close.js | leaked 2 window(s) until shutdown [url = http://example.org/browser/browser/base/content/test/alerts/file_dom_notifications.html]

Status: RESOLVED FIXED

(Toolkit Graveyard :: Notifications and Alerts, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://treeherder.mozilla.org/logviewer.html#?job_id=20857820&repo=mozilla-inbound

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I think this is going to take a while to fix as we are finally running e10s properly on taskcluster.  Many jobs are green, but we have a few to fix, including this one.  We will need to not run this chunk, or hide it by default until it can be fixed.

Comment 2

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I ended up hiding the bc1 chunk on tc-m-e10s until this can get sorted out. We can unhide once this gets greened up.

Comment 3

[Armen [:armenzg]]

Thanks and my apologies. This was my mistake. I only focused on disabled e10s mochitest-plain

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

here is a link to alog from a try push:
https://public-artifacts.taskcluster.net/Dv0tClU5TH-e_aMIrYEHqQ/0/public/logs/live_backing.log

this is a perma fail, you can see the trend here:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&selectedJob=16412567&filter-searchStr=bc1%20e10s

:jaws, can you help us figure out some next steps on how to fix this bug?

Comment 5

[Intermittent Failures Robot]

16 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 12
* mozilla-inbound: 4

Platform breakdown:
* linux64: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1244936&startday=2016-02-01&endday=2016-02-07&tree=all

Comment 6

[Armen [:armenzg]]

I changed the summary as this is not perma failing.
I don't see it happening much on try either.
https://treeherder.mozilla.org/#/jobs?repo=try&filter-searchStr=tc%20browser-chrome%20e10s%20M(bc1)&group_state=expanded&exclusion_profile=false&fromchange=b2e66fec77c6

Comment 7

[Armen [:armenzg]]

Well well, it has moved to bc4 (yes, it is a permanent issue):
https://treeherder.mozilla.org/#/jobs?repo=try&filter-searchStr=tc%20browser-chrome%20e10s%20m(bc4)&group_state=expanded&exclusion_profile=false&fromchange=b2e66fec77c6&selectedJob=16498244

Comment 8

[Armen [:armenzg]]

jaws: any suggestions on what to try?

Steps to reproduce (except the last command which I paste below):
https://pastebin.mozilla.org/8859022

Unfortunately, the test passes locally when:
* I test it directly - browser/base/content/test/alerts/browser_notification_close.js  
* I test it as a directory - browser/base/content/test/alerts

Filed bug 1246974 for lack of screenshots.

/home/worker/workspace/build/venv/bin/python -u /home/worker/workspace/build/tests/mochitest/runtests.py --e10s --appname=/home/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/TmJr0FB3QeWhpq1z1WN4zg/artifacts/public/build/target.crashreporter-symbols.zip --certificate-path=tests/certs --setpref=webgl.force-enabled=true --quiet --log-raw=/home/worker/workspace/build/blobber_upload_dir/browser-chrome-chunked_raw.log --log-errorsummary=/home/worker/workspace/build/blobber_upload_dir/browser-chrome-chunked_errorsummary.log --use-test-media-devices --screenshot-on-fail --browser-chrome --chunk-by-runtime browser/base/content/test/alerts/browser_notification_close.js

Comment 9

[Armen [:armenzg]]

jaws: did you have any chance to look at this?

This is one of the last issues before every job running on TaskCluster is green.
This has been a multi-quarter project and we would love to get your assitance.

Comment 10

[Armen [:armenzg]]

jaws: I forgot to mention, if you don't have cycles at the moment please let us know who else could give a hand in here.

Comment 11

[Jared Wein [:jaws] (please needinfo? me)]

Kit, this is failing in nsAlertService.cpp first at line 201 and next at line 179,

> 201 // nsIAlertsDoNotDisturb
> 201 NS_IMETHODIMP nsAlertsService::GetManualDoNotDisturb(bool* aRetVal)
> 201 {
> 197 #ifdef MOZ_WIDGET_ANDROID
> 198   return NS_ERROR_NOT_IMPLEMENTED;
> 199 #else
> 200   nsCOMPtr<nsIAlertsDoNotDisturb> alertsDND(GetDNDBackend());
> 201   NS_ENSURE_TRUE(alertsDND, NS_ERROR_NOT_IMPLEMENTED);
> 202   return alertsDND->GetManualDoNotDisturb(aRetVal);
> 203 #endif
> 204 }

> 175 nsresult rv;
> 176 // Try the system notification service.
> 177 if (mBackend) {
> 178   rv = mBackend->CloseAlert(aAlertName, aPrincipal);
> 179   if (NS_WARN_IF(NS_FAILED(rv))) {
> 180     // If the system backend failed to close the alert, fall back to XUL for
> 181     // future alerts.
> 182     mBackend = nullptr;
> 183   }
> 184 } else {

Do you know why we're getting an issue with DoNotDisturb on Linux?

Comment 12

[Jared Wein [:jaws] (please needinfo? me)]

I'm not sure at which point during the test it starts to hang. The logs posted don't have the individual results of each test condition (is(), ok(), etc). Armen, is it possible to enable those for this test?

browser_notification_do_not_disturb.js does a try/catch to make sure that DND is implemented or it skips the test. We shouldn't need to do that here since DND isn't referenced by this test.

Also, pertaining to comment #11, https://hg.mozilla.org/integration/fx-team/rev/3a496a28e22f mentions that Ubuntu Unity has incomplete support for libnotify and we fall back to the XUL notifications. Matt, you reviewed the changes in that patch. Does anything come to mind here for you?

Comment 13

[Lina Butler [:lina]]

I think the failure on line 201 happens because the test machines use libnotify, but we only support DND for XUL alerts. Line 179 is bug 1227730. I have a sneaking suspicion that's behind the leak, since it goes away if I run the test with that patch applied.

FWIW, if I run `mach mochitest --e10s browser/base/content/test/alerts/browser_notification_close.js` under Unity (where we use XUL alerts), I don't see the errors or leaks.

Comment 14

[Armen [:armenzg]]

(In reply to Jared Wein [:jaws] (please needinfo? me) from comment #12)
> I'm not sure at which point during the test it starts to hang. The logs
> posted don't have the individual results of each test condition (is(), ok(),
> etc). Armen, is it possible to enable those for this test?

Hi jaws, I'm sorry. What do you want me to enable? Some sort of debugging info?


FTR, this is back to bc1 instead of bc4. Here's the updated link [1]. It is hidden on treeherder (i.e. exclusion_profile=false).

Here's one of the logs [2]

[1] https://treeherder.mozilla.org/#/jobs?repo=try&filter-searchStr=tc%20browser-chrome%20e10s%20m%28bc1%29&group_state=expanded&fromchange=b2e66fec77c6&selectedJob=16498244&exclusion_profile=false
[2] https://public-artifacts.taskcluster.net/G9z9Mw9JQLuUKEzH-02QHQ/0/public/logs/live_backing.log

Comment 15

[Jared Wein [:jaws] (please needinfo? me)]

For tests that are failing, we should call SimpleTest.requestCompleteLog(); in the test, because right now the treeherder logs aren't as useful since it is only showing when the test starts and finishes, nothing in between.

Comment 16

[Armen [:armenzg]]

In the following push [1], I've added a bunch of SimpleTest.requestCompleteLog(); to tests that have given me trouble [2] including browser_notification_close.js.
To my untrainded eye, I don't see any more output.
Have I done it correctly? See link [2]
I see the test keeps on failing (as expected) [3]


One data point observation, I've seen browser_notification_tab_switching.js fail intermittently and I noticed it runs just two tests before browser_notification_close.js.
I've collected in [4] the speed at which it run and the memory allocation. In case it gives anything useful.
I doubt it is relevant, however, it caught my eye. I would have expected the test would run slower under docker.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=d9abb4499cbb
[2] https://hg.mozilla.org/try/rev/180e4ec5b3ff
[3] https://public-artifacts.taskcluster.net/IF1ddAIiTYK-7HkC3Qth6Q/0/public/logs/live_backing.log
[4]
TC
> 00:55:58     INFO -  MEMORY STAT | vsize 1080MB | residentFast 290MB | heapAllocated 90MB
> 00:55:58     INFO -  44 INFO TEST-OK | browser/base/content/test/alerts/browser_notification_tab_switching.js | took 2876ms
BB
> 05:12:57     INFO -  MEMORY STAT | vsize 719MB | residentFast 243MB | heapAllocated 75MB
> 05:12:57     INFO -  13 INFO TEST-OK | browser/base/content/test/alerts/browser_notification_tab_switching.js | took 5682ms

Comment 17

[Armen [:armenzg]]

I don't know if it helps or if it becomes more of a distraction, when I disable browser_notification_close.js then browser_notification_open_settings.js starts intermittently failing (The tests move to bc4 instead of bc1; I think).
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1dca45d3b4dd&filter-searchStr=e10s%20browser-chrome

When I disable this second test (browser_notification_open_settings.js) as well, it seems that a third test (browser_notification_remove_permission.js) *might* start failing (I'm still re-triggering to know)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a64ceebb39d4&filter-searchStr=e10s%20browser-chrome&group_state=expanded

Comment 18

[Intermittent Failures Robot]

11 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 11

Platform breakdown:
* linux64: 11

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1244936&startday=2016-02-08&endday=2016-02-14&tree=all

Comment 19

[Armen [:armenzg]]

kits' patch might be fixing this issue as well (bc4 e10s shows green):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=80126944609b&group_state=expanded&filter-searchStr=tc%20desktop

Comment 20

[Armen [:armenzg]]

It gets fixed by bug 1227730.

Comment 22

[Lina Butler [:lina]]

Let's leave this one open until bug 1227730 lands, so that we can star intermittents.

Comment 23

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=08b26af80390

Comment 24

[Intermittent Failures Robot]

13 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 6
* mozilla-inbound: 6
* mozilla-central: 1

Platform breakdown:
* linux64: 13

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1244936&startday=2016-02-22&endday=2016-02-28&tree=all

Comment 25

[Armen [:armenzg]]

That's a weekly summary.
There are no more reports since the 25th.