1010415 - Intermittent notification_resend_test.js | mozChromeNotifications: Sending one notification, remove from tray, resend

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Attachment: test screenshot

https://tbpl.mozilla.org/php/getParsedLog.php?id=39665712&tree=B2g-Inbound

b2g_ubuntu64_vm b2g-inbound opt test gaia-integration on 2014-05-14 09:19:40 PDT for push bd5ec747c11e
slave: tst-linux64-spot-617

09:35:01     INFO -  TEST-START | mozChromeNotifications: Sending one notification, remove from tray, resend
09:35:33     INFO -  Screenshot: 
09:35:33     INFO -  TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending one notification, remove from tray, resend
09:35:33     INFO -  TEST-END | mozChromeNotifications: Sending one notification, remove from tray, resend

Comment 1

[Legacy TBPL/Treeherder Robot]

Tomcat
https://tbpl.mozilla.org/php/getParsedLog.php?id=39726232&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-15 03:50:03
revision: 45ee96fe3807
slave: tst-linux64-spot-165

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending no notification, resends none
TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/activity_callee_alert_test.js | Camera app on call Should have only one alert
make: *** [test-integration-test] Error 2
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 2

[Legacy TBPL/Treeherder Robot]

Tomcat
https://tbpl.mozilla.org/php/getParsedLog.php?id=39796987&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-16 01:26:11
revision: 6deed68c2358
slave: tst-linux64-spot-589

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending two notifications, resends two
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 3

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=39870674&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-17 00:06:23
revision: 89964e80a627
slave: tst-linux64-spot-444

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending no notification, resends none
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 4

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=39892676&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-17 18:46:58
revision: 67e42658394e
slave: tst-linux64-spot-717

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending two notifications, resends two
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 5

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=39896511&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-17 20:52:25
revision: 62d1670e37d9
slave: tst-linux64-spot-488

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending one notification, remove from tray, resend
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 6

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=39982660&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-19 17:33:25
revision: 40c010000007
slave: tst-linux64-spot-776

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending two notifications, close one, resends one
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 7

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=40035182&tree=B2g-Inbound
b2g_ubuntu64_vm b2g-inbound opt test gaia-integration on 2014-05-20 12:14:36
revision: 53af735f5916
slave: tst-linux64-spot-555

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending one notification, remove from tray, resend
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 8

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=40055656&tree=B2g-Inbound
b2g_ubuntu64_vm b2g-inbound opt test gaia-integration on 2014-05-20 16:49:27
revision: 31b522e791cf
slave: tst-linux64-spot-010

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending two notifications, resends two
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 9

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=40070428&tree=Mozilla-Inbound
b2g_ubuntu64_vm mozilla-inbound opt test gaia-integration on 2014-05-20 20:56:32
revision: 71da35d73be5
slave: tst-linux64-spot-772

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending two notifications, resends two
make: *** [test-integration-test] Error 1
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 10

[Legacy TBPL/Treeherder Robot]

philor
https://tbpl.mozilla.org/php/getParsedLog.php?id=40440683&tree=Fx-Team
b2g_ubuntu64_vm fx-team opt test gaia-integration on 2014-05-27 00:22:02
revision: 02907d94fc2a
slave: tst-linux64-spot-348

TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/system/test/marionette/notification_resend_test.js | mozChromeNotifications: Sending one notification, remove from tray, resend
TEST-UNEXPECTED-FAIL | undefined | toggle calendar > regular event "before each" hook
TEST-UNEXPECTED-FAIL | undefined | toggle calendar > all day event "before each" hook
TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/communications/contacts/test/marionette/delete_contacts_test.js | Contacts > Delete  > Edit menu is not visible on search mode
make: *** [test-integration-test] Error 4
Return code: 2
Tests exited with return code 2: harness failures
# TBPL FAILURE #

Comment 11

[Kevin Grandon :kgrandon]

Disabling this test so we can turn on Gij on TBPL: https://github.com/mozilla-b2g/gaia/commit/a12ea45a5aca843105ea3188df146d48e84cf673

Adding leave-open to prevent auto-closing of this bug by sheriffs.

Here is a recent run failure of this test: https://tbpl.mozilla.org/php/getParsedLog.php?id=43553081&tree=B2g-Inbound

Ni? on Alex for awareness.

Comment 12

[Kevin Grandon :kgrandon]

Originally I had thought that the "Error: Polling socket recv() timeout!" error was framework related, but I've seen this same test fail multiple times with that, and other tests don't seem to fail so much. So we should investigate to see what that means.

Comment 13

[:gerard-majax]

I don't understand, is it an intermittent test or a permanent fail ?!

Comment 14

[Kevin Grandon :kgrandon]

(In reply to Alexandre LISSY :gerard-majax from comment #13)
> I don't understand, is it an intermittent test or a permanent fail ?!

This is an intermittent test. We currently have way too many intermittent tests and need to get down to about 1% to re-enable tests on TBPL. In order to do so we will be disabling a lot of tests unfortunately. After we are back up and running on TBPL I will be more than happy to help people debug these issues.

Regarding this specific test, I'm still not sure if it's an issue with the test, or a problem with the harness. We'll need to do a bit of investigation to figure that out.

Comment 15

[:gerard-majax]

(In reply to Kevin Grandon :kgrandon from comment #14)
> (In reply to Alexandre LISSY :gerard-majax from comment #13)
> > I don't understand, is it an intermittent test or a permanent fail ?!
> 
> This is an intermittent test. We currently have way too many intermittent
> tests and need to get down to about 1% to re-enable tests on TBPL. In order
> to do so we will be disabling a lot of tests unfortunately. After we are
> back up and running on TBPL I will be more than happy to help people debug
> these issues.
> 
> Regarding this specific test, I'm still not sure if it's an issue with the
> test, or a problem with the harness. We'll need to do a bit of investigation
> to figure that out.

Thanks. I spent some time looking at the test, I found some weirdness. I have no idea if it could explain the issue, but I filed bug 1037371 and I have a PR that makes use of as much as possible of addEventListener() inside executeAsyncScript() instead of mixing executeScript() and executeAsyncScript().

We'll see if it helps.

Meanwhile, I was not able to reproduce the issue locally.

Comment 16

[Michael Henretty [:mikehenrty][:mhenretty]]

Attachment: [Gaia PR] re-enable

Let's see if this test is still intermittent in tbpl.

Comment 17

[Michael Henretty [:mikehenrty][:mhenretty]]

Rebased: https://tbpl.mozilla.org/?rev=2ac05efeb7551dee362c8252014d1cd3218c2874&tree=Gaia-Try

Comment 18

[:gerard-majax]

Michael, it seems we now have a permafail. At least if we reproduce locally, we can investigate ... :)

https://tbpl.mozilla.org/?rev=99315d2bd079b7d14ae7b40b4edb6d5fbe45f4a4&tree=Gaia-Try&showall=1

Comment 19

[:gerard-majax]

  1) mozChromeNotifications: Sending one notification, remove from tray, resend:
     
  JavaScriptError: (17) NotFoundError: Node was not found
  Remote Stack:
  execute_async_script @undefined, line undefined
  inline javascript, line 72
  src: "      nodes = container.removeChild(nodes[0]).querySelectorAll(selector);"
      at Error.MarionetteError (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/error.js:67:13)
      at Object.Client._handleCallback (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:476:19)
      at /home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:510:21
      at TcpSync.send (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:153:10)
      at Object.send (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:457:36)
      at Object.Client._sendCommand (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:503:19)
      at Object._executeScript (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:1468:19)
      at Object.executeAsyncScript (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:1249:19)
      at Context.<anonymous> (/home/alex/codaz/Mozilla/b2g/gaia/apps/system/test/marionette/notification_resend_test.js:192:20)
      at Test.Runnable.run (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runnable.js:196:15)
      at Runner.runTest (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:373:10)
      at /home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:451:12
      at next (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:298:14)
      at /home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:308:7
      at next (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:246:23)
      at /home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runner.js:270:7
      at done (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)
      at /home/alex/codaz/Mozilla/b2g/gaia/node_modules/mocha/lib/runnable.js:199:9
      at Object.executeHook (/home/alex/codaz/Mozilla/b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:369:18)
      at process._tickCallback (node.js:419:13)

Comment 20

[:gerard-majax]

Attachment: Fix and re-enable test

Michael, this simple change does fix the test locally :)

Comment 21

[:gerard-majax]

The matching try is https://tbpl.mozilla.org/?rev=1bc1d74670c8e1f8a2da0df754d660ca8da9afb6&tree=Gaia-Try

Comment 22

[:gerard-majax]

So the gaia try with ~50 retriggers expose no failure at all of the notification test, but there are other failures.

Comment 23

[Michael Henretty [:mikehenrty][:mhenretty]]

master: https://github.com/mozilla-b2g/gaia/commit/c5ad0797cc83b9a6e6519e19fa8f401c021979bb

I'm going to look into some of those other notification errors.

Comment 24

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.