1007519 - Perma failing test, toggle calendar > regular event disable calendar day view

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Calendar, defect)

Description

[Kevin Grandon :kgrandon]

Perhaps related to a date change? Instead of closing the tree, let's disable this test and investigate. I don't think it's a gaia change as it started failing across an unrelated cset and pulll requests. Stack: 

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:682:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)

at Object.View.waitForDisplay (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/lib/views/view.js:43:24)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/toggle_calendar_test.js:76:23)

at callFn (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:223:21)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:216:7)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:374:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:452:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:299:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:309:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:247:23)

at Object._onImmediate (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:276:5)

at processImmediate [as _immediateCallback] (timers.js:330:15)

Comment 1

[Kevin Grandon :kgrandon]

Attachment: Screenshot at point of failure

Comment 2

[Kevin Grandon :kgrandon]

Test disabled in: https://github.com/mozilla-b2g/gaia/commit/38b81f8e017df32479cbae07f7ee415a78984d8b

Comment 4

[Miller Medeiros [:millermedeiros]]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20102

besides the fact that the new "settings drawer" have a different DOM structure it also had some really hard to reproduce race condition going on the create event page (don't know how other tests that creates events was not failing as well), which now should be fixed.

it was hard but I finally found out what was causing the race condition and made sure it's stable (executed more than 300 tests without failing).

the options on the calendar <select> are loaded asynchronously, so in some cases we where trying to select an <option> that did not exist yet.. only happened 1-3% of the time, so that explains why we did not see it happen so far.

Comment 5

[Jonathan Griffin (:jgriffin)]

So, this bug is marked as duping bug 1006322, which in turn is marked as duping bug 1007097.  That bug is the only thing preventing us from unhiding gaia-integration tests on TBPL.  Can we prioritize a fix here?

Comment 6

[Julien Wajsberg [:julienw]]

Alternative is to disable the test on TBPL (we have some others already).

Comment 7

[James Lal [:lightsofapollo]]

Lets disable this for now... Unhiding is more important

Comment 8

[James Lal [:lightsofapollo]]

Hidden in https://bugzilla.mozilla.org/show_bug.cgi?id=1023088 lets re-enable GI... I removed this bug as a blocker to turning GI on but this still need to be resolved (fixed) eventually.

Comment 9

[Gareth Aye [:gaye] (back from PTO)]

Comment on attachment 8435296 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20102

LGTM. Thanks miller!

Comment 10

[Miller Medeiros [:millermedeiros]]

landed into master: https://github.com/mozilla-b2g/gaia/commit/67b305497f3ac853c2434bde5969c4a629fca98b

Comment 11

[Julien Wajsberg [:julienw]]

https://travis-ci.org/mozilla-b2g/gaia/jobs/27372822
https://travis-ci.org/mozilla-b2g/gaia/jobs/27374219
https://travis-ci.org/mozilla-b2g/gaia/jobs/27374582

looks like it's still there.

Comment 12

[Julien Wajsberg [:julienw]]

disabled again in
master: a5cdbc08d023859f3f5a6b6163434a84f262ad85

Comment 13

[Miller Medeiros [:millermedeiros]]

damn.. I executed over 300 times each toggle calendar test before landing.. never seen it timeout once. only error that I was getting was the "Cannot call method 'scriptWith' of null", and the last 100+ times it passed without any error

https://travis-ci.org/millermedeiros/gaia/builds/26891442 (passed 50 times)
https://travis-ci.org/millermedeiros/gaia/builds/26871153 (passed 50 times)

maybe in some really hard to reproduce case the click that toggles the calendar visibility is not happening on the proper element; that would explain the timeout on the enable calendar test.

<rant>calendar integration tests improved a lot the past few months but it still drives me crazy when we have intermittent or perma red failures. So hard to debug and reproduce in some cases and not seeing the tests catching that many bugs anyway, so in my opinion I still think they cause way more headaches than they avoid... I don't think the effort is justified, specially since it takes so long to run on CI server and a good part of the tests are not reliable. When an "alarm" starts going off for no reason people learn to ignore it.</rant>

Comment 14

[Julien Wajsberg [:julienw]]

Yes, I agree, IMO integration tests should test the big cases using scenarios, especially cross-app stuff like activities or IAC, and we should not have a lot of them.

Here I saw the issue in 3 builds in a row, that's why I disabled the test. Maybe it was a moment where Travis' VM was especially loaded.

Comment 16

[Kevin Grandon :kgrandon]

New failures here on TBPL, so disabling there as well:
https://tbpl.mozilla.org/php/getParsedLog.php?id=43742342&tree=Gaia-Try#error3
https://tbpl.mozilla.org/php/getParsedLog.php?id=43742129&tree=Gaia-Try#error3
https://tbpl.mozilla.org/php/getParsedLog.php?id=43742583&full=1&branch=gaia-try

Disabled in: https://github.com/mozilla-b2g/gaia/commit/8ac73f82c61638545c1110336fd10cf743a1e221