Closed Bug 1007519 Opened 8 years ago Closed 7 years ago

Perma failing test, toggle calendar > regular event disable calendar day view


(Firefox OS Graveyard :: Gaia::Calendar, defect)

Not set


(Not tracked)



(Reporter: kgrandon, Assigned: gaye)




(2 files)

Perhaps related to a date change? Instead of closing the tree, let's disable this test and investigate. I don't think it's a gaia change as it started failing across an unrelated cset and pulll requests. Stack: 

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:682:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)

at Object.View.waitForDisplay (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/lib/views/view.js:43:24)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/toggle_calendar_test.js:76:23)

at callFn (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:223:21)

at (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:216:7)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:374:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:452:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:299:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:309:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:247:23)

at Object._onImmediate (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:276:5)

at processImmediate [as _immediateCallback] (timers.js:330:15)
Assignee: nobody → mmedeiros
Target Milestone: --- → 2.0 S3 (6june)
Duplicate of this bug: 1006322
besides the fact that the new "settings drawer" have a different DOM structure it also had some really hard to reproduce race condition going on the create event page (don't know how other tests that creates events was not failing as well), which now should be fixed.

it was hard but I finally found out what was causing the race condition and made sure it's stable (executed more than 300 tests without failing).

the options on the calendar <select> are loaded asynchronously, so in some cases we where trying to select an <option> that did not exist yet.. only happened 1-3% of the time, so that explains why we did not see it happen so far.
Attachment #8435296 - Flags: review?(gaye)
So, this bug is marked as duping bug 1006322, which in turn is marked as duping bug 1007097.  That bug is the only thing preventing us from unhiding gaia-integration tests on TBPL.  Can we prioritize a fix here?
Alternative is to disable the test on TBPL (we have some others already).
Lets disable this for now... Unhiding is more important
Hidden in lets re-enable GI... I removed this bug as a blocker to turning GI on but this still need to be resolved (fixed) eventually.
No longer blocks: 960072
Comment on attachment 8435296 [details] [review]
Link to Github pull-request:

LGTM. Thanks miller!
Attachment #8435296 - Flags: review?(gaye) → review+
landed into master:
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: 2.0 S3 (6june) → 2.0 S4 (20june)
disabled again in
master: a5cdbc08d023859f3f5a6b6163434a84f262ad85
damn.. I executed over 300 times each toggle calendar test before landing.. never seen it timeout once. only error that I was getting was the "Cannot call method 'scriptWith' of null", and the last 100+ times it passed without any error (passed 50 times) (passed 50 times)

maybe in some really hard to reproduce case the click that toggles the calendar visibility is not happening on the proper element; that would explain the timeout on the enable calendar test.

<rant>calendar integration tests improved a lot the past few months but it still drives me crazy when we have intermittent or perma red failures. So hard to debug and reproduce in some cases and not seeing the tests catching that many bugs anyway, so in my opinion I still think they cause way more headaches than they avoid... I don't think the effort is justified, specially since it takes so long to run on CI server and a good part of the tests are not reliable. When an "alarm" starts going off for no reason people learn to ignore it.</rant>
Yes, I agree, IMO integration tests should test the big cases using scenarios, especially cross-app stuff like activities or IAC, and we should not have a lot of them.

Here I saw the issue in 3 builds in a row, that's why I disabled the test. Maybe it was a moment where Travis' VM was especially loaded.
Target Milestone: 2.0 S4 (20june) → 2.0 S5 (4july)
Assignee: mmedeiros → nobody
Target Milestone: 2.0 S5 (4july) → ---
Duplicate of this bug: 1007097
Blocks: 1095712
Assignee: nobody → gaye
Closed: 8 years ago7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.