Closed Bug 1007519 Opened 6 years ago Closed 5 years ago

Perma failing test, toggle calendar > regular event disable calendar day view

Categories

(Firefox OS Graveyard :: Gaia::Calendar, defect)

x86
macOS
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kgrandon, Assigned: gaye)

References

Details

Attachments

(2 files)

Perhaps related to a date change? Instead of closing the tree, let's disable this test and investigate. I don't think it's a gaia change as it started failing across an unrelated cset and pulll requests. Stack: 

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:682:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)

at Object.View.waitForDisplay (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/lib/views/view.js:43:24)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/calendar/test/marionette/toggle_calendar_test.js:76:23)

at callFn (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:223:21)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:216:7)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:374:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:452:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:299:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:309:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:247:23)

at Object._onImmediate (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:276:5)

at processImmediate [as _immediateCallback] (timers.js:330:15)
Assignee: nobody → mmedeiros
Target Milestone: --- → 2.0 S3 (6june)
Duplicate of this bug: 1006322
besides the fact that the new "settings drawer" have a different DOM structure it also had some really hard to reproduce race condition going on the create event page (don't know how other tests that creates events was not failing as well), which now should be fixed.

it was hard but I finally found out what was causing the race condition and made sure it's stable (executed more than 300 tests without failing).

the options on the calendar <select> are loaded asynchronously, so in some cases we where trying to select an <option> that did not exist yet.. only happened 1-3% of the time, so that explains why we did not see it happen so far.
Attachment #8435296 - Flags: review?(gaye)
So, this bug is marked as duping bug 1006322, which in turn is marked as duping bug 1007097.  That bug is the only thing preventing us from unhiding gaia-integration tests on TBPL.  Can we prioritize a fix here?
Alternative is to disable the test on TBPL (we have some others already).
Lets disable this for now... Unhiding is more important
Hidden in https://bugzilla.mozilla.org/show_bug.cgi?id=1023088 lets re-enable GI... I removed this bug as a blocker to turning GI on but this still need to be resolved (fixed) eventually.
No longer blocks: 960072
Comment on attachment 8435296 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/20102

LGTM. Thanks miller!
Attachment #8435296 - Flags: review?(gaye) → review+
landed into master: https://github.com/mozilla-b2g/gaia/commit/67b305497f3ac853c2434bde5969c4a629fca98b
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: 2.0 S3 (6june) → 2.0 S4 (20june)
disabled again in
master: a5cdbc08d023859f3f5a6b6163434a84f262ad85
damn.. I executed over 300 times each toggle calendar test before landing.. never seen it timeout once. only error that I was getting was the "Cannot call method 'scriptWith' of null", and the last 100+ times it passed without any error

https://travis-ci.org/millermedeiros/gaia/builds/26891442 (passed 50 times)
https://travis-ci.org/millermedeiros/gaia/builds/26871153 (passed 50 times)

maybe in some really hard to reproduce case the click that toggles the calendar visibility is not happening on the proper element; that would explain the timeout on the enable calendar test.

<rant>calendar integration tests improved a lot the past few months but it still drives me crazy when we have intermittent or perma red failures. So hard to debug and reproduce in some cases and not seeing the tests catching that many bugs anyway, so in my opinion I still think they cause way more headaches than they avoid... I don't think the effort is justified, specially since it takes so long to run on CI server and a good part of the tests are not reliable. When an "alarm" starts going off for no reason people learn to ignore it.</rant>
Yes, I agree, IMO integration tests should test the big cases using scenarios, especially cross-app stuff like activities or IAC, and we should not have a lot of them.

Here I saw the issue in 3 builds in a row, that's why I disabled the test. Maybe it was a moment where Travis' VM was especially loaded.
Target Milestone: 2.0 S4 (20june) → 2.0 S5 (4july)
Assignee: mmedeiros → nobody
Target Milestone: 2.0 S5 (4july) → ---
Duplicate of this bug: 1007097
Blocks: 1095712
Assignee: nobody → gaye
Status: REOPENED → RESOLVED
Closed: 6 years ago5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.