1236251 - Intermittent apps/ftu/test/marionette/rtl_test.js | First Time Use > Change language to bidi english

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::First Time Experience, defect)

Description

[Phil Ringnalda (:philor)]

https://treeherder.mozilla.org/logviewer.html#?job_id=2988649&repo=mozilla-central

Comment 2

[Phil Ringnalda (:philor)]

Wrong one from the multiple bugs I filed off that single run, this one is a crash with no stack masquerading as just an ordinary failure, not the port silliness.

Comment 3

[Phil Ringnalda (:philor)]

Or a not-a-crash masquerading as a crash, whatever that is.

Comment 4

[Intermittent Failures Robot]

9 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 4
* fx-team: 2
* try: 1
* mozilla-central: 1
* b2g-inbound: 1

Platform breakdown:
* mulet-linux64: 8
* b2g-linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236251&startday=2016-01-04&endday=2016-01-10&tree=all

Comment 5

[GitHub Autolander Bot]

Attachment: [gaia] sfoster:ftu-rtl-gij-intermittent-bug-1236251 > mozilla-b2g:master

Comment 6

[Sam Foster [:sfoster] (he/him)]

Running just this test at https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=f334944730d5072245c14b2f049ade0abfe369e0 to investigate/confirm

Comment 7

[Sam Foster [:sfoster] (he/him)]

Works for me - see 20 green runs in Comment #6, and I don't see this failing any more on inbound? (this test is not blacklisted and still running there) Possibly fixed by bug 1188274 or bug 1233892?

Comment 8

[Phil Ringnalda (:philor)]

https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236251 says we saw one instance on each of mozilla-inbound and fx-team last Friday, three instances last Wednesday, zero on Thursday. So I'd expect it to be at most more like 0.5-1.0% than the 5% that would make 20 runs likely to hit it.

Comment 9

[Sam Foster [:sfoster] (he/him)]

(In reply to Phil Ringnalda (:philor) from comment #8)
> https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236251 says
> we saw one instance on each of mozilla-inbound and fx-team last Friday,
> three instances last Wednesday, zero on Thursday. So I'd expect it to be at
> most more like 0.5-1.0% than the 5% that would make 20 runs likely to hit it.

Ok. I see some stuff in this test that could be fixed, so I'll patch it and go from there.

Comment 10

[Sam Foster [:sfoster] (he/him)]

nm mhenretty, I'll flag you for review when I have something.

Comment 11

[Sam Foster [:sfoster] (he/him)]

https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=2df0650c5322d25753edf1655b3239208e42956f 

The PR fixes up the test to use the default timeouts and leverage the new helpers. I see 1/40 failures, and from the logs it looks like the FTU didn't start yet? 

NoSuchElement: Unable to locate element: #windows iframe[src*="app://ftu.gaiamobile.org"]

That's from:

           at getAppWithCheckOnOriginOnly (/home/worker/git_checkout/node_modules/marionette-apps/lib/getapp.js:113:10)
           at getApp (/home/worker/git_checkout/node_modules/marionette-apps/lib/getapp.js:63:10)
           at switchToApp (/home/worker/git_checkout/node_modules/marionette-apps/lib/switchtoapp.js:24:10)
           at Object.Ftu.switchToFtu (/home/worker/git_checkout/apps/ftu/test/marionette/lib/ftu.js:109:22)
           at Object.Ftu.waitForCurtainUp (/home/worker/git_checkout/apps/ftu/test/marionette/lib/ftu.js:94:10)
           at Object.Ftu.waitForFtuReady (/home/worker/git_checkout/apps/ftu/test/marionette/lib/ftu.js:114:10)
           at Context.<anonymous> (/home/worker/git_checkout/apps/ftu/test/marionette/rtl_test.js:24:9)

Would I expect to see something about switchToApp / findElement timing out here? I'm just infering this.

Comment 12

[Michael Henretty [:mikehenrty][:mhenretty]]

> NoSuchElement: Unable to locate element: #windows iframe[src*="app://ftu.gaiamobile.org"]

This means that findElement for the ftu app's iframe when doing a switchToApp is timing out internally inside the marionette driver. Sounds like bootup time for the system is slowing down in these cases, which is indeed not related to this test. I'm ok with closing this bug if that's the only problem we are seeing.

Comment 13

[Sam Foster [:sfoster] (he/him)]

Comment on attachment 8706510 [details] [review]
[gaia] sfoster:ftu-rtl-gij-intermittent-bug-1236251 > mozilla-b2g:master

I've re-triggered but this is looking good so far. The patch refactors the test to use new helpers and fixes a bad assertion. It doesnt attempt to address the occasional bootup timeout we've seen.

Comment 14

[Michael Henretty [:mikehenrty][:mhenretty]]

Comment on attachment 8706510 [details] [review]
[gaia] sfoster:ftu-rtl-gij-intermittent-bug-1236251 > mozilla-b2g:master

Much improved, thanks!

Comment 15

[Sam Foster [:sfoster] (he/him)]

Merged to master: https://github.com/mozilla-b2g/gaia/commit/137679df10f04d4935ba5668b71e30a09a16447b