1221876 - Intermittent root_settings_test.js | check root panel settings common tests airplane mode enable airplane mode

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Settings, defect)

Description

[Nigel Babu [:nigelb]]

https://treeherder.mozilla.org/logviewer.html#?job_id=16830181&repo=mozilla-inbound

 04:04:12     INFO -  TEST-END | apps/settings/test/marionette/tests/notifications_settings_test.js | manipulate notifications settings disable Show on lockscreen took 498 ms

 04:09:08     INFO -  ..............................[marionette-mocha]   1 failing

 04:09:08     INFO -  [marionette-mocha]

 04:09:08     INFO -  [marionette-mocha]   1) check root panel settings common tests airplane mode enable airplane mode:

 04:09:08     INFO -    Error: timeout exceeded!

 04:09:08     INFO -        at Object.Client.waitForSync (/home/worker/gaia/node_modules/marionette-client/lib/marionette/client.js:756:16)

 04:09:08     INFO -        at Object.Client.waitFor (/home/worker/gaia/node_modules/marionette-client/lib/marionette/client.js:722:60)

 04:09:08     INFO -        at Object.RootPanel.airplaneMode (/home/worker/gaia/apps/settings/test/marionette/app/regions/root.js:50:17)

 04:09:08     INFO -        at Context.<anonymous> (/home/worker/gaia/apps/settings/test/marionette/tests/root_settings_test.js:89:19)

 04:09:08     INFO -        at callFn (/home/worker/gaia/node_modules/mocha/lib/runnable.js:223:21)

 04:09:08     INFO -        at Test.Runnable.run (/home/worker/gaia/node_modules/mocha/lib/runnable.js:216:7)

 04:09:08     INFO -        at Test.MarionetteTest.run (/home/worker/gaia/node_modules/marionette-js-runner/lib/ui.js:25:31)

 04:09:08     INFO -        at Runner.runTest (/home/worker/gaia/node_modules/mocha/lib/runner.js:373:10)

 04:09:08     INFO -        at /home/worker/gaia/node_modules/mocha/lib/runner.js:451:12

 04:09:08     INFO -        at next (/home/worker/gaia/node_modules/mocha/lib/runner.js:298:14)

 04:09:08     INFO -        at /home/worker/gaia/node_modules/mocha/lib/runner.js:308:7

 04:09:08     INFO -        at next (/home/worker/gaia/node_modules/mocha/lib/runner.js:246:23)

 04:09:08     INFO -        at Object._onImmediate (/home/worker/gaia/node_modules/mocha/lib/runner.js:275:5)

 04:09:08     INFO -        at processImmediate [as _immediateCallback] (timers.js:354:15)

 04:09:08     INFO -  [marionette-mocha]

 04:09:08     INFO -  .

 04:09:08     INFO -  /home/worker/gaia/apps/settings/test/marionette/tests/root_settings_test.js failed. Will retry.

Comment 1

[Intermittent Failures Robot]

91 automation job failures were associated with this bug yesterday.

Repository breakdown:
* b2g-inbound: 76
* mozilla-inbound: 12
* fx-team: 2
* mozilla-b2g44_v2_5: 1

Platform breakdown:
* b2g-linux64: 56
* mulet-linux64: 35

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1221876&startday=2015-11-05&endday=2015-11-05&tree=all

Comment 2

[Nigel Babu [:nigelb]]

I think this is related to the changes made in bug 1220046. Can you take a look?

Comment 3

[[e-mail]]

Ya... now we need more time to wait until wifi panel is ready

Comment 4

[Intermittent Failures Robot]

117 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* b2g-inbound: 85
* mozilla-inbound: 22
* fx-team: 7
* gaia: 2
* mozilla-b2g44_v2_5: 1

Platform breakdown:
* b2g-linux64: 71
* mulet-linux64: 44
* gaia: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1221876&startday=2015-11-02&endday=2015-11-08&tree=all

Comment 5

[[e-mail]]

BTW, The default wifi status is changed to `Not connected` instead of `Disabled`

Comment 6

[[e-mail]]

I found wifi description does not respond in mulet, and new airplane mode enable/disable state is count on it.

Nigel could you help point me about the right command to run integration test on device?



The command
```
BUILDAPP=device make test-integration
```
does not bring test on device

And 

```
TEST_FILES="/<absolute gaia path>/apps/settings/test/marionette/tests/airplane_mode_settings_test.js" npm run marionette
```

also test on mulet.

Comment 7

[GitHub Autolander Bot]

Attachment: [gaia] gasolin:issue-1221876 > mozilla-b2g:master

Comment 8

[Nigel Babu [:nigelb]]

This is what I got from freddyb and albertopq on #fxos

https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Automated_testing/Gaia_integration_tests#Running_tests_on_device
https://developer.mozilla.org/en-US/Firefox_OS/Automated_testing/Gaia_unit_tests
https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/Different_ways_to_run_Gaia#Using_Gaia_in_Firefox_Mulet

Comment 9

[Michael Henretty [:mikehenrty][:mhenretty]]

We will need this test fixed if we want to unhide the intermittent failures on Treeherder. Let me know if I can be of assistance here.

Comment 10

[Michael Henretty [:mikehenrty][:mhenretty]]

Fred, do you still need help running this on a device? I imagine people like Eli or Gareth could help you out there. In the meantime, is there anything blocking fixing this test that I can help out with?

Comment 11

[[e-mail]]

I have no luck to run this on a device yet, will try again tomorrow.

Another issue I found in Comment 6 is the wifi status cannot update on mulet (might due to lack of wifi api mock).

Comment 12

[Michael Henretty [:mikehenrty][:mhenretty]]

Not sure, but this might help with running on device:

https://bugzilla.mozilla.org/show_bug.cgi?id=1195220#c16

Comment 13

[Intermittent Failures Robot]

63 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 31
* b2g-inbound: 12
* gaia: 10
* fx-team: 7
* mozilla-central: 3

Platform breakdown:
* b2g-linux64: 31
* mulet-linux64: 22
* gaia: 10

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1221876&startday=2015-11-09&endday=2015-11-15&tree=all

Comment 14

[GitHub Autolander Bot]

Attachment: [gaia] mikehenrty:bug-1221876-disable-root-settings-test > mozilla-b2g:master

Comment 15

[Michael Henretty [:mikehenrty][:mhenretty]]

Comment on attachment 8687888 [details] [review]
[gaia] mikehenrty:bug-1221876-disable-root-settings-test > mozilla-b2g:master

Fred, since you are working on a fix for this test already, and since for the time being this test is not provided us with too much protection based on the changes you are making, can we disable this test so we can move forward with bug 1222215 this week? I figure you can just remove this line in the patch you've already submitted.

Comment 16

[[e-mail]]

My patch has addressed issue in comment 5, if you known someone is able to test it on device, please reach him for help.

Comment 17

[Intermittent Failures Robot]

41 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 31
* fx-team: 5
* mozilla-central: 3
* try: 1
* b2g-inbound: 1

Platform breakdown:
* mulet-linux64: 23
* b2g-linux64: 18

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1221876&startday=2015-11-16&endday=2015-11-22&tree=all

Comment 18

[Michael Henretty [:mikehenrty][:mhenretty]]

disabled for now, on master: https://github.com/mozilla-b2g/gaia/commit/7420221a40c1a35bc8122ddd0813c1422a39c582

Comment 19

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Fred could you find someone to work on this?

Comment 20

[[e-mail]]

Now I can reproduce on device with new ROM.
The updated patch shows all green on treeherder

Comment 21

[[e-mail]]

Comment on attachment 8684773 [details] [review]
[gaia] gasolin:issue-1221876 > mozilla-b2g:master

Yura, could you help review it?

Comment 22

[Yura Zenevich [:yzen] (please use "needinfo")]

Comment on attachment 8684773 [details] [review]
[gaia] gasolin:issue-1221876 > mozilla-b2g:master

Thanks, looks good and test passes.

Comment 23

[[e-mail]]

merged the first commit in mozlando lol
https://github.com/mozilla-b2g/gaia/commit/76e076a4d013b39016891aae6cdc4886a29e1486

Thanks for review!

Comment 24

[Nigel Babu [:nigelb]]

Backed out for bustage on Gij(15) on B2G Desktop and Mulet

Link for failure: https://treeherder.mozilla.org/logviewer.html#?job_id=3524218&repo=b2g-inbound

Revert: https://github.com/mozilla-b2g/gaia/commit/6b430ea7274af4c352de16b75e6bb85d7621ca83

Comment 25

[GitHub Autolander Bot]

Attachment: [gaia] gasolin:issue-1221876-2 > mozilla-b2g:master

Comment 26

[Michael Henretty [:mikehenrty][:mhenretty]]

Since this test is disabled, I'm unblocking it from bug 1222215.

Comment 27

[[e-mail]]

* comment out intermittent cases and will fix in following bug
* re-enable gij test for the rest of test cases.

merged https://github.com/mozilla-b2g/gaia/commit/a85b8648d5b5fb831d3379387a5be282a6677dff