travis gaia-ui-tests busted with "unable to locate element" errors

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: bkelly, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
This morning we started getting travis bustage on the gaia-ui-tests with errors like these:

TEST-UNEXPECTED-FAIL | test_everythingme_launch_packaged_app.py TestEverythingMeSearchPanel.test_launch_packaged_app_from_search_panel | NoSuchElementException: Unable to locate element: button.keyboard-key[data-keycode="67"]

TEST-UNEXPECTED-FAIL | test_persona_app.py TestPersonaStandard.test_persona_standard_sign_in | TimeoutException: Element li.login not present before timeout

Updated

4 years ago
Component: Gaia → Gaia::UI Tests
(Reporter)

Comment 1

4 years ago
Bisecting suggests this is caused by:

  https://github.com/mozilla-b2g/gaia/commit/99096794

Note, its a bit intermittent, but reproduces about 50% or 60% of the time.
Blocks: 924764
(Reporter)

Comment 2

4 years ago
Backed out:

  https://github.com/mozilla-b2g/gaia/commit/f5c2e987fcc859e371fedb44006ca94fc0ebcb28
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
First off thanks for backing it out Ben! And of course sorry for introducing it.

Can someone from automation help me understand this? It looks like according to the errors it can't find the letter 'C' on the keyboard, or it can't find the "li.login" element - neither of which are relevant (as far as I know) to e.me code. Also it seems to work well on an actual device (Unagi).

So I'm just trying to make sense of this issue.
(Reporter)

Comment 4

4 years ago
(In reply to Evyatar 'Tron' Amitay (everything.me) from comment #3)
> Can someone from automation help me understand this? It looks like according
> to the errors it can't find the letter 'C' on the keyboard, or it can't find
> the "li.login" element - neither of which are relevant (as far as I know) to
> e.me code. Also it seems to work well on an actual device (Unagi).

First, the backout I made was mainly in regards to the keyboard error.  I didn't actually see the other error duplicating, so thats probably a rare intermittent.

I thought perhaps the keyboard issue might be related to the changes in keyboard visibility events for the refactor.  Perhaps its trying to type before the keyboard is actually displayed.

Comment 5

4 years ago
Created attachment 815486 [details]
eme_twitter_ss.png

Tron, if it helps we are seeing these tests fail on device but with different symptoms. (which is not uncommon because the device is less powerful than desktopb2g)
I've attached a screenshot that shows when typing some characters go missing. The search term is supposedto be "twitter". Visually watching the device during the test it's all 'a bit slow' as best as I can describe it.

Usually this is quite reliable since Ran made reliability fixes a couple of weeks ago. Ran can show you how to run these tests against a device locally.
(Reporter)

Comment 6

4 years ago
Zac, I think that might be a different problem.  In this case the test fails before ever typing a single character.  It can't find the first key to hit when it tries to start typing.

I would really recommend running against b2gdesktop in order to use the same environment travis is using.

(Having them work correctly on devices is important, too, of course!)
(Reporter)

Comment 7

4 years ago
To reproduce on b2gdesktop you can run the same steps travis uses:

  cd gaia
  npm install
  ./node_modules/.bin/travisaction gaia_ui_tests install
  ./node_modules/.bin/travisaction gaia_ui_tests script

That last step runs all the tests, which takes a long time.  So you can instead run just the single failing test with:

  gaiatest --app=b2gdesktop --binary=b2g/b2g-bin --profile=profile --restart tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini /srv/gaia-master/tests/python/gaia-ui-tests/gaiatest/tests/functional/everythingme/test_everythingme_launch_packaged_app.py
You need to log in before you can comment on or make changes to this bug.