1170148 - Fix test_toolbars.py for unified auto-complete feature

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We got another test failure this time in the toolbar unit tests, which is also most likely related to the enabling of the unified auto-complete via bug 1168811. 

Traceback (most recent call last):
  File "/home/mozauto/jenkins/workspace/mozilla-central_functional/venv/local/lib/python2.7/site-packages/marionette_client-0.14-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/home/mozauto/jenkins/workspace/mozilla-central_functional/firefox-ui-tests/firefox_ui_harness/decorators.py", line 23, in wrapper
    return target(self, *args, **kwargs)
  File "/home/mozauto/jenkins/workspace/mozilla-central_functional/firefox-ui-tests/firefox_puppeteer/tests/test_toolbars.py", line 101, in test_popup_elements
    int(results.get_attribute('itemCount')))

AssertionError: 1 != 8


Problem here is that we are waiting for "len(results) > 0", which will return directly because there is the new search entry present. Auto-complete results will be populated asynchronously, so the number of results will be >1.

We should find a way to harden the test module and make it more reliable. For now I would just skip it.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: skip patch

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Skip patch for autocomplete results landed as:
https://github.com/mozilla/firefox-ui-tests/commit/67bab5bb5b41411d021ec8a21c3082615e90355a

Marco, I have some questions you can hopefully answer:

1. Is there an event we could wait for when autopopup results have been populated? Would this work with unified autocomplete enabled and disabled?
2. Are all autocomplete results injected at the same time into the popup? Or does it happen step by step whenever a result is found?

Comment 3

[Marco Bonardo [:mak]]

(In reply to Henrik Skupin (:whimboo) from comment #2)
> 1. Is there an event we could wait for when autopopup results have been
> populated? Would this work with unified autocomplete enabled and disabled?

there's no event, but you can poll for gURLBar.controller.searchStatus, see
http://mxr.mozilla.org/mozilla-central/source/browser/base/content/test/general/head.js#812

> 2. Are all autocomplete results injected at the same time into the popup? Or
> does it happen step by step whenever a result is found?

It happens step by step, cause we want to give feedback to the user as soon as possible, so we can't wait for all the results to arrive.

Comment 4

[Barbara Miller (:galgeek)]

Attachment: github pull request

Comments on the github PR.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We clearly need this fix ASAP to avoid more test failure regressions coming up. Barbara, please let me know if you will be able to get your patch fixed up in a short time.

Comment 6

[Barbara Miller (:galgeek)]

Comment on attachment 8622123 [details] [review]
github pull request

I've commented on github.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8622123 [details] [review]
github pull request

Close but not done yet. The next round should be fine.

Comment 8

[Barbara Miller (:galgeek)]

Comment on attachment 8622123 [details] [review]
github pull request

Thanks for your review, Henrik!

I've updated and added brief comments on github.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

https://github.com/mozilla/firefox-ui-tests/commit/0310d279ecb2944130a6d3e18071330ccd589da2

Thanks Barbara!