780554 - Mozmill test failure testSearch/testAddMozSearchProvider | Current URL contains the wrong search term

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Maniac Vlad Florin (:vladmaniac)]

This is failing constantly on all platforms since MDN change lately and this test uses a search engine which points to a MDN search 

https://developer.mozilla.org/en-US/

Local error log: 
"message": "Current URL contains the search term - got 'https://developer.mozilla.org/en-us/users/login?next=/en-us/docs/new%3fslug%3dspecial%253asearch', expected 'firefox'"
matches mozmill-ci report: 

http://mozmill-ci.blargon7.com/#/functional/report/29fc09ba0c8360d637617903a01a793b

Mozmill version: 1.5.17
Platforms: all

Comment 1

[Maniac Vlad Florin (:vladmaniac)]

Attachment: quickey v1.0

So this is a simple fix - 
* just changed the search term because there were no search results in the new MDN for "Firefox" string.
* also, in the long term, I would suggest switching MDN with a mozqa page - not a local search engine (because this is the only test remaining to use the modal dialog and thoroughly test the UI and IMO we need it). MDN points to a browser ID login at the moment and I don't think we want to use credentials in our tests (I mean we did not so far)

Comment 2

[Maniac Vlad Florin (:vladmaniac)]

The patch will apply cleanly also on mozilla-esr10 branch

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 649192 [details] [diff] [review]
quickey v1.0

>-  searchBar.search({text: "Firefox", action: "goButton"});
>+  searchBar.search({text: "Mozilla", action: "goButton"});

This is not a valid change. The string 'mozilla' appears in the URL because of the domain name, not because the search term has been passed correctly. This makes the test a false positive as the search functionality is not being tested at all.

We should change this test to use a local page, or at least to use mozqa.com.

Comment 4

[Maniac Vlad Florin (:vladmaniac)]

Attachment: fix patch 2.0

Fixed to use a page from mozqa.com 
Thanks Dave for the fast review

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 649217 [details] [diff] [review]
fix patch 2.0

If we are going to use mozqa.com we have to update the whole XML file. Just replacing the Url entry is not all as you can see when closely watching the file.

Also, I still want to see this test using a local search provider, but that's clearly a different bug.

Comment 6

[Maniac Vlad Florin (:vladmaniac)]

Attachment: patch v2.1

You're right, sorry for missing the rest of the details.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 649220 [details] [diff] [review]
patch v2.1

>+  <ShortName>Moz Search</ShortName>

Please use 'mozqa.com' as short name.

>+  <Description>Mozilla QA Search Results Page</Description>

Same here: 'Mozqa.com search results'

Comment 8

[Maniac Vlad Florin (:vladmaniac)]

Attachment: patch v2.2

fixed

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 649248 [details] [diff] [review]
patch v2.2

I will land it across branches now. Vlad, please file an infrastructure bug so we can get the XML file updated in the litmus-data repository.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

http://hg.mozilla.org/qa/mozmill-tests/rev/c9cc7d5a8269 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/1227f70eb535 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/ec753932eb96 (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/9c64d7f8ff0f (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/29843ea9d97b (esr10)