1446938 - ThirdPartySearchTest should not use Google's I'm Feeling Lucky as its custom engine

Status: RESOLVED FIXED

(Firefox for iOS :: Browser, defect, P3)

Description

[Isabel Rios[:isabel_rios]]

Reproducible on master and latest v11.x

Please see the test that is failing: 
https://github.com/mozilla-mobile/firefox-ios/blob/master/XCUITests/ThirdPartySearchTest.swift#L129

And the failure:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5aac689169cbb9000125e63c/test/0917891b-6ed4-443b-8c9a-9c197fbdb2d5/61293218-bd29-4e19-9e03-ca8493199823

Steps:
-Create and save a custom search engine: 
--Name-Feeling Lucky, 
--CustomEngineURL: http://www.google.com/search?q=%s&btnI

-Go to a new tab and search, as the test example: "stream charm"
-Select the custom search engine to perform the search

Expected:
User is taken to google site showing the different results for the search

Actual:
User is directly taken to different websites (youtube or rainbow6.ubisoft.com)

Comment 1

[Justin D'Arcangelo [:justindarc]]

Isabel, I noticed that your expected/actual are incorrect for this particular custom search engine. The custom engine URL you're using is the Google "I'm Feeling Lucky" search which should take the user directly to a website and not actually show search results.

That being said, it is possible for this test to intermittently fail. A quick glance at the test reveals that it looks for the `&btnI` suffix in the URL. However, if the page loads quick enough, its possible that the address bar will change to the URL of the first search result before this check happens.

IMO, I would rework this test to use a different custom search engine that produces more consistent results. In the meantime, I'm going to demote this bug to tracking+ P3 and rename the summary to accurately describe that this is an issue with the test.

Comment 2

[Isabel Rios[:isabel_rios]]

Thanks Justin for taking a look at it and for your explanation here. What you comment is exactly what happens when the page loads quickly. I will investigate and re-write the test using a better custom search engine. In the meantime we will disable the test (bug 1447918)

Thanks!

Comment 3

[Isabel Rios[:isabel_rios]]

Attachment: Pull Request

Comment 4

[Farhan Patel [:farhan]]

merged