1654348 - Cleanup of browser search engine tests

Status: RESOLVED FIXED

(Firefox :: Search, task)

Description

[Tracy Walker [:tracy]]

I would like to work on automating the manually run test cases for search codes of top search engines Yandex: Russia and Baidu - China.

Should I add these checks to browser_searchEngine_behaviors.js or create individual tests for each search engine, like there is in Google's case?

Also, would it be appropriate for the Google test, browser_google_behavior.js, to be folded back into browser_searchEngine_behaviors.js since the TODO blocker bug 1315953, in the latter, was fixed?

Comment 1

[Mark Banner (:standard8)]

Hi Tracy,

We already have extensive test cases that live here: https://searchfox.org/mozilla-central/source/toolkit/components/search/tests/xpcshell/searchconfigs - these test the output from the backend search service directly.

We would like to keep browser_*behavior.js as integration tests, so that we're testing at least some of the top engines from the UI, and ensuring that the urls are being handled properly everywhere.

However, adding Yandex & Baidu would be slightly complicated as we would have to do region changes and maybe locale changes, so given that we have the searchconfigs tests, we don't think it is worh adding those.

We would like to merge browser_searchEngine_behaviors.js with browser_google_behavior.js - as you're right, we should have done that previously (I think it is possible).

We should also take the opportunity to remove the browser_<enginename>.js files since these have been made obsolete by the searchconfigs.

Would you be happy to do those changes?

Comment 2

[Tracy Walker [:tracy]]

Hi Mark,

Yes, I'd be glad make those changes.

Comment 3

[Tracy Walker [:tracy]]

It looks like browser_google_behavior.js is doing some checks for region and esr that should not be merged into browser_searchEngine_behaviors.js. Perhaps the Google behavior test should remain on its own? What do you think?

Comment 4

[Mark Banner (:standard8)]

(In reply to Tracy Walker [:tracy] from comment #3)

It looks like browser_google_behavior.js is doing some checks for region and esr that should not be merged into browser_searchEngine_behaviors.js. Perhaps the Google behavior test should remain on its own? What do you think?

Oh that's true. Probably better to keep those separate for now then.

Comment 5

[Tracy Walker [:tracy]]

Attachment: Bug 1654348 - Remove obsolete browser_<enginename>.js tests r=standard8

Comment 6

[Tracy Walker [:tracy]]

Unfortunately, I missed that the browser_google.js test was also being used in google_codes, one directory deeper. I'll remove it from the browser.ini there.

However, I am seeing the following when attempting to run this test group locally.

ErrorMessage: Symlink target path does not exist: /Users/tracy/mozilla-central/browser/components/search/test/browser/browser_amazon.js

  File "/Users/tracy/mozilla-central/testing/mochitest/mach_commands.py", line 343, in run_mochitest_general
    driver.install_tests()
  File "/Users/tracy/mozilla-central/python/mozbuild/mozbuild/controller/building.py", line 1386, in install_tests
    '_tests')
  File "/Users/tracy/mozilla-central/python/mozbuild/mozbuild/testing.py", line 202, in install_test_files
    remove_unaccounted=False)
  File "/Users/tracy/mozilla-central/python/mozbuild/mozpack/copier.py", line 424, in copy
    copy_results.append((destfile, f.copy(destfile, skip_if_older)))
  File "/Users/tracy/mozilla-central/python/mozbuild/mozpack/files.py", line 389, in copy
    raise ErrorMessage('Symlink target path does not exist: %s' % self.path)

I am unable to figure out what is trying to find browser_amazon.js (which has been removed by this patch).

Comment 7

[Mark Banner (:standard8)]

Did you rebuild after making the changes? Normally that happens if you adjust files in a .ini file and you haven't re-built before running tests.

Comment 8

[Tracy Walker [:tracy]]

Attachment: Bug 1654348 - Remove test case from additional browser.ini file r=standard8

Depends on D84569

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/838b5ddfbcb4
Remove obsolete browser_<enginename>.js tests r=Standard8

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/838b5ddfbcb4