1541210 - Write unit tests to ensure that the correct default engines are present for all locales/regions

Status: RESOLVED FIXED

(Firefox :: Search, task)

Description

[Mark Banner (:standard8)]

We need to add some unit tests to ensure that we have the correct default engines present in all region/locales.

As part of this we need to:

• Setup generic test harness to make writing tests easier
• Have a template per engine (aka file), to group rules per engines together
• Make sure all region/locales pairs have the correct defaults

Comment 1

[Mark Banner (:standard8)]

Attachment: Bug 1541210 - Write unit tests to ensure that the correct default engines are present for all locales/regions. r?daleharvey

Comment 2

[Mark Banner (:standard8)]

Try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5d76853f50e77a7ac948912beaaa3e08bd75f2d

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9e9812ed59a5
Write unit tests to ensure that the correct default engines are present for all locales/regions. r=daleharvey

Comment 4

[Razvan Maries]

Backed out for causing perma failures on: test_baidu.js, test_bing.js, test_google.js and test_yahdex.js.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=success%2Cpending%2Crunning%2Ctestfailed%2Cbusted%2Cexception&searchStr=linux%2Cx64%2Casan%2Cxpcshell%2Ctests%2Ctest-linux64-asan%2Fopt-xpcshell-e10s-7%2Cx%28x7%29&revision=9e9812ed59a5f2a79ee0376cc5dcc0dbc7c54910&selectedJob=242090183

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242090183&repo=autoland&lineNumber=2123

[task 2019-04-23T16:47:39.798Z] 16:47:39 INFO - TEST-START | toolkit/components/reputationservice/test/unit/test_app_rep_maclinux.js
[task 2019-04-23T16:47:40.507Z] 16:47:40 INFO - TEST-PASS | toolkit/components/reputationservice/test/unit/test_app_rep_maclinux.js | took 717ms
[task 2019-04-23T16:47:40.523Z] 16:47:40 INFO - TEST-START | toolkit/components/search/tests/xpcshell/searchconfigs/test_baidu.js
[task 2019-04-23T16:57:40.527Z] 16:57:40 WARNING - TEST-UNEXPECTED-TIMEOUT | toolkit/components/search/tests/xpcshell/searchconfigs/test_baidu.js | Test timed out
[task 2019-04-23T16:57:40.528Z] 16:57:40 INFO - TEST-INFO took 600001ms
[task 2019-04-23T16:57:40.528Z] 16:57:40 INFO - >>>>>>>
[task 2019-04-23T16:57:40.529Z] 16:57:40 INFO - (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2019-04-23T16:57:40.530Z] 16:57:40 INFO - (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2019-04-23T16:57:40.531Z] 16:57:40 INFO - (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2019-04-23T16:57:40.532Z] 16:57:40 INFO - running event loop
[task 2019-04-23T16:57:40.532Z] 16:57:40 INFO - toolkit/components/search/tests/xpcshell/searchconfigs/test_baidu.js | Starting setup
[task 2019-04-23T16:57:40.533Z] 16:57:40 INFO - (xpcshell/head.js) | test setup pending (2)
[task 2019-04-23T16:57:40.534Z] 16:57:40 INFO - PID 11041 | 1556038061025 addons.manager DEBUG Application has been upgraded
[task 2019-04-23T16:57:40.536Z] 16:57:40 INFO - PID 11041 | (xpcshell:11041): Gtk-CRITICAL **: gtk_settings_get_for_screen: assertion 'GDK_IS_SCREEN (screen)' failed
[task 2019-04-23T16:57:40.537Z] 16:57:40 INFO - PID 11041 | 1556038061082 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider", "XPIInternal"]
[task 2019-04-23T16:57:40.541Z] 16:57:40 INFO - PID 11041 | 1556038061102 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/GMPProvider.jsm
[task 2019-04-23T16:57:40.544Z] 16:57:40 INFO - PID 11041 | 1556038061113 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/PluginProvider.jsm
[task 2019-04-23T16:57:40.545Z] 16:57:40 INFO - PID 11041 | 1556038061114 addons.manager DEBUG Starting provider: XPIProvider
[task 2019-04-23T16:57:40.546Z] 16:57:40 INFO - PID 11041 | 1556038061115 addons.xpi DEBUG startup
[task 2019-04-23T16:57:40.547Z] 16:57:40 INFO - PID 11041 | 1556038061117 addons.xpi WARN List of valid built-in add-ons could not be parsed.: [Exception... "Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIXPCComponents_Utils.readUTF8URI]" nsresult: "0xc1f30001 (NS_ERROR_NOT_INITIALIZED)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: startup :: line 2243" data: no] Stack trace: startup()@resource://gre/modules/addons/XPIProvider.jsm:2243
[task 2019-04-23T16:57:40.548Z] 16:57:40 INFO - PID 11041 | callProvider()@resource://gre/modules/AddonManager.jsm:200
[task 2019-04-23T16:57:40.548Z] 16:57:40 INFO - PID 11041 | _startProvider()@resource://gre/modules/AddonManager.jsm:651
[task 2019-04-23T16:57:40.548Z] 16:57:40 INFO - PID 11041 | startup()@resource://gre/modules/AddonManager.jsm:807
[task 2019-04-23T16:57:40.549Z] 16:57:40 INFO - PID 11041 | startup()@resource://gre/modules/AddonManager.jsm:2806
[task 2019-04-23T16:57:40.549Z] 16:57:40 INFO - PID 11041 | observe()@resource://gre/modules/addonManager.js:65
[task 2019-04-23T16:57:40.549Z] 16:57:40 INFO - PID 11041 | promiseStartupManager()@resource://testing-common/AddonTestUtils.jsm:750
[task 2019-04-23T16:57:40.549Z] 16:57:40 INFO - PID 11041 | setup()@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/search/tests/xpcshell/searchconfigs/head_searchconfig.js:82
[task 2019-04-23T16:57:40.550Z] 16:57:40 INFO - PID 11041 | setup()@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/search/tests/xpcshell/searchconfigs/test_baidu.js:21
[task 2019-04-23T16:57:40.550Z] 16:57:40 INFO - PID 11041 | run_next_test/_run_next_test/<()@/builds/worker/workspace/build/tests/xpcshell/head.js:1437
[task 2019-04-23T16:57:40.550Z] 16:57:40 INFO - PID 11041 | _run_next_test()@/builds/worker/workspace/build/tests/xpcshell/head.js:1437
[task 2019-04-23T16:57:40.551Z] 16:57:40 INFO - PID 11041 | run()@/builds/worker/workspace/build/tests/xpcshell/head.js:688
[task 2019-04-23T16:57:40.551Z] 16:57:40 INFO - PID 11041 | _do_main()@/builds/worker/workspace/build/tests/xpcshell/head.js:227
[task 2019-04-23T16:57:40.552Z] 16:57:40 INFO - PID 11041 | _execute_test()@/builds/worker/workspace/build/tests/xpcshell/head.js:529
[task 2019-04-23T16:57:40.553Z] 16:57:40 INFO - PID 11041 | -e:1
[task 2019-04-23T16:57:40.554Z] 16:57:40 INFO - PID 11041 | 1556038061128 addons.xpi INFO SystemAddonLocation directory is missing
[task 2019-04-23T16:57:40.555Z] 16:57:40 INFO - PID 11041 | 1556038061170 addons.xpi INFO Removing all system add-on upgrades.
[task 2019-04-23T16:57:40.556Z] 16:57:40 INFO - PID 11041 | 1556038061172 addons.xpi DEBUG checkForChanges
[task 2019-04-23T16:57:40.557Z] 16:57:40 INFO - PID 11041 | 1556038061174 addons.xpi DEBUG Loaded add-on state: ${}
[task 2019-04-23T16:57:40.558Z] 16:57:40 INFO - PID 11041 | 1556038061176 addons.xpi DEBUG scanForChanges changed: false, state: {}
[task 2019-04-23T16:57:40.559Z] 16:57:40 INFO - PID 11041 | 1556038061178 addons.xpi DEBUG Empty XPI database, setting schema version preference to 29
[task 2019-04-23T16:57:40.563Z] 16:57:40 INFO - PID 11041 | 1556038061178 addons.xpi DEBUG No changes found
[task 2019-04-23T16:57:40.564Z] 16:57:40 INFO - PID 11041 | 1556038061208 addons.xpi-utils DEBUG Error: Synchronously loading the add-ons database(resource://gre/modules/addons/XPIDatabase.jsm:1351:15) JS Stack trace: [e-mail]:1351:15
[task 2019-04-23T16:57:40.564Z] 16:57:40 INFO - PID 11041 | [e-mail]:2289:23
[task 2019-04-23T16:57:40.565Z] 16:57:40 INFO - PID 11041 | [e-mail]:200:31
[task 2019-04-23T16:57:40.567Z] 16:57:40 INFO - PID 11041 | [e-mail]:651:5
[task 2019-04-23T16:57:40.567Z] 16:57:40 INFO - PID 11041 | [e-mail]:807:14
[task 2019-04-23T16:57:40.569Z] 16:57:40 INFO - PID 11041 | [e-mail]:2806:26
[task 2019-04-23T16:57:40.570Z] 16:57:40 INFO - PID 11041 | [e-mail]:65:29
[task 2019-04-23T16:57:40.571Z] 16:57:40 INFO - PID 11041 | [e-mail]:750:34
[task 2019-04-23T16:57:40.572Z] 16:57:40 INFO - PID 11041 | setup@head_searchconfig.js:82:26
[task 2019-04-23T16:57:40.573Z] 16:57:40 INFO - PID 11041 | setup@test_baidu.js:21:14
[task 2019-04-23T16:57:40.574Z] 16:57:40 INFO - PID 11041 | run_next_test/_run_next_test/<@head.js:1437:22
[task 2019-04-23T16:57:40.576Z] 16:57:40 INFO - PID 11041 | [e-mail]:1437:38
[task 2019-04-23T16:57:40.576Z] 16:57:40 INFO - PID 11041 | [e-mail]:688:9
[task 2019-04-23T16:57:40.578Z] 16:57:40 INFO - PID 11041 | [e-mail]:227:6
[task 2019-04-23T16:57:40.578Z] 16:57:40 INFO - PID 11041 | [e-mail]:529:5
[task 2019-04-23T16:57:40.579Z] 16:57:40 INFO - PID 11041 | @-e:1:1
[task 2019-04-23T16:57:40.581Z] 16:57:40 INFO - PID 11041 | 1556038061211 addons.xpi-utils DEBUG Starting async load of XPI database /tmp/xpc-profile-CLYXu9/extensions.json
[task 2019-04-23T16:57:40.581Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061025 addons.manager DEBUG Application has been upgraded"
[task 2019-04-23T16:57:40.583Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061082 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider", "XPIInternal"]"
[task 2019-04-23T16:57:40.584Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061102 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/GMPProvider.jsm"
[task 2019-04-23T16:57:40.585Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061113 addons.manager DEBUG Loaded provider scope for resource://gre/modules/addons/PluginProvider.jsm"
[task 2019-04-23T16:57:40.587Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061114 addons.manager DEBUG Starting provider: XPIProvider"
[task 2019-04-23T16:57:40.588Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061115 addons.xpi DEBUG startup"
[task 2019-04-23T16:57:40.590Z] 16:57:40 INFO - "CONSOLE_MESSAGE: (info) 1556038061117 addons.xpi WARN List of valid built-in add-ons could not be parsed.: [Exception... "Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIXPCComponents_Utils.readUTF8URI]" nsresult: "0xc1f30001 (NS_ERROR_NOT_INITIALIZED)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: startup :: line 2243" data: no] Stack trace: startup()@resource://gre/modules/addons/XPIProvider.jsm:2243
[task 2019-04-23T16:57:40.591Z] 16:57:40 INFO - callProvider()@resource://gre/modules/AddonManager.jsm:200
[task 2019-04-23T16:57:40.592Z] 16:57:40 INFO - _startProvider()@resource://gre/modules/AddonManager.jsm:651
[task 2019-04-23T16:57:40.592Z] 16:57:40 INFO - startup()@resource://gre/modules/AddonManager.jsm:807
[task 2019-04-23T16:57:40.593Z] 16:57:40 INFO - startup()@resource://gre/modules/AddonManager.jsm:2806
[task 2019-04-23T16:57:40.594Z] 16:57:40 INFO - observe()@resource://gre/modules/addonManager.js:65
[task 2019-04-23T16:57:40.595Z] 16:57:40 INFO - promiseStartupManager()@resource://testing-common/AddonTestUtils.jsm:750
[task 2019-04-23T16:57:40.595Z] 16:57:40 INFO - setup()@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/search/tests/xpcshell/searchconfigs/head_searchconfig.js:82
[task 2019-04-23T16:57:40.596Z] 16:57:40 INFO - setup()@/builds/worker/workspace/build/tests/xpcshell/tests/toolkit/components/search/tests/xpcshell/searchconfigs/test_baidu.js:21
[task 2019-04-23T16:57:40.597Z] 16:57:40 INFO - run_next_test/_run_next_test/<()@/builds/worker/workspace/build/tests/xpcshell/head.js:1437
[task 2019-04-23T16:57:40.598Z] 16:57:40 INFO - _run_next_test()@/builds/worker/workspace/build/tests/xpcshell/head.js:1437
[task 2019-04-23T16:57:40.599Z] 16:57:40 INFO - run()@/builds/worker/workspace/build/tests/xpcshell/head.js:688
[task 2019-04-23T16:57:40.600Z] 16:57:40 INFO - _do_main()@/builds/worker/workspace/build/tests/xpcshell/head.js:227
[task 2019-04-23T16:57:40.601Z] 16:57:40 INFO - _execute_test()@/builds/worker/workspace/build/tests/xpcshell/head.js:529
[task 2019-04-23T16:57:40.602Z] 16:57:40 INFO - -e:1"

Backout: https://hg.mozilla.org/integration/autoland/rev/783efcde8dfbff4f9bef21fbc4e92abfe73f960d

Comment 5

[Mark Banner (:standard8)]

The failure is in the slower builds of asan. I think as we're already disabling for debug, then we'll disable for asan as well.

Hopefully the future changes to search will mean that we can speed these tests up.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5a728b586c41
Write unit tests to ensure that the correct default engines are present for all locales/regions. r=daleharvey

Comment 7

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/5a728b586c41