Open Bug 1556789 Opened 5 years ago Updated 2 years ago

Refactor search extension loading in search service

Categories

(WebExtensions :: General, defect, P3)

Product:
WebExtensions
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
REOPENED

People

(Reporter: mixedpuppy, Unassigned)

References

Details

Attachments

(3 obsolete files)

After bug 1552559 the extension loading is working around how extension loading/startup normally happens. This is due to a lack of a good way to know when an extension has both started and made a search engine available, so searchservice init has no good way to wait on that. The refactor will provide a wrapper for loading that will enable a promise based mechanism to load the search engines.

This provides a set of promises that the searchservice resolves once the search engine has been configured

Attachment #9069699 - Attachment description: Bug 1556789 Refactor extension install in searchservice to use promises → Bug 1556789 - Part 1: Refactor extension install in searchservice to use promises
Attachment #9069824 - Attachment is obsolete: true
Attachment #9069699 - Attachment description: Bug 1556789 - Part 1: Refactor extension install in searchservice to use promises → Bug 1556789 - Refactor extension install in searchservice to use promises

700K extension reloads! This minimizes tests to using the
necessary extensions as well as avoiding reloads of extensions while
the search config tests are run. Local machine can now run a single
search config test file in under 2 minutes. This was impossible before.

Blocks: 1559530
Pushed by scaraveo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/021d8cd8ba70 Refactor extension install in searchservice to use promises r=robwu,daleharvey https://hg.mozilla.org/integration/autoland/rev/5488dbd0fa34 minimize the chaos of search config testing r=daleharvey
Pushed by scaraveo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/0acf6bafda0f Refactor extension install in searchservice to use promises r=robwu,daleharvey https://hg.mozilla.org/integration/autoland/rev/2d10e95cf0cd minimize the chaos of search config testing r=daleharvey
Backout by rgurzau@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e94e4cbfcecb Backed out 2 changesets for assertion failures in FF functional tests on a CLOSED TREE.

Backed out 2 changesets (bug 1556789) for assertion failures in FF functional tests on a CLOSED TREE.

Backout link: https://hg.mozilla.org/integration/autoland/rev/e94e4cbfcecb8b21c305a726c0354cc4fc2c32cf

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&revision=2d10e95cf0cd0a37793ebc0a6f8e64cc79b599eb&selectedJob=256084522

Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=256084522&repo=autoland&lineNumber=34752

Log snippet:

[task 2019-07-12T01:09:08.447Z] 01:09:08 INFO - JavaScript error: resource://gre/modules/SearchService.jsm, line 608: NS_ERROR_FAILURE: SearchService previously failed to initialize
[task 2019-07-12T01:09:08.544Z] 01:09:08 INFO - 1562893748541 Marionette DEBUG 34 -> [0,300,"WebDriver:GetChromeWindowHandles",{}]
[task 2019-07-12T01:09:08.545Z] 01:09:08 INFO - 1562893748542 Marionette DEBUG 34 <- [1,300,null,["1"]]
[task 2019-07-12T01:09:08.600Z] 01:09:08 INFO - 1562893748593 Marionette DEBUG 34 -> [0,301,"Marionette:GetContext",{}]
[task 2019-07-12T01:09:08.601Z] 01:09:08 INFO - 1562893748594 Marionette DEBUG 34 <- [1,301,null,{"value":"chrome"}]
[task 2019-07-12T01:09:08.601Z] 01:09:08 INFO - 1562893748596 Marionette DEBUG 34 -> [0,302,"Marionette:SetContext",{"value":"chrome"}]
[task 2019-07-12T01:09:08.602Z] 01:09:08 INFO - 1562893748597 Marionette DEBUG 34 <- [1,302,null,{"value":null}]
[task 2019-07-12T01:09:08.609Z] 01:09:08 INFO - 1562893748603 Marionette DEBUG 34 -> [0,303,"WebDriver:TakeScreenshot",{"highlights":null,"full":true,"hash":false,"id":null,"scroll":true}]
[task 2019-07-12T01:09:08.610Z] 01:09:08 INFO - ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2019-07-12T01:09:08.611Z] 01:09:08 INFO - 1562893748608 Marionette DEBUG 34 <- [1,303,{"error":"no such window","message":"Browsing context has been discarded","stacktrace":"WebDriverError@chrome://marion ... t@chrome://marionette/content/server.js:249:9\n_onJSONObjectReady/<@chrome://marionette/content/transport.js:503:20\n"},null]
[task 2019-07-12T01:09:08.619Z] 01:09:08 WARNING - Failed to gather test failure debug: Browsing context has been discarded
[task 2019-07-12T01:09:08.620Z] 01:09:08 WARNING - stacktrace:
[task 2019-07-12T01:09:08.621Z] 01:09:08 WARNING - WebDriverError@chrome://marionette/content/error.js:175:5
[task 2019-07-12T01:09:08.622Z] 01:09:08 WARNING - NoSuchWindowError@chrome://marionette/content/error.js:409:5
[task 2019-07-12T01:09:08.622Z] 01:09:08 WARNING - assert.that/<@chrome://marionette/content/assert.js:428:13
[task 2019-07-12T01:09:08.623Z] 01:09:08 WARNING - assert.open@chrome://marionette/content/assert.js:183:72
[task 2019-07-12T01:09:08.624Z] 01:09:08 WARNING - GeckoDriver.prototype.takeScreenshot@chrome://marionette/content/driver.js:3026:20
[task 2019-07-12T01:09:08.624Z] 01:09:08 WARNING - despatch@chrome://marionette/content/server.js:305:40
[task 2019-07-12T01:09:08.625Z] 01:09:08 WARNING - execute@chrome://marionette/content/server.js:275:16
[task 2019-07-12T01:09:08.626Z] 01:09:08 WARNING - onPacket/<@chrome://marionette/content/server.js:248:20
[task 2019-07-12T01:09:08.626Z] 01:09:08 WARNING - onPacket@chrome://marionette/content/server.js:249:9
[task 2019-07-12T01:09:08.627Z] 01:09:08 WARNING - _onJSONObjectReady/<@chrome://marionette/content/transport.js:503:20
[task 2019-07-12T01:09:08.628Z] 01:09:08 WARNING -
[task 2019-07-12T01:09:08.628Z] 01:09:08 INFO - TEST-UNEXPECTED-FAIL | testing/firefox-ui/tests/functional/private_browsing/test_about_private_browsing.py TestAboutPrivateBrowsingWithSearch.testCheckAboutPrivateBrowsingWithSearch | AssertionError: url bar prepends the @search shortcut
[task 2019-07-12T01:09:08.628Z] 01:09:08 INFO - Traceback (most recent call last):
[task 2019-07-12T01:09:08.630Z] 01:09:08 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-07-12T01:09:08.630Z] 01:09:08 INFO - testMethod()
[task 2019-07-12T01:09:08.631Z] 01:09:08 INFO - File "/builds/worker/workspace/build/tests/firefox-ui/tests/testing/firefox-ui/tests/functional/private_browsing/test_about_private_browsing.py", line 63, in testCheckAboutPrivateBrowsingWithSearch
[task 2019-07-12T01:09:08.632Z] 01:09:08 INFO - 'url bar prepends the @search shortcut')
[task 2019-07-12T01:09:08.632Z] 01:09:08 INFO - TEST-INFO took 6839ms

Pushed by scaraveo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/19419ff4e756 Refactor extension install in searchservice to use promises r=robwu,daleharvey https://hg.mozilla.org/integration/autoland/rev/ca5fdf3a0ddd minimize the chaos of search config testing r=daleharvey

https://hg.mozilla.org/mozilla-central/rev/19419ff4e756
https://hg.mozilla.org/mozilla-central/rev/ca5fdf3a0ddd

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox70: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla70

(sorry for the yelling.)

This fix may be resetting search.json.mozlz4 to its default upon updating to Nightly 2019-07-13 build 20190713095401.

All custom search engines are lost and I haven't found a way to recover them besides a separate backup.

I am still investigating the actual regression but the data loss is confirmed on Reddit's Nightly thread.

Regression range:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ddd8910e7d83e2a2c43395357c9ae889a5a5aa6f&tochange=e712c1b80f6485ce0246c8c380193c51321c5a60

It contains bug 1558944 and bug 1556789. I am pretty sure this bug is causing a data loss.
I am filing a new bug regarding the issue.

Regressions: 1565836

Backed out 2 changesets (Bug 1556789) on request from aryx for causing Bug 1565836.
Backout: https://hg.mozilla.org/mozilla-central/rev/74c0d994aa18fdc4b30d18ecb4d5de95b82298d6

Status: RESOLVED → REOPENED
status-firefox70: fixed → ---
Resolution: FIXED → ---
Target Milestone: mozilla70 → ---
Flags: needinfo?(dharvey)
Flags: needinfo?(dharvey)
Regressions: 1565839

Note that this also caused test failures when running toolkit/components/search/tests/xpcshell/test_validate_engines.js in the Thunderbird environment, see bug 1565841. After the backout, the test no longer fails.

Blocks: 1566117
No longer blocks: 1566117

The backout caused :

== Change summary for alert #21956 (as of Fri, 19 Jul 2019 04:28:39 GMT) ==

Improvements:

2% ts_paint_webext windows7-32-shippable opt e10s stylo 329.42 -> 324.17

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21956

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:mixedpuppy, could you have a look please?
For more information, please visit auto_nag documentation.

Flags: needinfo?(mixedpuppy)

This is being re-thinked.

Flags: needinfo?(mixedpuppy)
Priority: P1 → P2
Priority: P2 → P3
Attachment #9072311 - Attachment is obsolete: true
Attachment #9069699 - Attachment is obsolete: true
No longer blocks: 1559530
Assignee: mixedpuppy → nobody
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: