Closed Bug 1549349 Opened 1 year ago Closed 1 year ago

communicate suggest_url https requirement change to developers

Categories

(WebExtensions :: Developer Outreach, defect, P1)

68 Branch
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: darkspirit, Assigned: caitmuenster)

References

(Regression)

Details

(Keywords: nightly-community, privacy)

Nightly 20190505213908, fresh profile

Open https://addons.mozilla.org/en-US/firefox/addon/bing-homepage-searchengine/
Click on "Add to Firefox"
Red bar is displayed: "Download failed. Please check your connection."

It does neither work with a 2019-05-01 Nightly with xpinstall.signatures.required:false and it only has 87 users.

Info from the console:

1557173356752 addons.webextension.<unknown> ERROR Loading extension 'null': Reading manifest: Error processing chrome_settings_overrides.search_provider.suggest_url: String "http://www.bing.com/osjson.aspx?FORM=U510DF&PC=U510&query={searchTerms}" must match /^https://.$|^$/ Log.jsm:679
append resource://gre/modules/Log.jsm:679
log resource://gre/modules/Log.jsm:360
error resource://gre/modules/Log.jsm:368
_logMessage resource://gre/modules/Extension.jsm:383
logError resource://gre/modules/Extension.jsm:379
packagingError resource://gre/modules/Extension.jsm:366
manifestError resource://gre/modules/Extension.jsm:352
parseManifest resource://gre/modules/Extension.jsm:663
InterpretGeneratorResume self-hosted:1284
AsyncFunctionNext self-hosted:839
1557173356753 addons.xpi WARN Download of https://addons.mozilla.org/firefox/downloads/file/1176291/bing_homepage_and_search_engine-1.0.3.2-fx.xpi?src=dp-btn-primary failed: Error: Extension is invalid(resource://gre/modules/addons/XPIInstall.jsm:426:17) JS Stack trace: loadManifestFromWebManifest@XPIInstall.jsm:426:17
async
loadManifest@XPIInstall.jsm:568:19
async*loadManifest@XPIInstall.jsm:1366:28
onStopRequest@XPIInstall.jsm:2159:14

Maybe a dupe of bug 1549145?

Component: Search → Add-ons Manager
Product: Firefox → Toolkit

I reproduced this in nightly 68 but not release (66.0.4).

I don't think this is related to the signing issues, removing the cert2019 label, we can re-add it if that turns out to be wrong.

Anyway, validation of suggest_urls got stricter in bug 1496075. Dale, this is going to break existing search extensions, what was the rationale for being more strict? If we want to make a change like this, ideally we announce it ahead of time and notify extension authors to give them time to update any affected extensions.

Flags: needinfo?(dharvey)
Whiteboard: [cert2019]

[Tracking Requested - why for this release]:

I think this is a product question first. Do we message/fix extensions on AMO that have http urls in them, or do we remove those
restrictions (except within our builtin extensions).

Flags: needinfo?(mconnor)
Flags: needinfo?(mconca)
Summary: Not possible to add Bing search addon to latest Nightly → search extensions on AMO with http urls can no longer be installed

Anyway, validation of suggest_urls got stricter in bug 1496075. Dale, this is going to break existing search extensions, what was the rationale for being more strict? If we want to make a change like this, ideally we announce it ahead of time and notify extension authors to give them time to update any affected extensions.

Shane was fairly insistent we only supported https here (there was a follow up considering relaxing the https restriction)

Flags: needinfo?(dharvey)

Of course it must be https://, no discussion. ;) Or would anyone want to transmit their search queries via plaintext?
Such betraying/abusive/illegal behavior should be stopped immediately. Otherwise users would trust AMO and lose privacy for another 2 months (until Firefox 68 is released).

  • AMO's error message is misleading: "Download failed. Please check your connection."
  • Broken addons should either be removed from AMO or tagged as broken and not appear in search results by default.
  • Only if tried to install from file we get a more reasonable message: "This addon could not be installed because it appears to be corrupt."
  • If you would add a browser-side workaround to upgrade to https, addon authors might rest on this.
  • Please notify (those few?) search extension authors as soon as possible and/or replace http:// with https:// in their sourcecodes.
Type: defect → task
Has Regression Range: --- → yes
Has STR: --- → yes
Regressed by: 1496075
Keywords: privacy
OS: Linux → All
Hardware: x86_64 → All

(In reply to Dale Harvey (:daleharvey) from comment #8)

Anyway, validation of suggest_urls got stricter in bug 1496075. Dale, this is going to break existing search extensions, what was the rationale for being more strict? If we want to make a change like this, ideally we announce it ahead of time and notify extension authors to give them time to update any affected extensions.

Shane was fairly insistent we only supported https here (there was a follow up considering relaxing the https restriction)

And I still am.

We may have to loosen it for user extensions for some period of time and do some outreach (thus ni? to product), we should not introduce any http in our builtin engines.

Priority: -- → P1
Component: Add-ons Manager → General
Product: Toolkit → WebExtensions
Version: unspecified → Firefox 68

(In reply to Shane Caraveo (:mixedpuppy) from comment #11)

(In reply to Dale Harvey (:daleharvey) from comment #8)

Anyway, validation of suggest_urls got stricter in bug 1496075. Dale, this is going to break existing search extensions, what was the rationale for being more strict? If we want to make a change like this, ideally we announce it ahead of time and notify extension authors to give them time to update any affected extensions.

Shane was fairly insistent we only supported https here (there was a follow up considering relaxing the https restriction)

And I still am.

We may have to loosen it for user extensions for some period of time and do some outreach (thus ni? to product), we should not introduce any http in our builtin engines.

The suggest_url is part of the documented WebExtensions API and has been since Firefox 66, the currently released version of Firefox. Normally we would follow our documented deprecation policy to announce the HTTPS requirement now and phase it in over three releases.

However, as this is a user security and privacy issue, I believe we should short-circuit the normal procedure and allow the current HTTPS requirement ride the trains to release on Firefox 68. We can use the remainder of Nightly and the entire Beta cycle to communicate this to developers.

Flags: needinfo?(mconca)

Re-purposing this bug then to communicate the change.

Component: General → Developer Outreach
Flags: needinfo?(mconnor)
Summary: search extensions on AMO with http urls can no longer be installed → communicate suggest_url https requirement change to developers

MDN documentation updated.

Comms process initiated with :caitmuenster.

Flags: needinfo?(cneiman)

While it looks like this issue is limited to suggest_url, I would make sure any statement is along the line that "all urls related to a search provider are required to be https"

Assignee: nobody → cneiman
Flags: needinfo?(cneiman)

Removing tracking since this was repurposed.

Legal has approved the content of the email.

Email is currently queued up to go out on May 14, 2019.

Email was sent on May 15.

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Version: Firefox 68 → 68 Branch
You need to log in before you can comment on or make changes to this bug.