Last Comment Bug 758857 - Use Wikipedia's HTTPS search by default for Firefox desktop and Android
: Use Wikipedia's HTTPS search by default for Firefox desktop and Android
Status: VERIFIED FIXED
: feature, privacy, relnote
Product: Firefox
Classification: Client Software
Component: Search (show other bugs)
: unspecified
: All All
: -- normal with 4 votes (vote)
: Firefox 35
Assigned To: Andre Klapper
: Camelia Badau, QA [:cbadau]
Mentors:
https://www.wikipedia.org
: 958877 (view as bug list)
Depends on: 1069123
Blocks: 771788
  Show dependency treegraph
 
Reported: 2012-05-26 01:08 PDT by Chris Peterson [:cpeterson]
Modified: 2014-10-03 03:14 PDT (History)
29 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
wontfix
wontfix
+
verified
verified
34+


Attachments
bug-758857-wikipedia-https.patch (5.38 KB, patch)
2012-05-26 01:15 PDT, Chris Peterson [:cpeterson]
gavin.sharp: review+
Details | Diff | Review
wikipedia-https.patch (13.30 KB, patch)
2014-09-15 11:00 PDT, Chris Peterson [:cpeterson]
gavin.sharp: review+
mark.finkle: review+
lmandel: approval‑mozilla‑aurora+
Details | Diff | Review

Description Chris Peterson [:cpeterson] 2012-05-26 01:08:38 PDT
Wikipedia now supports HTTPS at https://www.wikipedia.org, not just https://secure.wikimedia.org.

Would Mozilla need to coordinate with Wikimedia before switching from HTTP to HTTPS?
Comment 1 Chris Peterson [:cpeterson] 2012-05-26 01:15:57 PDT
Created attachment 627450 [details] [diff] [review]
bug-758857-wikipedia-https.patch

This patch changes both Firefox and Mobile's Wikipedia search plugins. Other locales would need to be changed, too.
Comment 2 :Gavin Sharp [email: gavin@gavinsharp.com] 2012-05-27 10:49:26 PDT
Comment on attachment 627450 [details] [diff] [review]
bug-758857-wikipedia-https.patch

Yes, we'll need to coordinate this with them before landing this. Kev can probably help with that.
Comment 3 Chris Peterson [:cpeterson] 2012-05-29 10:54:59 PDT
Thanks, Gavin.

Kev, do you have a contact with Wikimedia's web team? What is the process for coordinating a code change that affects an external org like this?
Comment 4 Amir Aharoni 2012-06-19 10:41:44 PDT
Reply from Ryan Lane, Wikimedia ops:

"Please don't do so. HTTPS is a new service, and we haven't properly
load tested it yet. The first target for production load testing is
for logged-in users.

I'm not opposed to the change completely, but I'd prefer to let you
guys know when we're ready."
Comment 5 Axel Hecht [:Pike] 2012-06-19 10:44:54 PDT
Resolving this WORKSFORME for now, then.

Would you reopen this bug once Ryan gives us a go?
Comment 6 Amir Aharoni 2012-06-19 10:47:20 PDT
(In reply to Axel Hecht [:Pike] from comment #5)
> Would you reopen this bug once Ryan gives us a go?

OK.
Comment 7 Axel Hecht [:Pike] 2012-12-27 12:02:27 PST
It's about half a year, just checking in. Any news here from the wikipedia front, Amir?
Comment 8 Amir Aharoni 2012-12-28 07:39:55 PST
(In reply to Axel Hecht [:Pike] from comment #7)
> It's about half a year, just checking in. Any news here from the wikipedia
> front, Amir?

I pinged. Thanks for the reminder. I'm reopening; I only know about https stuff as a user, but AFAIK it's been for over a year and it's quite stable. Other people may have a different perspective, though.
Comment 9 Axel Hecht [:Pike] 2012-12-28 08:29:35 PST
CCing Ryan, I hope I reverse-engineered the right email address.

Ryan, if you're the wikipedia operations Ryan, are you OK to use https for Firefox search from an infrastructure-load-pov?
Comment 10 Ryan Lane 2012-12-28 09:54:47 PST
We're still waiting on some core changes in MediaWiki to switch logged-in users to HTTPS. I'd prefer to have that happen first.
Comment 11 Axel Hecht [:Pike] 2012-12-28 15:07:00 PST
Hi Ryan, thanks for the update.

Can I leave it up to you to reopen this bug once you're good from the infra side?

I'd leave that as a needinfo in this bug on you. That way, you'll have a reminder in your "My Requests" section. If that ends up sending you whine emails once a week, feel free to cancel that request, I don't know as my whine emails end up in the Trash anyway :-/.

Moving back to RESOLVED and WORKSFORME until we get something actionable.
Comment 12 Chris Peterson [:cpeterson] 2013-08-02 13:36:04 PDT
hi Ryan, I just read your blog post about "The future of HTTPS on Wikimedia projects". [1] That's great news!

When your team is ready for Firefox's search bar to switch to Wikipedia's HTTPS search, just let me know. Mozilla only needs to flip a switch. :)

[1] https://blog.wikimedia.org/2013/08/01/future-https-wikimedia-projects/
Comment 13 Ryan Lane 2013-08-02 13:52:38 PDT
I think we still need a while for this. We've seen a fairly substantial increase in HTTPS traffic and need to make some architecture adjustments. We'll probably need to wait until we move our terminators to our frontends.
Comment 14 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2014-01-11 18:23:26 PST
I filed another bug for this: bug 958877. I was just told about this bug.

It doesn't make sense to resolve a bug WORKSFORME if the problem/enhancement isn't resolved. It makes it harder for people to track the issue and results in duplicate work. Let's either reopen this bug and resolve bug 958877 as a dupe of it, or resolve this bug as a duplicate of bug 958877 and leave bug 958877 open so we can track it. In particular, we need a bug that stays open for the purposes of tracking the overall (unofficial) project to have all these things HTTPS in bug 771788.
Comment 15 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2014-01-11 18:25:10 PST
*** Bug 958877 has been marked as a duplicate of this bug. ***
Comment 16 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2014-01-12 12:07:39 PST
I suspect that the vast majority of people leave the default search provider at Google or whatever they use for general-purpose search. Could we share the numbers for "org.mozilla.searches.counts" / "wikipedia.searchbar" and "wikipedia.contextmenu" here, or with Wikimedia privately? I think if we could give them "the average number of wikipedia searches per day that would be impacted by this change," that would be helpful to them.
Comment 17 Gregory Szorc [:gps] 2014-01-12 18:51:17 PST
mconnor is both the gatekeeper and the keymaster for FHR search data.
Comment 18 Mike Connor [:mconnor] 2014-01-12 21:15:37 PST
Wikipedia was not on the list of engines we were reporting separately (verified that wikipedia searches bucket into "other" in Fx26).  The fix to unfilter those was in bug 925521, but that won't hit release until Fx27 ships in early February.  I'd say that we can't get a decent estimate until February 28th at the earliest (time enough to get a reasonably representative population on Fx27).
Comment 19 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2014-01-12 21:58:27 PST
Thanks Mike. I will NEEDINFO myself as a reminder of this.
Comment 20 Chris Peterson [:cpeterson] 2014-01-13 00:05:41 PST
To determine the number of requests that would be switched to HTTPS, can't Wikipedia just search their server logs for page requests that include Firefox's "Mozilla-search" URL parameter?
Comment 21 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2014-03-10 01:03:23 PDT
Clearing NEEDINFO.
Comment 22 :Ms2ger 2014-04-01 06:17:11 PDT
Hi Ryan, does Wikimedia still want us to hold off?
Comment 23 Andre Klapper 2014-04-08 15:42:44 PDT
:Ms2ger: According to WMF Technical Operations, Wikimedia isn't there yet when it comes to scaling. 
I've posted a longer version in https://bugzilla.wikimedia.org/show_bug.cgi?id=45765#c15

I'll assign this to myself for the time being, until this is sorted out on the Wikimedia side.
Comment 24 :Ms2ger 2014-04-09 06:37:33 PDT
Thanks Andre, much appreciated.
Comment 25 Mike Connor [:mconnor] 2014-09-09 14:32:41 PDT
Andre, is there any updated timelines?  We're considering making SSL a requirement for all bundled search providers, and Wikipedia is one of the bigger question marks.  Is there anything we can do on our end to move this along?
Comment 26 Andre Klapper 2014-09-15 06:16:28 PDT
Hi, thank you for the ping (and patience).

Brought this up again and Wikimedia Operations said you can go ahead and use HTTPS for Wikipedia search.

(For my own records: WMF ops@ list; thread named "Firefox search and HTTPS access to Wikipedia search")
Comment 27 Chris Peterson [:cpeterson] 2014-09-15 11:00:52 PDT
Created attachment 8489505 [details] [diff] [review]
wikipedia-https.patch

Use Wikipedia's HTTPS search by default for Firefox desktop and Android.

Gavin, this is a rebased version of a patch you r+'d in 2012. I'm asking for re-review because I wanted to give you and mfinkle a heads up because this is a small but important product change. The Wikimedia Operations team gave us the OK to move forward in comment 26.
Comment 28 Mark Finkle (:mfinkle) (use needinfo?) 2014-09-15 11:12:04 PDT
Comment on attachment 8489505 [details] [diff] [review]
wikipedia-https.patch

mobile part LGTM
Comment 29 Chris Peterson [:cpeterson] 2014-09-16 18:39:51 PDT
Landed Android change while waiting for Gavin's desktop review.

https://hg.mozilla.org/integration/mozilla-inbound/rev/612fedcc26e1
Comment 30 Ryan VanderMeulen [:RyanVM] 2014-09-17 11:51:13 PDT
https://hg.mozilla.org/mozilla-central/rev/612fedcc26e1
Comment 31 Chris Peterson [:cpeterson] 2014-09-17 14:37:32 PDT
Landed desktop changes:
https://hg.mozilla.org/integration/mozilla-inbound/rev/13c6ae2f5f3e

Andre: this change should land in tomorrow's Firefox Nightly 35 build (with a couple million users) and reach all Firefox users on 2015-01-06. If you see any HTTPS problems on the server side, please let me know. Because this is a small but important change, I would like to uplift this fix to Firefox Aurora 34 (which would reach all Firefox users sooner, on 2014-11-25) if you don't see any problems.
Comment 32 Chris Peterson [:cpeterson] 2014-09-17 14:52:12 PDT
(In reply to Chris Peterson (:cpeterson) from comment #31)
> Andre: this change should land in tomorrow's Firefox Nightly 35 build (with
> a couple million users) ...

Correction: the Nightly user population is only about 250K users. The total prerelease user population (Nightly, Aurora, and Beta) is about 8M.
Comment 33 Masatoshi Kimura [:emk] 2014-09-17 15:10:31 PDT
Is this en.wikipedia only?
Comment 34 Chris Peterson [:cpeterson] 2014-09-17 16:04:57 PDT
(In reply to Masatoshi Kimura [:emk] from comment #33)
> Is this en.wikipedia only?

Thanks for asking! My patch only changed the en-US search plugin in mozilla-central, but (based on my brief testing) I think Wikipedia can support HTTPS for all its *.wikipedia.org properties.

emk: what is the process for letting localizers know that they should update their locale's Wikipedia search plugin?
Comment 35 Chris Peterson [:cpeterson] 2014-09-17 16:53:36 PDT
Release Note Request (optional, but appreciated)
[Why is this notable]: Improved security
[Suggested wording]: Firefox's Wikipedia search now uses HTTPS for secure searching on desktop and Android.
[Links (documentation, blog post, etc)]:
Comment 36 Masatoshi Kimura [:emk] 2014-09-17 17:08:13 PDT
(In reply to Chris Peterson (:cpeterson) from comment #34)
> (In reply to Masatoshi Kimura [:emk] from comment #33)
> > Is this en.wikipedia only?
> 
> Thanks for asking! My patch only changed the en-US search plugin in
> mozilla-central, but (based on my brief testing) I think Wikipedia can
> support HTTPS for all its *.wikipedia.org properties.
> 
> emk: what is the process for letting localizers know that they should update
> their locale's Wikipedia search plugin?

Announcing at dev-l10n?
Comment 37 Chris Peterson [:cpeterson] 2014-09-17 17:34:44 PDT
Thanks. I just posted an announcement on dev-l10n:

https://groups.google.com/d/msg/mozilla.dev.l10n/KL4R3FnN_64/4d9tHa_DbJoJ
Comment 38 Alex Bardas :alexbardas 2014-09-17 18:45:05 PDT
https://tbpl.mozilla.org/?tree=Try&rev=f38e6352d079
Comment 39 Ryan VanderMeulen [:RyanVM] 2014-09-17 19:23:36 PDT
https://hg.mozilla.org/mozilla-central/rev/13c6ae2f5f3e
Comment 40 Federico 2014-09-17 21:54:09 PDT
(In reply to Chris Peterson (:cpeterson) from comment #37)
> Thanks. I just posted an announcement on dev-l10n:
> 
> https://groups.google.com/d/msg/mozilla.dev.l10n/KL4R3FnN_64/4d9tHa_DbJoJ

Thank you. Is there some place where to check progress for the various locales? Is this something where others (native speakers?) can help, or some matter of self-governance where the decision is delegated to localisers?

(In reply to Chris Peterson (:cpeterson) from comment #35)
> Release Note Request (optional, but appreciated)
> [Why is this notable]: Improved security
> [Suggested wording]: Firefox's Wikipedia search now uses HTTPS for secure
> searching on desktop and Android.

Rectius: "English Wikipedia".

Given it's about security, what's the process to update all the addons for Wikipedia's sister projects *.wiktionary.org, *.wikiquote.org etc.? Are https://addons.mozilla.org/firefox/search/?q=wiktionary etc. complete lists or are there other significant distribution channels? If those can't be changed centrally, is there a way to provide "official" options, using HTTPS (and with more complete coverage; plugins are very partial).

Sorry for the many questions... A consistent behaviour for Wikimedia projects would be very appreciated.
Comment 41 Francesco Lodolo [:flod] 2014-09-17 22:39:01 PDT
(In reply to Chris Peterson (:cpeterson) from comment #37)
> Thanks. I just posted an announcement on dev-l10n:
> https://groups.google.com/d/msg/mozilla.dev.l10n/KL4R3FnN_64/4d9tHa_DbJoJ

Next time ask someone from the l10n-drivers before jumping the gun? The guy who usually fixes these bugs (me) would be a good choice ;-)

All productization files (searchplugins, region.properties, etc.) are subject to rules: you can't change anything unless you file a bug and get a patch reviewed, but in some cases we whitelist those changes. 

But even in that case I still need a tracking bug and checks to see which locales out of 114 didn't update the files (and I'll have to do it for most of them).
Comment 42 Chris Peterson [:cpeterson] 2014-09-17 22:56:16 PDT
(In reply to Francesco Lodolo [:flod] from comment #41)
> Next time ask someone from the l10n-drivers before jumping the gun? The guy
> who usually fixes these bugs (me) would be a good choice ;-)

Sorry, Francesco! I was not familiar with Firefox's localization process. I thought dev-l10n was the appropriate venue. I don't see an l10n-drivers mailing list or newsgroup. If there is anything I can do to help now, just let me know.


> But even in that case I still need a tracking bug and checks to see which
> locales out of 114 didn't update the files (and I'll have to do it for most
> of them).

Also, Wikipedia may not support the same 114 locales as Firefox. :)


(In reply to Federico from comment #40)
> Given it's about security, what's the process to update all the addons for
> Wikipedia's sister projects *.wiktionary.org, *.wikiquote.org etc.? Are
> https://addons.mozilla.org/firefox/search/?q=wiktionary etc. complete lists
> or are there other significant distribution channels? If those can't be
> changed centrally, is there a way to provide "official" options, using HTTPS
> (and with more complete coverage; plugins are very partial).

This change only applies to the "search plugins" [1] bundled with Firefox, not third-party search add-ons from AMO. Mozilla does not maintain or bundle search plugins for wiktionary.org or wikiquote.org. Any add-on developers with Wikipedia-related add-ons are responsible for updating their add-ons to Wikipedia HTTPS URLs.

[1] http://kb.mozillazine.org/Search_Bar
Comment 43 Francesco Lodolo [:flod] 2014-09-17 23:00:29 PDT
(In reply to Chris Peterson (:cpeterson) from comment #42)
> Sorry, Francesco! I was not familiar with Firefox's localization process. I
> thought dev-l10n was the appropriate venue. I don't see an l10n-drivers
> mailing list or newsgroup. If there is anything I can do to help now, just
> let me know.

No problem. For future reference (see Contact at the end of the page)
https://wiki.mozilla.org/L10n:Mozilla_Team

> This change only applies to the "search plugins" [1] bundled with Firefox,
> not third-party search add-ons from AMO. Mozilla does not maintain or bundle
> search plugins for wiktionary.org or wikiquote.org. Any add-on developers
> with Wikipedia-related add-ons are responsible for updating their add-ons to
> Wikipedia HTTPS URLs.

We actually ship some Wiktionary searchplugins, but I'd need confirmation from Wikimedia that's fine to use HTTPS for those as well
http://mxr.mozilla.org/l10n-mozilla-aurora/search?string=wiktionary&find=list.txt
Comment 44 Federico 2014-09-17 23:12:03 PDT
(In reply to Francesco Lodolo [:flod] from comment #43)
> We actually ship some Wiktionary searchplugins, but I'd need confirmation
> from Wikimedia that's fine to use HTTPS for those as well
> http://mxr.mozilla.org/l10n-mozilla-aurora/
> search?string=wiktionary&find=list.txt

Traffic on those wiktionaries is negligible, no problem for Wikimedia.
Comment 45 Chris Peterson [:cpeterson] 2014-09-22 23:30:45 PDT
Comment on attachment 8489505 [details] [diff] [review]
wikipedia-https.patch

Approval Request Comment

[Feature/regressing bug #]: New feature, not a regression.

[User impact if declined]: Aurora 34's Wikipedia search bar will continue to use HTTP instead of HTTPS.

[Describe test coverage new/current, TBPL]: Tested on Nightly for 5 days without problems reported by Nightly users or Wikipedia's server ops team.

[Risks and why]: This change has been a long time in the making. Firefox's search bar uses HTTPS for Google, Yahoo, and Twitter (but HTTP for Amazon, Bing, and eBay). Wikipedia has supported HTTPS for years, but their server ops team asked us to hold our change until their server infrastructure was ready for more HTTPS users. They say they are now ready (comment 26).

[String/UUID change made/needed]: I made the following string changes and have communicated them to l10n drivers (bug 1069123):
1. Changed Wikipedia search URL from http to https.
2. Capitalized the word "free" in "Free Encyclopedia" like Wikipedia's logo does.
Comment 46 Lawrence Mandel [:lmandel] (use needinfo) 2014-09-23 08:10:19 PDT
Comment on attachment 8489505 [details] [diff] [review]
wikipedia-https.patch

Thank you for working with l10n on this. Given that l10n is already working on the changes, this is approved for Aurora.
Comment 47 Ryan VanderMeulen [:RyanVM] 2014-09-23 12:39:33 PDT
https://hg.mozilla.org/releases/mozilla-aurora/rev/44c19edf82f3
Comment 48 Camelia Badau, QA [:cbadau] 2014-09-25 06:42:35 PDT
Firefox Desktop: verified on Windows 7 64bit, Ubuntu 13.10 32bit and Mac OS X 10.9.5 using:
                 - latest Nightly 35.0a1 (buildID: 20140924030204) - it works ok
                 - latest Aurora 34.0a2 (buildID: 20140925004001) - one mention should be done here: if I write "wikipedia.org" on the url bar and then press Enter -> the opened page doesn't use https ("http://www.wikipedia.org/") - this is ok? 

Firefox Android: - verified on build 35.0a1 (2014-09-24) - it works ok 
                 - verified on build 34.0a2 (2014-09-25) - it doesn't work. Wikipedia search bar continue to use HTTP instead of HTTPS. The fix for this bug is also on Aurora 34.0a2 (2014-09-25) on Android?
Comment 49 Chris Peterson [:cpeterson] 2014-09-25 09:13:00 PDT
(In reply to Camelia Badau, QA [:cbadau] from comment #48)
> Firefox Desktop: verified on Windows 7 64bit, Ubuntu 13.10 32bit and Mac OS
> X 10.9.5 using:
>                  - latest Nightly 35.0a1 (buildID: 20140924030204) - it
> works ok
>                  - latest Aurora 34.0a2 (buildID: 20140925004001) - one
> mention should be done here: if I write "wikipedia.org" on the url bar and
> then press Enter -> the opened page doesn't use https
> ("http://www.wikipedia.org/") - this is ok? 

That is the expected behavior. This change only affects the search bar, not the address bar. Entering "http://www.wikipedia.org/" in the address bar is not expected to redirect to "https://www.wikipedia.org/".

> Firefox Android: - verified on build 35.0a1 (2014-09-24) - it works ok 
>                  - verified on build 34.0a2 (2014-09-25) - it doesn't work.
> Wikipedia search bar continue to use HTTP instead of HTTPS. The fix for this
> bug is also on Aurora 34.0a2 (2014-09-25) on Android?

Oops! You are correct. This was an oversight.

My patch on this bug included both Firefox desktop and Android changes, but I split the patch into separate Android and desktop changesets (comment 29 and comment 31, respectively) when I landed (because the Android and desktop changes were r+'d at different times). Ryan's uplift (comment 47) only caught the desktop changeset.

I just uplifted the Android change to mozilla-aurora, so the next Android build 34.0a2 should work correctly:

https://hg.mozilla.org/releases/mozilla-aurora/rev/36f2693ea7c1
Comment 50 Camelia Badau, QA [:cbadau] 2014-09-26 06:02:07 PDT
Thank you for your answer, Chris! 

Verified on Firefox Android on build 34.0a2 (2014-09-26) and it works ok now.
Comment 51 Sylvestre Ledru [:sylvestre] 2014-10-03 03:14:53 PDT
Added to the release notes with "Wikipedia search now uses HTTPS for secure searching" as wording.

Note You need to log in before you can comment on or make changes to this bug.