1019255 - Enable certificate pinning on Fennec

Status: RESOLVED FIXED

(Core :: Security: PSM, defect)

Description

[[:mmc] Monica Chew (no longer reading bugmail)]

in 33 or 34.

Comment 1

[[:mmc] Monica Chew (no longer reading bugmail)]

See bug 1012882 for why we did this.

Comment 2

[[:mmc] Monica Chew (no longer reading bugmail)]

Hey Mark,

Desktop will have certificate pinning enabled starting in 32, with no major problems thus far:

https://wiki.mozilla.org/SecurityEngineering/Public_Key_Pinning
http://people.mozilla.org/~mchew/pinning_dashboard/

Current nightly's in 34. Time to enable it for Fennec? It would mean that any chemspills for pinning would have to be done for Fennec as well (see https://bugzilla.mozilla.org/show_bug.cgi?id=1012882#c6 and c7)

Thanks,
Monica

Comment 3

[Mark Finkle (:mfinkle) (use needinfo?)]

I had no real background on "pinning" so I had to read the Wiki link to get some context. This seems like something we could turn on in Fennec, but maybe wait for Fx35 to start since I let this stall until the end of the cycle. We could uplift the changes to Fx34 once we see it working OK.

CC'ing Brad and Richard for any thoughts too.

Comment 4

[Richard Newman [:rnewman]]

Concur with mfinkle's 35/34 plan.

One thing to note: a significant amount of Fennec network traffic does not use Necko. We fetch updates and favicons, upload stumbler data, perform syncs and FxA operations, and upload FHR data (and maybe others) using pure-Java networking, often without Gecko running.

Assumptions about the benefits of pinning need to be considered in this light -- e.g., "so that our users know that the updates we serve actually come from us" -- and potentially more work needs to be done to extend those benefits to different network stacks.

I have no idea if Android's certificate handling code (which the Java stacks will use) already includes pinning on recent versions.

Comment 5

[Camilo Viecco (:cviecco)]

> 
> One thing to note: a significant amount of Fennec network traffic does not
> use Necko. We fetch updates and favicons, upload stumbler data, perform
> syncs and FxA operations, and upload FHR data (and maybe others) using
> pure-Java networking, often without Gecko running.

This comes as a huge surprise to me. Are there any technical reasons why is this done
this way and not using Necko? Is there a plan to migrate to using Necko?

Comment 6

[Camilo Viecco (:cviecco)]

oops, can you answer comment 5 please?

Comment 7

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Mark Finkle (:mfinkle) from comment #3)
> I had no real background on "pinning" so I had to read the Wiki link to get
> some context. This seems like something we could turn on in Fennec, but
> maybe wait for Fx35 to start since I let this stall until the end of the
> cycle. We could uplift the changes to Fx34 once we see it working OK.
> 
> CC'ing Brad and Richard for any thoughts too.

I'm more bullish on this and wouldn't be opposed to landing it at the end of the 34 cycle. My assumption is that we won't get meaningful testing until beta anyway

Comment 8

[Richard Newman [:rnewman]]

(In reply to Camilo Viecco (:cviecco) from comment #5)

> This comes as a huge surprise to me. Are there any technical reasons why is
> this done this way and not using Necko? Is there a plan to migrate to using Necko?

Lots and lots of reasons. See Bug 507641 Comment 27 (and the rest).

There are two different concepts here: Necko and Gecko. The idea of using just Necko -- i.e., building some separate small .so that's cheap to link and use from Java -- is attractive. Alas, it's somewhat stymied by the need to have a profile directory (where do all of the prefs come from?), and the general intertwingledness of everything in mozilla-central.

Gecko has a long startup time (5-30 seconds on current phones), huge memory consumption that causes it to be killed by Android, and other complexities that hamper its use in background operations.

In order to save our sanity, save the user's battery, make background services possible on low-end hardware, and more natively integrate with Android's patterns, we use Gecko for browsing the web, and Java for everything else that we can.

Ideally we will move *more* features out of Gecko, where 'background' work tends to be done while the user is browsing the web, and into background services where they can be efficiently scheduled in suitable downtime. (Headless Gecko is an option for this, but Gecko remains a huge memory-stomping rhino.)

I would like to resolve this separation, but the only solution I think is adequate for all of our supported platforms is for the networking team to produce a fast, compact libnecko.so that can be successfully used from multiple threads via JNI, with *concurrent access to the user's profile* shared by the Gecko thread.

In the absence of that pipe-dream, we're instead attacking the main gaps (e.g., parsing Gecko's proxy settings and using those in Java).

Comment 9

[[:mmc] Monica Chew (no longer reading bugmail)]

Attachment: Enable pinning on fennec (

Comment 10

[[:mmc] Monica Chew (no longer reading bugmail)]

https://tbpl.mozilla.org/?tree=Try&rev=4025b13e1e75

Comment 11

[Richard Newman [:rnewman]]

Comment on attachment 8483588 [details] [diff] [review]
Enable pinning on fennec (

Review of attachment 8483588 [details] [diff] [review]:
-----------------------------------------------------------------

Assuming that try build comes back green, and this doesn't make autophone scream, roll on!

Comment 12

[[:mmc] Monica Chew (no longer reading bugmail)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/1b4ba4d597ed

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/1b4ba4d597ed

Comment 14

[Richard Newman [:rnewman]]

ni on Monica as a reminder to request uplift after bake time.

Comment 15

[[:mmc] Monica Chew (no longer reading bugmail)]

I put it on my calendar in 3 weeks time. Btw, do fennec results get reported together with desktop on telemetry.mozilla.org? Because that would be really terrible.

Comment 16

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to [:mmc] Monica Chew (please use needinfo) from comment #15)
> I put it on my calendar in 3 weeks time. Btw, do fennec results get reported
> together with desktop on telemetry.mozilla.org? Because that would be really
> terrible.

To see Fennec data on telemetry.mozilla.org, you need to choose, "saved_session" and then "Fennec"

Comment 17

[Richard Newman [:rnewman]]

E.g., 

http://telemetry.mozilla.org/#filter=nightly%2F35%2FCERT_PINNING_MOZ_RESULTS_BY_HOST%2Fsaved_session%2FFennec%2FAndroid&aggregates=multiselect-all!Submissions&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph

Comment 18

[[:mmc] Monica Chew (no longer reading bugmail)]

Thanks! Jonas, how can I do comment 17 in telemetry.js? Current dashboard code is in https://github.com/monicachew/pinning-dashboard

Comment 19

[Jonas Finnemann Jensen (:jonasfj)]

@mmc,
When you have loaded a histogram evolution you'll often want to filter it (see [1]).
You can do this using:
  Telemetry.HistogramEvolution.filterName
  Telemetry.HistogramEvolution.filterOptions
  Telemetry.HistogramEvolution.filter

Ideally, you shouldn't rely on what the filters are called or what order they are in, as telemetry.js doesn't promise any of that. This leaves you with two options:

Option A: Quick and dirty, rely on ordering of filters
Just, do the filtering after loading the measures:

  hgramEvo = hgramEvo.filter('saved_session').filter('Fennec');

If you choose the option A, you should add a few sanity checks.
Just make sure that "saved_session" is an option form filterOptions() and filterName() is "reason", and similar with the "Fennec" filtering step. Just, so you can show the user an error/alert if/when this hack no longer works.

Note, you really should almost always filter with 'saved_session', right now we don't aggregate 'idle_daily', and I don't think we're going to. But for sanity it's good to filter down to exactly the data you want.

Option B: Reuse/hack telemetry.jquery.js to work for you
You can steal and hack the filtering logic we have for telemetry-dashboard, see [2]. But that might be slightly complicated. Using CSS to hide the measure filter is possible, but somewhat sketchy too :)
And when loading your additional measures, you would need to validate the assumption that they have the same filters, and apply these filters. But it would allow you to filter deeper, as in filter on OSes, etc.

References:
[1] http://telemetry.mozilla.org/docs.html#Telemetry.HistogramEvolution.filter
[2] https://github.com/mozilla/telemetry-dashboard/blob/master/src/jquery.telemetry.js

Comment 20

[[:mmc] Monica Chew (no longer reading bugmail)]

Comment on attachment 8483588 [details] [diff] [review]
Enable pinning on fennec (

Review of attachment 8483588 [details] [diff] [review]:
-----------------------------------------------------------------

rnewman, blassey: numbers look good. Are you still bullish on uplifting to Aurora for 34?

Comment 21

[[:mmc] Monica Chew (no longer reading bugmail)]

8 failures of 4M checks since this was enabled in Nightly, other pinning metrics look similar

http://telemetry.mozilla.org/#filter=nightly%2F35%2FCERT_PINNING_RESULTS%2Fsaved_session%2FFennec%2FAndroid&aggregates=multiselect-all!Submissions&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Table

Comment 22

[Richard Newman [:rnewman]]

(In reply to [:mmc] Monica Chew (please use needinfo) from comment #20)

> rnewman, blassey: numbers look good. Are you still bullish on uplifting to
> Aurora for 34?

I approve.

Comment 23

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Richard Newman [:rnewman] from comment #22)
> (In reply to [:mmc] Monica Chew (please use needinfo) from comment #20)
> 
> > rnewman, blassey: numbers look good. Are you still bullish on uplifting to
> > Aurora for 34?
> 
> I approve.

yes

Comment 24

[[:mmc] Monica Chew (no longer reading bugmail)]

Somehow this didn't show up in comment 20.

Approval Request Comment
[Feature/regressing bug #]:https://bugzilla.mozilla.org/show_bug.cgi?id=1004350
[User impact if declined]: Pinning won't be available on Fennec until FF 35. Desktop had this in 32.
[Describe test coverage new/current, TBPL]: telemetry
[Risks and why]: Pretty low based on the metrics, pinning is working similar to desktop with very little breakage. If we have to chemspill or hotfix for pinning this expands the work needed to include fennec.
[String/UUID change made/needed]: None.

Comment 25

[Lawrence Mandel [:lmandel] (use needinfo)]

Comment on attachment 8483588 [details] [diff] [review]
Enable pinning on fennec (

Aurora+

Comment 26

[Lawrence Mandel [:lmandel] (use needinfo)]

Monica - Can you confirm that we want to call this out in the release notes like we did on desktop in 32?

https://www.mozilla.org/en-US/firefox/32.0/releasenotes/

Comment 27

[[:mmc] Monica Chew (no longer reading bugmail)]

(In reply to Lawrence Mandel [:lmandel] from comment #26)
> Monica - Can you confirm that we want to call this out in the release notes
> like we did on desktop in 32?
> 
> https://www.mozilla.org/en-US/firefox/32.0/releasenotes/

Yes, we should call it out in the release notes.

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/3ad931b42b9d

Comment 29

[Sylvestre Ledru [:Sylvestre]]

Added to the release notes. I reused the note that we had for desktop.