1199014 - Reduce Shavar Costs for Tracking Protection in PBM

Status: RESOLVED FIXED

(Firefox :: Private Browsing, defect)

Description

[Erin Lancaster [:elan]]

Services Ops has found TP/PBM server costs are significant and in order to reduce them, Server Team plans to:
(1) Move to download full lists and updates from a CDN
(2) Only update when there’s actually a change upstream, instead of every 30 minutes

ETA for these things= Prior to Beta Merge Sept 21

Services Ops requests Platform Sec evaluates the following Client optimizations for Fx42: 
(1) Poll Shavar less often (1x daily)
(2) Can we download the full list only upon first entry into PBM vs. all users?
(related to bug 1196989). 
(3) Currently we download a copy of the list for each profile. Can we share across profiles, instead? 
NB: Laura is exploring an option with RelEng to bundle an initial list with the Fx42 release in case the answer to (2) is "no".

Comment 1

[François Marier [:francois]]

(In reply to Erin Lancaster [:elan] from comment #0)
> Services Ops requests Platform Sec evaluates the following Client
> optimizations for Fx42: 
> (1) Poll Shavar less often (1x daily)

The polling interval is controlled by the server:

  https://github.com/mozilla-services/shavar/issues/51

> (2) Can we download the full list only upon first entry into PBM vs. all
> users?
> (related to bug 1196989).

If we are OK with disclosing to local users the fact that PBM has been used before, we could do:

  https://bugzilla.mozilla.org/show_bug.cgi?id=1196989#c1

If we think that's too much of a privacy leak, then I proposed an alternate scheme which would reduce the initial load at least:

  https://bugzilla.mozilla.org/show_bug.cgi?id=1196989#c9

> (3) Currently we download a copy of the list for each profile. Can we share
> across profiles, instead?

I haven't seen a precedent for something like this. It doesn't look like we store anything outside of the profile directory. Do we have any reasons to believe that there are enough users using multiple profiles for this to make a meaningful difference?

Comment 2

[Javaun Moradi [:javaun]]

(In reply to Erin Lancaster [:elan] from comment #0)
> Services Ops has found TP/PBM server costs are significant and in order to
> reduce them, Server Team plans to:
> (1) Move to download full lists and updates from a CDN
> (2) Only update when there’s actually a change upstream, instead of every 30
> minutes

Right. By our new plan, we're going to move our canonical list to Github, manually approve updates, and pull from Github and not the list URL. Manual approval means we control the frequency of list changes. 


> ETA for these things= Prior to Beta Merge Sept 21
> 
> Services Ops requests Platform Sec evaluates the following Client
> optimizations for Fx42: 
> (1) Poll Shavar less often (1x daily)

Does polling frequency matter if we're not actually updating the list? If (as above) we have version controlled lists? 

> (2) Can we download the full list only upon first entry into PBM vs. all
> users?
> (related to bug 1196989). 
Agree w/ Francois. This is a privacy leak. Beyond that, I'm not sure it's significant savings. Beyond normal PBM users, we're going to be promoting PBM within the browser and outside the browser. We'll have new people visiting it at least once. At best you'd see 2-3x savings. The real bang for the buck is going to be incremental.

> (3) Currently we download a copy of the list for each profile. Can we share
> across profiles, instead? 
Also agree w/ Francois: I don't know how common a use case this really is.

> NB: Laura is exploring an option with RelEng to bundle an initial list with
> the Fx42 release in case the answer to (2) is "no".
I think bundling the initial list is an idea worth exploring regardless!

Comment 3

[Javaun Moradi [:javaun]]

I'm so unclear today! Did not mean to imply 2-3x is "incremental", I meant that the real bang for the buck in reducing bandwidth is going to be incremental list updates. i.e. instead of sending the full 350KB entity list or 50KB blocklist, just send a few KB for the list changes.

Comment 4

[Erin Lancaster [:elan]]

Decisions made! We are going to do the following server and client changes by GTB for Beta 1 on Sept 21:

Services:
1) List generation scripts only update Block and Entity White Lists when there is a change upstream (vs every 30 min)

2) Shavar Server: Change polling to 1x per hour for Beta 1 and move upward to 1x per day as needed (anything except for 45 min). Client respects server polling intervals.

Firefox Client:
(1) Include Block List and Entity White List in the installer payload for Fx42+ (we need tickets tracking this work, this impacts installer, build system, and we need RelMan awareness to figure out a way to manage updates to list)

Process Needing definition, more details soon:
1) Path to Production: Testing for List Updates when reviewed and sanctioned by Javaun
2) What the SOP is when we get a bad entry

Comment 5

[Erin Lancaster [:elan]]

Need a ticket filed to cover the RelEng work that covers (1) under Firefox Client. Ben, I'm hoping you can help us with this. Many thanks.

Comment 6

[[e-mail] (:bhearsum)]

(In reply to Javaun Moradi [:javaun] from comment #2)
> > NB: Laura is exploring an option with RelEng to bundle an initial list with
> > the Fx42 release in case the answer to (2) is "no".
> I think bundling the initial list is an idea worth exploring regardless!

I spoke with Laura about this briefly last week and suggested that if there's an in-tree file that gets packaged with Firefox, we might be able to update it on a regular basis like we do with the blocklist, eg: http://hg.mozilla.org/releases/mozilla-esr38/rev/cd0293592d00

Note that RelEng doesn't own any of the build system bits here though - that part needs to exist before we can do anything. This would have to go through other channels for prioritization too - I can't volunteer myself for this given the existing work on my plate.

Comment 7

[Erin Lancaster [:elan]]

:mossop, we're needing a consultation on who could help us with getting the block and white lists for tracking protection bundled (and updated) akin to: http://hg.mozilla.org/releases/mozilla-esr38/rev/cd0293592d00.

We are aiming to resolve this be Sept 21st GTB for Firefox 42.0 Beta 1. Many thanks for any leads/advice.

Comment 8

[Dave Townsend [:mossop]]

Tim, this seems like something that should fall into your teams remit?

Comment 9

[Erin Lancaster [:elan]]

Update on other tasks above: 

Services:
1) List generation scripts only update Block and Entity White Lists when there is a change upstream (vs every 30 min)

This is done and deployed as of this afternoon per Francois (thank you rtilder). 

2) Shavar Server: Change polling to 1x per hour for Beta 1 and move upward to 1x per day as needed (anything except for 45 min). Client respects server polling intervals.

This should be fixed as part of 0.6.5 which is slated to go to Stage on Thursday.

Comment 10

[Javaun Moradi [:javaun]]

Here are the top-level questions we need to discuss in desktop standup tomorrow Tim: 
https://etherpad.mozilla.org/C6SKGwcRNg

Comment 11

[Tim Taubert [:ttaubert] (inactive)]

(In reply to Dave Townsend [:mossop] from comment #8)
> Tim, this seems like something that should fall into your teams remit?

Yes, at least something we should track. Thanks.

Comment 12

[Erin Lancaster [:elan]]

It sounds like we don't need to do the work outlined in comment 6 and comment 7 per conversations with a few people today. One time change and it doesn't support change rate. Need final sign off from dcamp on this. He's also proxy for Laura. :dcamp, are we good de-scope bundling the lists in the binary from Fx42.0?

Comment 13

[Erin Lancaster [:elan]]

All the things in Comment 9 are working so I think after we have resolution on the above. We can close this ticket out as 'fixed'.

Comment 14

[Erin Lancaster [:elan]]

Probably don't need formal sign off on decision to not bundle lists. The collective agrees. :) The thing we need to decide is polling interval for Server (and therefore client). 1 hour as of Beta 1 and it may change prior to GA. I will file a ticket to track that specific decision. 

Thank you, everyone for all of the help!