1122305 - Make use of mozAnon / loadFlags LOAD_ANONYMOUS when checking for updates and when downloading an update

Status: NEW

(Toolkit :: Application Update, defect, P3)

Description

[Robert Strong (they/them - no direct email)]

This will prevent the use of cookies and lessen privacy concerns.

Comment 1

[Mike Connor [:mconnor]]

At some point AUS (or blocklist) used cookies to avoid double counting for metrics purposes.  It's worth making sure we don't break this stuff.

Comment 2

[Robert Strong (they/them - no direct email)]

Chris, fyi that I'd like to prevent cookies from being used by the update check and the update download to lessen privacy concerns. If you aren't the right person to notify please let me know. Thanks!

Comment 3

[Robert Strong (they/them - no direct email)]

Chris ^^

Comment 4

[Chris More [:cmore]]

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #2)
> Chris, fyi that I'd like to prevent cookies from being used by the update
> check and the update download to lessen privacy concerns. If you aren't the
> right person to notify please let me know. Thanks!

Hi Rob.

Does anyone do anything with the cookies now? What domain sets/reads the cookie now?

Comment 5

[Robert Strong (they/them - no direct email)]

Hi Chris, here is info posted to governance. Many years ago we used cookies for tracking update pings and I was told they would be removed. I have no idea if they are still used or who would be using them but they were used by metrics.

Typical url:
https://aus4.mozilla.org/update/3/Firefox/38.0a1/20150113030205/WINNT_x86-msvc/en-US/nightly/Windows_NT%206.3.0.0%20(x64)/default/default/update.xml?force=1

"Please remember that you still send cookies; here's what I got out of Firefox debugging itself as I went to Help -> About:

optimizelySegments=%7B%22245875585%22%3A%22direct%22%2C%22245617832%22%3A%22none%22%2C%22246048108%22%3A%22false%22%2C%22245677587%22%3A%22ff%22%2C%22869421433%22%3A%22true%22%7D; optimizelyEndUserId=oeu1421293036707r0.8592334519134582; optimizelyBuckets=%7B%7D; __utma=150903082.1914521133.1421293040.1421293040.1421293040.1; __utmb=150903082.2.10.1421293040; __utmc=150903082; __utmz=150903082.1421293040.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1

So it's got Google Analytics and Optimizely; both are for tracking.

Steps:
1) New download of latest release, 35.0
2) Start with firefox -profile (empty directory) -offline
3) Turn on browser + remote debugging; open the browser toolbox in Network tab.
4) Click on Try Again in the first run page to go online and trigger things
5) Help -> About Firefox.

I have not checked the addon update ping; that presumably has similar behaviour.  Being privacy-oriented there would likely involve fetching updates for each addon separately over a period of time to avoid the ability to track people by the combination of addons they have installed.

Quick scan of things with timestamps in prefs: app update; addon update; telemetry; FHR; sync; openh264 / gmp; safebrowsing; phishing.  Not sure if things like social that I've disabled involve pings.

When trading between user privacy and designing a better web site, user privacy lost.  (It should be obvious, but: I believe the right choice here would be a request with no identifying information beyond the build of Firefox in use, the OS it's running on, and the source IP address so it can send the response.)"

Comment 6

[:Mook]

Hi! Sorry, there seems to be some confusion. (I was the person to provide the info in comment 5.)

The update service itself does not set any cookies; however, the firstrun page pulls in Google analytics (and, judging from the cookies above, optimizely) which sets them for mozilla.org across the whole TLD.  They're not likely to be something you're actively looking for (since you didn't set them in the first place, www.m.o did), but it's still there and needlessly raises privacy concerns.

Comment 7

[Robert Strong (they/them - no direct email)]

Thanks for the additional info Mook.

Comment 8

[Chris More [:cmore]]

(In reply to :Mook from comment #6)
> Hi! Sorry, there seems to be some confusion. (I was the person to provide
> the info in comment 5.)
> 
> The update service itself does not set any cookies; however, the firstrun
> page pulls in Google analytics (and, judging from the cookies above,
> optimizely) which sets them for mozilla.org across the whole TLD.  They're
> not likely to be something you're actively looking for (since you didn't set
> them in the first place, www.m.o did), but it's still there and needlessly
> raises privacy concerns.

Ok, this makes more sense as I was confused about this bug and the summary. 

Yes, www.mozilla.org (among 70 other websites at Mozilla) use Google Analytics premium to understand how our websites are working, to find problems, and to improve the user experience. Our Google Analytics premium account is set to opt-out on all of 3rd party uses of the data and the only people who have access to the anonymous aggregated data is Mozilla Employees. This is not the normal Google Analytics setup that most people use on other websites. 

Also, to increase privacy we flipped the anonymize flag in the Google Analytics request via bug 946705 since that is not something enabled by default. We also create a single Google Analytics web property per sub-domain (www.mozilla.org, developer.mozilla.org, support.mozilla.org, etc.) and don't do any cross-domain cookies within Google Analytics. Google Analytics on www.mozilla.org is set to the www sub-domain and not the entire TLD of mozilla.org.

As for Optimizely, we use that for A/B testing on key www.mozilla.org pages (as do other Mozilla teams on their websites) to test out theories on improvements to interfaces and content that resonates better with our visitors.

Let me know if you have any other questions.

Comment 9

[:Gavin Sharp [email: [e-mail]]]

The question that needed answering, specifically, is "does aus4.mozilla.org (or any other AUS back-end) read any cookies from incoming requests ever?". Sounds like the answer is "not as far as Chris knows", but probably someone from the release engineering team (who administer the update servers) can say conclusively.

Comment 10

[Robert Strong (they/them - no direct email)]

The cookies in the past were used by metrics and the cookies at this time are set by Google Analytics and Optimizely across mozilla.org. Is releng reading these cookies from Google Analytics and Optimizely?

Comment 11

[[e-mail] (:bhearsum)]

(In reply to :Gavin Sharp [email: [e-mail]] from comment #9)
> The question that needed answering, specifically, is "does aus4.mozilla.org
> (or any other AUS back-end) read any cookies from incoming requests ever?".
> Sounds like the answer is "not as far as Chris knows", but probably someone
> from the release engineering team (who administer the update servers) can
> say conclusively.

AUS4 (Balrog) definitely does not. We look at the URL, query args, and User-Agent header: https://github.com/mozilla/balrog/blob/master/auslib/web/views/client.py. Requests hit a load balance before the app, but I highly doubt it's looking at cookies. We can ask WebOps to confirm if you like.

AUS3 (which is basically unused at this point) seems to set a cookie for a reason that I don't understand: http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/index.php#64. https://bugzilla.mozilla.org/show_bug.cgi?id=481313 has a small amount of background, I'm still not sure what it was for though.

Comment 12

[Robert Strong (they/them - no direct email)]

I would be surprised if this wasn't the metrics app update cookie but the people that I talked with about the app update cookie (Ken Kovacs and Daniel Einspanjer) are no longer at Mozilla.

Comment 13

[Chris AtLee [:catlee]]

We've never used it AFAIK. Nick may have more information.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

The update server has never used the cookie to change the response, and I think it must have been for metrics. Also, I think it's been broken for a long time, as the cookie is set for aus2.mozilla.org and we moved to aus3.m.o years ago. See bug 1122814 comment #1 for the value of COOKIE_DOMAIN, to go with http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/index.php#64.

Comment 15

[Chris More [:cmore]]

Hi everyone. A quick related update.

* I re-opened bug 1000504 because I didn't like how the Optimizely opt-out was mentioned in the websites privacy policy. It could be made easier for users and thus I suggested changes to make it one-click for www.mozilla.org.

* I opened bug 1125995 to make Optimizely's cookies to be specific to the sub-domain instead of associated with the mozilla.org root domain. The root domain setup is the default behavior in Optimizely and the code mentioned in that bug will improve that setup.

* I flipped the bit on each optimizely project to have this turned on by default "Anonymize IP addresses for this project and remove the last octet of IP addresses prior to logging"

Comment 16

[Chris More [:cmore]]

(In reply to Nick Thomas [:nthomas] from comment #14)
> The update server has never used the cookie to change the response, and I
> think it must have been for metrics. Also, I think it's been broken for a
> long time, as the cookie is set for aus2.mozilla.org and we moved to
> aus3.m.o years ago. See bug 1122814 comment #1 for the value of
> COOKIE_DOMAIN, to go with
> http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/index.php#64.

Interesting. I never knew the legacy aus server did anything with cookies. Since aus2 is gone, is this bug invalid?

Comment 17

[:Gavin Sharp [email: [e-mail]]]

No, this bug is about stopping the client from sending cookies. First we needed to confirm no one was depending on them being there. I think we've done that, so now we should go ahead and change Firefox to not send the cookies.

(Good to hear that you adjusted the Optimizely cookie setting for mozilla.org, that seems like a good idea anyways.)

Comment 18

[Chris More [:cmore]]

(In reply to :Gavin Sharp [email: [e-mail]] from comment #17)
> No, this bug is about stopping the client from sending cookies. First we
> needed to confirm no one was depending on them being there. I think we've
> done that, so now we should go ahead and change Firefox to not send the
> cookies.
> 
> (Good to hear that you adjusted the Optimizely cookie setting for
> mozilla.org, that seems like a good idea anyways.)

+1

Comment 19

[Robert Strong (they/them - no direct email)]

Attachment: patch rev1

Might as well just take care of this.

Comment 20

[Robert Strong (they/them - no direct email)]

I'm not going to be able to look into this more anytime soon so unassigning

Comment 21

[rahul.rajesh.bhat]

Hi,

I am looking to contribute to mozilla.
Possible to pick up this issue?

Comment 22

[Robert Strong (they/them - no direct email)]

I don't think this is a good first bug due to there being issues with LOAD_ANONYMOUS that will need to be researched before this bug can be fixed.
https://groups.google.com/forum/#!topic/mozilla.dev.platform/pxBCKzxBMDA

Comment 23

[<i>Tester</i>]

<i>tester</i>

Comment 24

[<i>Tester</i>]

Attachment: image.png.svg

clickkk

Comment 25

[<i>Tester</i>]

Attachment: aaaaaaa<i>aaaa</i>

dsadasd

Comment 26

[<i>Tester</i>]

Attachment: zZZZ

sssS

Comment 27

[<i>Tester</i>]

Attachment: CCCCCC

ASDASD

Comment 28

[<i>Tester</i>]

Attachment: AAAAA

Comment 29

[<i>Tester</i>]

Attachment: qqqqq.svg

sdasda

Comment 30

[<i>Tester</i>]

Attachment: xxxxxx

asxxxxxxxxx

Comment 31

[<i>Tester</i>]

Attachment: pppppp

zzzzz

Comment 32

[testingweb]

Attachment: CLICKHEREEEEEEEE

hello i am hacker

Comment 33

[tetrerer]

Attachment: clickmeeeeee

i am hacker