1512451 - Read certificates revocation list from security-states/onecrl instead of blocklists/certificates

Status: RESOLVED FIXED

(Toolkit :: Blocklist Implementation, enhancement, P3)

Description

[Mathieu Leplatre [:leplatrem]]

In the RemoteSettings client instantiation, the bucketName should now be a new preference with the value "security-state":

https://searchfox.org/mozilla-central/rev/69f9d5002c6e3c5c571a348916fb174e6a7b4acd/services/common/blocklist-clients.js#177

And this preference value changed to "onecrl":

https://searchfox.org/mozilla-central/rev/adec563403271e78d1a057259b3e17fe557dfd91/modules/libpref/init/all.js#2732

Comment 1

[J.C. Jones [:jcj] (he/they)]

(In reply to Mathieu Leplatre [:leplatrem] from comment #1)
> Until we declare security-state/onecrl as the new source, the security team
> would have to publish their changes in both places.

Note, we are not keeping the new and old buckets in-sync at this time. The permissions changes between the buckets yield enough breakage in the tooling that we will have to dedicate time to fixing them after the holidays.

Comment 2

[Mathieu Leplatre [:leplatrem]]

> Note, we are not keeping the new and old buckets in-sync at this time. The permissions changes between the buckets yield enough breakage in the tooling that we will have to dedicate time to fixing them after the holidays.

I'm sorry to hear that. Let's get back to this in January. We don't have OPs 'till then anyway...

Comment 3

[Mathieu Leplatre [:leplatrem]]

Part of this ticket, we should also remove the JSON dump blocklist/certificates.json and add security-state/onecrl.json.

Comment 4

[Mathieu Leplatre [:leplatrem]]

Attachment: Bug 1512451 - Read OneCRL blocklist from security-states/onecrl

Read OneCRL blocklist from security-states/onecrl

Comment 5

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6842a3a277adbece0dcf844e5a90ea3af334475c

Comment 6

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=324059e4c6169174d8768517d68b669147aebd63

Comment 7

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fda4e785d22483854aeda1e71c5ef9163f78efea

Comment 8

[Mathieu Leplatre [:leplatrem]]

Mark, could you please give a final r+ so that we can ship this in 68? It would be nice not to have another ESR pulling from the legacy endpoint.
Thanks!

Comment 9

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a766031441805ce7f08ed20751d86c4d197e3e74

Comment 10

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=82cff5ae36437c156a89c9e6fbcf81cf35709fa5 **now passing**

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a19d696f96fb
Read OneCRL blocklist from security-states/onecrl r=jcj,mgoodwin,glasserc

Comment 12

[Andreea Pavel [:apavel]]

Backed out for failing bc at browser_all_files_referenced.js

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=242149255&revision=a19d696f96fbf2375fbf3cf107a3d974262a7d5e

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242149255&repo=autoland&lineNumber=1215

Bacout: https://hg.mozilla.org/integration/autoland/rev/19fc231ad8d7763370dcb8766c5e0883e669a49c

Comment 13

[Ionuț Goldan [:igoldan]]

Before this got backed out, we noticed this installer size increase:

== Change summary for alert #20584 (as of Tue, 23 Apr 2019 20:10:32 GMT) ==

Regressions:

300KBytes installer size osx-shippable opt nightly 77,004,021.58 -> 77,326,302.42
300KBytes installer size osx-shippable opt gcp nightly 77,006,034.21 -> 77,324,841.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=20584

Comment 14

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=87bb65dfcd47a3111e29ad7c47e28f913f25463f

Comment 15

[Mathieu Leplatre [:leplatrem]]

Andreea, sorry for the troubles :/ I only had ran xpcshell tests...

Ionut, this is normal indeed. With this patch, we now take into account one of the JSON dumps of Remote Settings [0] (We introduced it in https://bugzilla.mozilla.org/show_bug.cgi?id=1541841 but was not mentioned in package-manifest.in)

[0] https://firefox-source-docs.mozilla.org/services/common/services/RemoteSettings.html#initial-data

Comment 16

[:Gijs (he/him)]

(In reply to Mathieu Leplatre [:leplatrem] from comment #15)

Andreea, sorry for the troubles :/ I only had ran xpcshell tests...

Ionut, this is normal indeed. With this patch, we now take into account one of the JSON dumps of Remote Settings [0] (We introduced it in https://bugzilla.mozilla.org/show_bug.cgi?id=1541841 but was not mentioned in package-manifest.in)

[0] https://firefox-source-docs.mozilla.org/services/common/services/RemoteSettings.html#initial-data

Is 300k the smallest we can get this? Because as installer size goes, that's pretty sizable, esp. on mobile (where it seems we are also packaging this...).

Comment 17

[Mathieu Leplatre [:leplatrem]]

Is 300k the smallest we can get this? Because as installer size goes, that's pretty sizable, esp. on mobile (where it seems we are also packaging this...).

On Mobile there should be no increase, since we already shipped the same data under a different name

Comment 18

[:Gijs (he/him)]

(In reply to Mathieu Leplatre [:leplatrem] from comment #17)

Is 300k the smallest we can get this? Because as installer size goes, that's pretty sizable, esp. on mobile (where it seems we are also packaging this...).

On Mobile there should be no increase, since we already shipped the same data under a different name

So we ate that same cost earlier - if we could slim this down e.g. by compressing it somehow, that seems worth at least thinking about...

Comment 19

[Mathieu Leplatre [:leplatrem]]

that seems worth at least thinking about...

I created https://bugzilla.mozilla.org/show_bug.cgi?id=1546675 and https://bugzilla.mozilla.org/show_bug.cgi?id=1546678

Comment 20

[Mathieu Leplatre [:leplatrem]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8164d486a176a1d906a1488fd1b19f1acf95b280

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e1ab2cda0424
Read OneCRL blocklist from security-states/onecrl r=jcj,mgoodwin,glasserc

Comment 22

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/e1ab2cda0424

Comment 23

[Andreea Pavel [:apavel]]

(In reply to Mathieu Leplatre [:leplatrem] from comment #15)

Andreea, sorry for the troubles :/ I only had ran xpcshell tests...

Ionut, this is normal indeed. With this patch, we now take into account one of the JSON dumps of Remote Settings [0] (We introduced it in https://bugzilla.mozilla.org/show_bug.cgi?id=1541841 but was not mentioned in package-manifest.in)

[0] https://firefox-source-docs.mozilla.org/services/common/services/RemoteSettings.html#initial-data

No problem, glad this got fixed in the meantime.