1619994 - Add addons-mlbf collection to blocklists bucket

Status: RESOLVED FIXED

(Cloud Services :: Operations: Kinto, task)

Description

[Andrew Williamson [:eviljeff]]

Supports https://github.com/mozilla/addons-server/issues/13616 and the v3 blocklist project. (If the collection name is non-standard I'm okay with it being normalized). We need this on stage and production kinto.

Comment 1

[Andrew Williamson [:eviljeff]]

The collection is intended to contain bloom filters as attachments - AMO will generate these a few times each day.

Comment 2

[Andrew Williamson [:eviljeff]]

The collection should be writeable by the user added in https://bugzilla.mozilla.org/show_bug.cgi?id=1615337

Comment 3

[Mathieu Leplatre [:leplatrem]]

I'm OK with the name if it makes sense to everyone. There's no really necessity to keep it short. addons-bloomfilters or whatever ml stands for would also be fine.

• Should the attachments be mandatory?
• Is there any JSON schema for the record fields?
• What record fields should be shown in the UI?
• Do you need JEXL filters (clients targeting)?

Comment 4

[Andrew Williamson [:eviljeff]]

addons-bloomfilters wfm me instead. (ml is multi level - they're bloom filters within bloom filters to guarantee no false positives/negatives)

• attachments should be mandatory
• The JSON schema isn't finalised yet - we're still working to see what Firefox will need. Can we update later?
• the UI doesn't concern me that much. All the "data" is in the attachment.
• I don't know about JEXL filters so I'm going to say no. The blocklist will be application agnostic.

Comment 5

[Andrew Williamson [:eviljeff]]

The plan is to push these bloom filters on a schedule (up to 4 times a day) via cron job, pre-signed by autograph, so we would want the submissions direct to a publicly available collection. Ideally "buckets/blocklist/collections/addons-bloomfilters" (i.e. not to "buckets/staging/..." where dual signoff would be required via kinto admin). While we await final agreement from security can you just make the change to kinto stage (don't make any change to kinto production). We will file a follow-up bug for production.

Comment 6

[Mathieu Leplatre [:leplatrem]]

Technically speaking you'll still upload the records to staging. But instead of switching status to to-review you'd switch it to to-sign and records will be published to blocklists immediately

Comment 7

[Andrew Williamson [:eviljeff]]

Ah! Good to know (and less of a change needed on the AMO side too)

Comment 8

[:wezhou]

I believe this can be closed. Feel free to reopen if not.