1438006 - Limit blocklist entries for all non-flash plugins to <=Fx57

Status: VERIFIED FIXED

(Toolkit :: Blocklist Policy Requests, enhancement)

Description

[:Gijs (he/him)]

As far as I can tell the flash ones are:

p330
p332
p1054

832dc9ff-3314-4df2-abcf-7bd65a645371
49b843cc-a8fc-4ede-be0c-a0da56d0214f


and maybe p1055 .

(Checked by looking through the existing firefox-59-served blocklist for "flash" and "shockwave")

I don't know if it's easy to combine some of these, if so that would also help further.

Comment 1

[:Gijs (he/him)]

Attachment: Plugins that can be removed for Fx58 and later

These are all the plugin items that I see in the current list off https://blocklists.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/58.0.2/Firefox/ that I think can be restricted to earlier versions so that 58+ won't see them anymore.

Comment 2

[:Gijs (he/him)]

Attachment: Plugins that can be removed for Fx58 and later

Sans 1055 (see comment #0).

Comment 3

[:Gijs (he/him)]

Mathieu, who has the ability to bulk-update these entries (see attachment) so that:

- if they don't have a targetApplication that matches Firefox's guid, they get a targetApplication of Firefox ( <targetApplication id="{ec8030f7-c20a-464f-9b0e-13a3a9e97384}"> ) and maxVersion 57.0.* )
- if they do have a targetApplication that matches Firefox's guid, we update the maxVersion for it to 57.0.* ?)

Comment 4

[Mathieu Leplatre [:leplatrem]]

> who has the ability to bulk-update these entries (see attachment) so that:

Currently any member of the plugins-editors group can do it:
https://github.com/mozilla-services/cloudops-deployment/blob/ee69dd00d1471bbcf86332576153601a0f97045a/projects/kinto/puppet/yaml/permissions/kinto.prod.yaml#L1125-L1133

In order to do it, they simply have to run a small Python script from a terminal (while connected to the VPN).

    $ unset HISTFILE
    $ pip install --user kinto-http
    $ USERNAME=[e-mail] PASSWORD=ldap-pwd python batch.py 

It could be more a less like this (untested):
https://gist.github.com/leplatrem/391d20cdc590feba5b49488351f853c3

Comment 5

[:Gijs (he/him)]

Perfect, thanks Mathieu!

Andreas, looks like you're in that list of users. I modified Mathieu's script (still untested, of course) to avoid the 6 items I mentioned in comment #0. Could you run this? ( https://gist.github.com/gijsk/888354f4ffa9ad3272f64fb6921c3a16 )

Comment 6

[Andreas Wagner [:TheOne] [use NI]]

The script needed a few adaptions (for reference: https://gist.github.com/wagnerand/29aab2087c0031496d3da834e398d5d3)

The update records have been created and are currently pending review.
Jorge, could you review this, please?

Comment 7

[Jorge Villalobos [:jorgev] (he/him)]

As far as I understand the code, it looks right to me.

Comment 8

[Jorge Villalobos [:jorgev] (he/him)]

I also approved the staged changes now.

Comment 9

[Mathieu Leplatre [:leplatrem]]

XML size is now 2X smaller for >=58:

    $ curl -I https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/57.0/
    HTTP/1.1 200 OK
    Content-Length: 308721

vs.

    $ curl -I https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/58.0/
    HTTP/1.1 200 OK
    Content-Length: 133869

Comment 10

[:Gijs (he/him)]

(In reply to Mathieu Leplatre (:leplatrem) from comment #9)
> XML size is now 2X smaller for >=58:
> 
>     $ curl -I
> https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-
> c20a-464f-9b0e-13a3a9e97384%7D/57.0/
>     HTTP/1.1 200 OK
>     Content-Length: 308721
> 
> vs.
> 
>     $ curl -I
> https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-
> c20a-464f-9b0e-13a3a9e97384%7D/58.0/
>     HTTP/1.1 200 OK
>     Content-Length: 133869

For me, it seems the plugin entries are still there (and they don't have an applicationTarget with maxversion of 57). The reduction in size seems to be mostly a result of removing the certificates (bug 1435609).

Are the plugin changes that Andreas wrote and Jorge reviewed live?

Comment 11

[Mathieu Leplatre [:leplatrem]]

> For me, it seems the plugin entries are still there

Really? I only overlooked but:

    $ curl -s https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/57.0/ | grep "<pluginItem" | wc -l
    103

    $ ~ curl -s https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/58.0/ | grep "<pluginItem" | wc -l
    43

You are right, some of them don't have a targetApplication, but those who have are set with `maxVersion=*` which is consistent.

> Are the plugin changes that Andreas wrote and Jorge reviewed live?

Yes, look at the records modified since 2018-02-19:

https://firefox.settings.services.mozilla.com/v1/buckets/blocklists/collections/plugins/records?_since=1519047754330

Comment 12

[:Gijs (he/him)]

Hrm. Basically, I would have expected the list for 58 to have only 6 plugin items (the ones whose ids are in comment 0). I looked at the script again and I can't see why there's 43 - 6 = 37 others as well...

Comment 13

[:Gijs (he/him)]

From https://hg.mozilla.org/mozilla-central/rev/28d364d49f99 it seems some of the entries moved, but it's not clear to me why or if anything else about them changed. Some of the items also just stayed in place. :-(

Comment 14

[Mathieu Leplatre [:leplatrem]]

It looks like the remaining ones all have `*` as `maxVersion`.

It seems related to this line of the script:
https://gist.github.com/wagnerand/29aab2087c0031496d3da834e398d5d3#file-bulk-py-L42
where we don't force `maxVersion` but only set it if missing (and thus didn't replace those `*`)

Comment 15

[:Gijs (he/him)]

(In reply to Mathieu Leplatre (:leplatrem) from comment #14)
> It looks like the remaining ones all have `*` as `maxVersion`.
> 
> It seems related to this line of the script:
> https://gist.github.com/wagnerand/29aab2087c0031496d3da834e398d5d3#file-bulk-
> py-L42
> where we don't force `maxVersion` but only set it if missing (and thus
> didn't replace those `*`)

Ah, yes, for some of them this is indeed the case.

It looks like the other ones don't internally have a blockID. They have an 'id' property instead, and in the XML output this is rendered as 'blockID', but the python / internal stuff doesn't see that. So my filtering for items with a specific blockID at the top of the script is also broken and is skipping those items. I've forked and updated the script for these two cases:

https://gist.github.com/gijsk/c22ca81bf0c10710f07c8284b77274f9


Andreas, sorry for doing this to you twice, but would it be possible for you to rerun the script as above (I made sure to clone your script as linked above so it shouldn't error like the earliest versions - https://gist.github.com/gijsk/c22ca81bf0c10710f07c8284b77274f9/revisions shows my changes )? Thanks!

Comment 16

[Andreas Wagner [:TheOne] [use NI]]

I ran the updated script.

Jorge, would you mind reviewing and approving?

Comment 17

[Jorge Villalobos [:jorgev] (he/him)]

Done.

Comment 18

[Mathieu Leplatre [:leplatrem]]

https://firefox.settings.services.mozilla.com/v1/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/58.0/ now only has 6 items :)

Comment 19

[:Gijs (he/him)]

\o/

The blocklist size is now 62% smaller than the same list for 57 (and 52esr etc).