1387855 - getHSTSPreloadList.js only adds but never removes domains

Status: NEW

(Core :: Security: PSM, enhancement, P3)

Description

[Tim Taubert [:ttaubert] (inactive)]

We seem to not properly synchronize from Chrome's list. Here's a diff that removes schokokeks.org subdomains from the HSTS preload list on June 16th:

https://chromium.googlesource.com/chromium/src/net/+/3ef56b8a745da84b7ccd98446984b96feeed21eb%5E%21/#F1

We however still have all of them in nsSTSPreloadList.inc. It seems that getHSTSPreloadList.js retrieves the current list of hosts, and then only adds new ones.

Why is that? Couldn't we only use Chrome's list? If we have a separate list with additional hosts we should probably keep it separate and combine it every time we retrieve updates.

Comment 1

[Liz Henry (:lizzard) (privacy/fingerprinting team)]

David, is this something we may want to fix for 56 or 57?  If so, can you find someone to investigate before Tim gets back? Thanks.

Comment 2

[Dana Keeler (she/her) [:keeler]]

If I recall correctly, the original intention was to only remove hosts if our automated script visited them and saw a header with max-age less than 18 weeks. If there are hosts on the list we can't connect to, then to prevent intermittent network outages from unintentionally removing them from the list, we again keep them on the list. We could revisit these decisions, but I don't think there's anything urgent about this, unless there are entries on the list that are causing problems. We could also maybe occasionally hand-cull sites that are clearly defunct.

Comment 3

[Hanno Boeck]

Some context: I informed Tim about this issue, because I noted that some redundant entries for schokokeks.org are still in the list. I originally submitted some subdomains from schokokeks.org to Chrome (back when their preload list was manually handled by Adam Langley). Later I submitted the whole schokokeks.org Domain (with includesubdomains enabled) to chrome's preload list and asked them to remove the old entries. This doesn't really cause any issues, it's just stale, redundant entries that serve no purpose.

It seems Mozilla automatically imports entries from Chrome, but doesn't remove them when Chrome removes them. I think that's bad practice. Apart from that maybe you want to run a consistency check on the list for redundant entries (aka if example.com with includesubdomains is in the list then subdomain.example.com doesn't have to be).