HSTS preload list script: factor out obtaining the seed list




6 years ago
3 years ago


(Reporter: keeler, Assigned: keeler)



Firefox Tracking Flags

(Not tracked)



(1 attachment, 1 obsolete attachment)

Google recently changed the web interface to Chrome's code, and it is not currently possible to get a raw source file over http(s). Consequently, the HSTS preload list script fails when it attempts to get the seed list. It's increasingly obvious that we can't rely on that file being available or even in a format we expect. So, I propose we refactor that step out of building our preload list. That is, instead of fetching and parsing some remote file for the seed list, we supply a local file we know is good. This way, we can actually expand our preload list beyond what is in Chrome's list if/when we decide to do so. For right now, we would probably just run `svn export https://src.chromium.org/chrome/trunk/src/net/http/transport_security_state_static.json' and then use the checked out file.
Actually, I take half of that back: is looks like the list is available as a raw file at https://src.chromium.org/chrome/trunk/src/net/http/transport_security_state_static.json. However, the point still stands that we can't rely on this always being available at this location (or any location we don't control) and in a format we expect.
Posted patch patch (obsolete) — Splinter Review
The one thing I'm concerned about is that this changes how the script is called, which requires changes in the automation infrastructure. I could hard-code our two expected input files, but I'm not sure that's the best solution.
Attachment #762811 - Flags: review?(bsmith)
Comment on attachment 762811 [details] [diff] [review]

Review of attachment 762811 [details] [diff] [review]:

Camilo, can you trade me this review for reviews of your insanity patches?
Attachment #762811 - Flags: superreview+
Attachment #762811 - Flags: review?(cviecco)
Attachment #762811 - Flags: review?(bsmith)
Comment on attachment 762811 [details] [diff] [review]

Review of attachment 762811 [details] [diff] [review]:

However I would put in some defaults (ie, do we need this patch?)
Attachment #762811 - Flags: review?(cviecco) → review+
Posted patch patch v1.1Splinter Review
Thanks for the review. I tried to think of a good way to be able to just use defaults in the script, but for the Mozilla list at least, I think it's important to be able to have a local seed file, and I don't think it's a good idea to hard-code the path to that file. So, passed-in parameters it is.

I made some changes to guard against duplicates, but nothing functional changed.
Carrying over reviews.
Attachment #762811 - Attachment is obsolete: true
Attachment #767863 - Flags: superreview+
Attachment #767863 - Flags: review+
coop - this will require changes to how the script is run. In particular, it will need to check out a copy of the hsts_preload_candidates.json file this patch adds. Then, it just needs to pass "https://src.chromium.org/chrome/trunk/src/net/http/transport_security_state_static.json" and the absolute path to hsts_preload_candidates.json as additional arguments to the script. Is this reasonable? We might also want to go back to doing dry runs until we're confident this does the right thing. Thanks!
Flags: needinfo?(coop)
Existing script is here:


Could easily jerry-rig it to update a user repo for manual testing (possibly just by setting the BRANCH) until you're satisfied it's working correctly.

I'm away right now, so won't have a chance to look at this myself until late next week.
Flags: needinfo?(coop)
What we have is working for now.
Closed: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.