getHSTSPreloadList.js takes too long
Categories
(Core :: Security: PSM, task)
Tracking
()
People
(Reporter: jcristau, Assigned: keeler)
References
Details
(Whiteboard: [psm-assigned])
Attachments
(2 files)
Updating the HSTS preload list is now taking roughly 5 hours, with about 100k domains on the list. Can we somehow improve that?
Assignee | ||
Comment 1•3 years ago
|
||
Before this patch, getHSTSPreloadList.js would have no more than 250 requests
in flight at any given time. In the past, this limit was 500. After a refactor
that inadvertantly completely removed the limit, it was reduced to 250.
Presumably increasing it to 500 again will not have negative effects and will
allow the script to run faster.
Updated•3 years ago
|
Assignee | ||
Updated•3 years ago
|
Pushed by dkeeler@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/042c5a186d30 increase the parallelism of getHSTSPreloadList.js r=releng-reviewers,jmaher
Comment 3•3 years ago
|
||
bugherder |
Reporter | ||
Comment 4•3 years ago
|
||
Monday's update (https://firefox-ci-tc.services.mozilla.com/tasks/flPOTfqjQoigj1eFY1kQbg) ran for 297 minutes.
Today's update (https://firefox-ci-tc.services.mozilla.com/tasks/IjAP2s7fQ3yggIpmwgot3A) finished in 157 minutes.
Thanks Dana!
Reporter | ||
Comment 5•3 years ago
|
||
Now that getHSTSPreloadList.js runs with increased parallelism, it goes
much faster and we don't need to let the task run for 6 hours.
The timeout was 3 hours for a while, then bumped to 5 hours in bug
1570646, then to 6 hours in bug 1705639 (with 250 in-flight requests).
The new run time with 500 in-flight requests seems to be around 2.5
hours, so pick 4 hours as a compromise.
Pushed by jcristau@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/1650540c413b reduce max-run-time for repo-update task. r=RyanVM,releng-reviewers,bhearsum DONTBUILD
Comment 7•3 years ago
|
||
bugherder |
Reporter | ||
Comment 8•3 years ago
|
||
bugherder uplift |
https://hg.mozilla.org/releases/mozilla-beta/rev/7e05197cbf4e
https://hg.mozilla.org/releases/mozilla-beta/rev/e5d9cf590ce2
Reporter | ||
Comment 9•3 years ago
|
||
bugherder uplift |
https://hg.mozilla.org/releases/mozilla-esr78/rev/405eaeaadb98
https://hg.mozilla.org/releases/mozilla-esr78/rev/1d47c7b0d2d0
Description
•