1398829 - Periodic update script makes it difficult to add HSTS/HPKP script dependencies

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[:Cykesiopka]

The periodic update script currently hard codes the HSTS and HPKP scripts and dependencies. This unfortunately means introducing additional dependencies for the scripts (common JSMs and so on) requires the periodic update script be updated as well. This is not ideal, and requires coordinating landings between two repositories as well.

If possible, it would be nice to have the periodic update script support this scenario.

Comment 1

[:Cykesiopka]

Attachment: bug1398829_make-periodic-script-dl-entire-tools-folder_WIPv1.patch

This patch implements a potential solution.

I'm not sure if there was a reason for the hard-coding previously, so just asking for feedback for now. I also haven't tested this script works with repos other than m-c yet.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8906657 [details] [diff] [review]
bug1398829_make-periodic-script-dl-entire-tools-folder_WIPv1.patch

Hmm, a wget-based sparse clone. If we had hg 4.3+ available I'd suggest https://hg.mozilla.org/mozilla-central/json-file/default/security/manager/tools/ instead. I think that's only on taskcluster though, and then landing changes would be a problem. You may find the url https://hg.mozilla.org/mozilla-central/json-file/default/security/manager/tools/ is helpful to discover the files, rather than scraping.

I wonder if we can convert this whole thing to an artifact build. So clone gecko, './mach build' with artifact build enabled in the mozconfig to get xpcshell and other binaries, then call scripts (or even add mach targets) to run the updates for blocklist/hsts/hpkp. Eventually this would run on taskcluster and we'd have to solve how to land, but maybe it can work in buildbot in the meantime (with the gtk3 LD_LIBRARY_PATH hacks).

Comment 3

[Aki Sasaki (not active)]

We're using a docker image now that should be super easy to update.