1571589 - Misc improvements for fetch-content

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Mike Hommey [:glandium]]

Some of what's coming supersede bug 1479533.

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1571589 - Use urlparse rather relying on just splitting on / being enough.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1571589 - Abstract opening a temporary file and renaming it after close.

And use that in git_checkout_archive.

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1571589 - Allow to repack downloaded archives "on the fly".

Bug 1479533 was proposing to add a similar functionality, but this
iteration avoids actually unpacking anything, and ensures
reproducibility by relying on the reproducible bits from the original
archives: file ordering, flags, etc. (since they are checksummed, those
are never going to change for a given archive).

Another notable difference is that this applies the repack on the fetch
task itself, rather than create a separate task to apply the repack. The
latter has advantages, in that it allows to change the repacking without
redownloading the original file from a third-party server, but in
practice, most changes to the repacking would trigger the download tasks
anyways.

This patch only takes care of changing the archive type (zip->tar), and
the compression type (anything->zstandard).

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1571589 - Allow simple manipulation of file paths in fetched archives.

Namely:

• adding a prefix,
• stripping path components.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/95f15f6c8fb9
Use urlparse rather relying on just splitting on / being enough. r=tomprince
https://hg.mozilla.org/integration/mozilla-inbound/rev/44f285f40796
Abstract opening a temporary file and renaming it after close. r=tomprince
https://hg.mozilla.org/integration/mozilla-inbound/rev/a801d59b70a5
Allow to repack downloaded archives "on the fly". r=tomprince
https://hg.mozilla.org/integration/mozilla-inbound/rev/dc8f0af3a4d6
Allow simple manipulation of file paths in fetched archives. r=tomprince

Comment 6

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/95f15f6c8fb9
https://hg.mozilla.org/mozilla-central/rev/44f285f40796
https://hg.mozilla.org/mozilla-central/rev/a801d59b70a5
https://hg.mozilla.org/mozilla-central/rev/dc8f0af3a4d6