Make ZIP packages and l10n repacks deterministic by using `zip.py` instead of `make_zip.py`
Categories
(Firefox :: Installer, enhancement)
Tracking
()
Tracking | Status | |
---|---|---|
firefox98 | --- | fixed |
People
(Reporter: nalexander, Assigned: nalexander, NeedInfo)
References
Details
Attachments
(1 file)
Right now, l10n repacks (and, I think, regular old packages) are not deterministic: the ZIP files encode build timestamps. This usage is the single remaining consumer of make_zip.py
. I think we can have deterministic packages and repacks if we start using zip.py
instead. The thing that may be different between that invocation and other invocations of zip.py
are the compression flags.
It's possible that couldn't, at one time rely on enough of mach
being available to do this in all packaging/repacking scenarios, but I don't think that's true at this point.
Assignee | ||
Comment 1•3 years ago
|
||
When I try this, I get a ZIP that looks pretty similar but has a slightly different ordering of the input files. I don't think that matters. What might matter is the following, as reported by diffoscope
:
...
│
│ -Filename: firefox/maintenanceservice.exe
│ +Filename: firefox/xul.dll
│ Comment:
│
│ -Filename: firefox/nss3.dll
│ +Filename: firefox/xul.dll.sig
│ Comment:
│
│ Zip file comment:
├── filetype from file(1)
│ @@ -1 +1 @@
│ -Zip archive data, at least v1.0 to extract, compression method=store
│ +Zip archive data, at least v2.0 to extract, compression method=deflate
It's not clear to me if that is just for the xul.dll.sig
file or if it's more general. :glandium, do you have thoughts on the proposed change and the ZIP file comment discrepancy? Maybe you've seen this before with diffoscope
?
Assignee | ||
Comment 2•3 years ago
|
||
The compression options appear to be the same: mozjar
defaults to
level 9 compression. The mozjar
code is definitely slower than the
native ZIP executable, but not, I think, by enough to not do this.
This is the sole consumer of make_zip.py
in the tree, so we remove
the file entirely.
Updated•3 years ago
|
Comment 3•3 years ago
|
||
(In reply to Nick Alexander :nalexander [he/him] from comment #1)
It's not clear to me if that is just for the
xul.dll.sig
file or if it's more general. :glandium, do you have thoughts on the proposed change and the ZIP file comment discrepancy? Maybe you've seen this before withdiffoscope
?
can you link to the diffoscope job with that output?
Updated•3 years ago
|
Comment 4•3 years ago
|
||
There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:nalexander, could you have a look please?
For more information, please visit auto_nag documentation.
Updated•3 years ago
|
Comment 5•2 years ago
|
||
The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Mach Core' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.
Updated•2 years ago
|
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/3fcafb7a4946 Make ZIP packages and l10n repacks deterministic by using `zip.py` instead of `make_zip.py`. r=firefox-build-system-reviewers,glandium
Comment 7•2 years ago
|
||
bugherder |
Description
•