Closed Bug 1717683 Opened 3 years ago Closed 2 years ago

Make ZIP packages and l10n repacks deterministic by using `zip.py` instead of `make_zip.py`

Categories

(Firefox :: Installer, enhancement)

enhancement

Tracking

()

RESOLVED FIXED
98 Branch
Tracking Status
firefox98 --- fixed

People

(Reporter: nalexander, Assigned: nalexander, NeedInfo)

References

Details

Attachments

(1 file)

Right now, l10n repacks (and, I think, regular old packages) are not deterministic: the ZIP files encode build timestamps. This usage is the single remaining consumer of make_zip.py. I think we can have deterministic packages and repacks if we start using zip.py instead. The thing that may be different between that invocation and other invocations of zip.py are the compression flags.

It's possible that couldn't, at one time rely on enough of mach being available to do this in all packaging/repacking scenarios, but I don't think that's true at this point.

When I try this, I get a ZIP that looks pretty similar but has a slightly different ordering of the input files. I don't think that matters. What might matter is the following, as reported by diffoscope:

...
│
│ -Filename: firefox/maintenanceservice.exe
│ +Filename: firefox/xul.dll
│  Comment:
│
│ -Filename: firefox/nss3.dll
│ +Filename: firefox/xul.dll.sig
│  Comment:
│
│  Zip file comment:
├── filetype from file(1)
│ @@ -1 +1 @@
│ -Zip archive data, at least v1.0 to extract, compression method=store
│ +Zip archive data, at least v2.0 to extract, compression method=deflate

It's not clear to me if that is just for the xul.dll.sig file or if it's more general. :glandium, do you have thoughts on the proposed change and the ZIP file comment discrepancy? Maybe you've seen this before with diffoscope?

Flags: needinfo?(mh+mozilla)

The compression options appear to be the same: mozjar defaults to
level 9 compression. The mozjar code is definitely slower than the
native ZIP executable, but not, I think, by enough to not do this.

This is the sole consumer of make_zip.py in the tree, so we remove
the file entirely.

Assignee: nobody → nalexander
Status: NEW → ASSIGNED

(In reply to Nick Alexander :nalexander [he/him] from comment #1)

It's not clear to me if that is just for the xul.dll.sig file or if it's more general. :glandium, do you have thoughts on the proposed change and the ZIP file comment discrepancy? Maybe you've seen this before with diffoscope?

can you link to the diffoscope job with that output?

Flags: needinfo?(mh+mozilla)
Flags: needinfo?(nalexander)

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:nalexander, could you have a look please?
For more information, please visit auto_nag documentation.

Flags: needinfo?(nalexander)
Flags: needinfo?(mh+mozilla)
Flags: needinfo?(mh+mozilla)

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Mach Core' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: General → Mach Core
Component: Mach Core → Installer
Product: Firefox Build System → Firefox
Pushed by nalexander@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3fcafb7a4946
Make ZIP packages and l10n repacks deterministic by using `zip.py` instead of `make_zip.py`. r=firefox-build-system-reviewers,glandium
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 98 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: