Use lzip for compressing source tarballs instead of bzip2 : smaller archive and faster decompression, same compatibility

RESOLVED DUPLICATE of bug 807637

Status

Release Engineering
General Automation
RESOLVED DUPLICATE of bug 807637
4 years ago
2 years ago

People

(Reporter: Juan Francisco Cantero Hurtado, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
User Agent: Mozilla/5.0 (X11; OpenBSD amd64; rv:25.0) Gecko/20100101 Firefox/25.0 (Beta/Release)
Build ID: 20131106144203

Steps to reproduce:

This bug report is related to the bug #807637 but specific to the source code tarballs. The other bug report is related to the release tarballs.

I'm asking to compress the official source code tarballs with lzip in addition (or not) to bzip2. Advantages over bzip2:
- Smaller tarballs
- Faster decompression
- Same interface and exit status than bzip2 (https://lists.gnu.org/archive/html/bug-tar/2013-05/msg00001.html). This is a drop-in replacement for bzip2.
- lzip has an official multithreaded compressor/decompressor.
- lunzip is really tiny and the next version will have an option to limit the memory use. This is perfect for memory-constrained enviroments. BTW, the standard version uses only 33MB of RAM for files compressed with -9.
-the lzip project has differents compressors and decompressors written in C and C++. So mozilla would have always an alternative plan if something goes wrong.
- clzip and lunzip support any OS with a C compiler, no external dependencies. There is also a windows version.
- the format is focused on long term archiving and high compression ratios.

 622M firefox-25.0.source.tar
 122M firefox-25.0.source.tar.bz2
98.0M firefox-25.0.source.tar.lz
97.9M firefox-25.0.source.tar.xz

http://lzip.nongnu.org/

"Why not xz?". I'm adding lzip support to the OpenBSD ports framework (for upstream's source tarballs) and lzip has every xz advantage but without its disadvantages. We support some mozilla products on very old platforms and there the memory use is very important. bzip2 and lunzip has options to limit the memory use during the decompression, despite of lunzip only uses by default 33MB compared to the 65MB used by xz, lzip can decompress a file compressed with -9 with less than 1MB of RAM.

Let me know what do you think and if I can help you in something.

Updated

4 years ago
Status: UNCONFIRMED → NEW
Ever confirmed: true
I'm really tempted to WONTFIX this in favour of xz. Having two competing proposals for new compression schemes in bugs isn't a good idea, and my sense is that xz is far more popular. We won't be supporting all of bzip2, xz and lzip. At most we'll support bzip2 and one of the new contenders, but I'm hoping we can choose just one.

Please raise this issue in the newsgroups for discussion if you really want to see this change happen.
(Reporter)

Comment 2

4 years ago
What's the correct newsgroup to start the discussion?.

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 807637

Comment 4

2 years ago
For your information xz is able to deal with low memory systems.
The man page gives the required memory for compressing and uncompressing depending on the preset you choose for compression :
---------
Preset   DictSize   CompCPU   CompMem   DecMem
  -0     256 KiB       0        3 MiB    1 MiB
  -1       1 MiB       1        9 MiB    2 MiB
  -2       2 MiB       2       17 MiB    3 MiB
  -3       4 MiB       3       32 MiB    5 MiB
  -4       4 MiB       4       48 MiB    5 MiB
  -5       8 MiB       5       94 MiB    9 MiB
  -6       8 MiB       6       94 MiB    9 MiB
  -7      16 MiB       6      186 MiB   17 MiB
  -8      32 MiB       6      370 MiB   33 MiB
  -9      64 MiB       6      674 MiB   65 MiB
----------

But, once again, the requirement page tells Firefox 44 requires 512MB of RAM for operation on Windows (32 or 64bits) or Mac OS X.
You need to log in before you can comment on or make changes to this bug.