36358 - Mozilla milestones/nightlies/sourcedrops should also be available in .tar.bz2 format.

Status: VERIFIED FIXED

(SeaMonkey :: Build Config, enhancement, P3)

Description

[David Hallowell]

As mentioned by a poster on today's Slashdot.org article about the M15 release,
I believe it would be a good idea making the tarballs available with both bzip2
and gzip compression.

The smaller size of the bzip2 compressed tarballs would be of particular benefit
to those people who wish to try Mozilla but are on modem lines where they have
to pay for their online time.

The Linux Kernel is an example of something available in both .tar.gz and
.tar.bz2 formats.

Comment 1

[cls]

Reassigning to leaf as he actually handles the releases.  

Comment 2

[Stephan Niemz]

    I would also be really happy if there would be bzip2-compressed tarballs
of the nighly builds available. It's just one more command which has to be added
to a script.

    By the way, isn't this a duplicate of bug 36287? And I think the OS
shouldn't only be Linux.

Comment 3

[David Krause]

Many people in #mozillazine are expressing interest in having nightly/milestone
builds available in tar.bz2 format for linux and win32 and possibly mac formats.
The file size is smaller and many of the testers are on dialup and could
download more often

Comment 4

[David Krause]

*** Bug 36287 has been marked as a duplicate of this bug. ***

Comment 5

[David Krause]

Of all the available compression formats tar.bz2 has the best compression ratio
and the most crossplatform support.

Example programs:
PowerArchiver for Windows supports tar.bz2
Aladdin Stuffit for Macintosh supports tar.bz2
Most *NIX systems support tar.bz2

smaller file size == more downloads == bugzilla reports with more recent builds

Comment 6

[Oleg Rekutin]

It'd be SUPER if Mozilla nightlies for Windows were available in .tar.bz2, as 
well as usual .zip.

I just downloaded the (latest at time of downloading) 6/27 nightly. A comparison 
table: (numbers rounded to nearest integer)

    format | size  | comp. ratio | gain over .zip
  ---------+-------+-------------+----------------
  .zip     | 7503k | 38%         | 0
  .tar.gz  | 6683k | 34%         | 820k
  .tar.bz2 | 6023k | 30%         | 1480k

All of these archivation methods are free (A shareware archiver JAR was able to 
achieve 2007k gain over .zip). 

Anyway, the savings with .tar.bz2 are 1.45MB! That's 5-6 minutes savings on 56.6 
modem and 9-10 minutes on 28.8 (assuming decent connections).

I think that these are pretty big savings for us poor dial-up users.

Further savings could be gained if someone could make tar group input files by 
file extension (all .xul files together, all .html files together, etc). 
Probably something that a Unix command-tool guru could hack up :)

People say that Mac's Stuffit handles bzip2, so I imagine similar savings could 
result in for Mac users. Linux handles bzip2 very well (like duh :) ), so they 
would obviously benefit from it too.

I haven't tried recompressing Mac or Linux builds, but I imagine that the size 
savings would be in the similar range.

Comment 7

[David Krause]

Similar results can be obtained on linux.
A recent nightly was 767 KB and when ungzipped and then rezipped with bzip2 the
file size was 669 KB, a savings of 1 MB!!

Comment 8

[Ben Bucksch (:BenB)]

leaf?

Comment 9

[Daniel (Leaf) Nunes]

fixed for milestone source drops. will take a bit to fix automation and tools
for nightly binaries.

Comment 10

[J.J. Enser]

unfortunately we cannot use this format for Mac bits, even though Stuffit 
supports it. bz2, like many other encoding algorithms, doesn't preserve mac files 
resource fork, which we need to deliver self-extracting archives.
the only encoding format I know of that are "mac compatible" are BinHex and 
MacBinary (which we currently use).

Another reason is that you need a fairly recent version of Stuffit Expander 
(5.5?) to understand and decode bz2 files. We cannot assume that TODAY most -if 
not all- of our users have such a version on their machine. 

Finally, the gain over MacBinary is almost insignificant:
mozilla-mac-MN6.sea     = 9,064,719 (compressed but not encoded)
mozilla-mac-MN6.sea.bin = 9,064,960 (macBinary2)
mozilla-mac-MN6.sea.bz2 = 8,967,316 (e.g. 97K = 1%)

Comment 11

[Stephan Niemz]

    I understand the problems with the Mac, but can't you just add bz2
compressed nightlies for the platforms that actually support them, like
Linux? This bug is now open for more than half a year, and I know I'm not
the only one who would be happy to see the bzip2 support added. Thanks!

Comment 12

[Ben Bucksch (:BenB)]

Attachment: Fix, Step 1, Version 1

Comment 13

[timeless]

[e-mail] you managed to change this bug from assigned to leaf to new for 
leaf. How did you do that? cc:tara

fwiw, for me, .tar.bz2 is not optimal on windows.

I d/l a .zip and then have my unzipper only extract updated files.  .tar.bz2 
requires an expensive algorithm (read google about why they don't use it) to 
extract the entire tarball and then you have to go through and do something 
(most likely overwrite all files).

Comment 14

[Ben Bucksch (:BenB)]

The attached patch contains the required Mozilla source tree changes. Do the
following to get a bz2 tarball:

cd mozilla/xpinstall/package;MOZ_PKG_FORMAT=BZ2 BUILD_OFFICIAL=1 make

The patch also changes the top-level dir in the tarball from "package" to "mozilla".

What do you all think about making bz2 the default (what is used when you do
just |make| in an offical build (|BUILD_OFFICIAL=1|)?

I just noticed that |strip| gives a lot of unnecessary errors. I updated the
file extensions to ignore. Will attach new patch. We might want to switch to an
opt-in instead of opt-out list.

Comment 15

[Ben Bucksch (:BenB)]

timeless, then continure to use the zip tarballs. Other people might prefer bz2
tarballs.

> What do you all think about making bz2 the default

Only for Unix, that is. Also, I do not propose that mozilla.org provides only
bz2 tarballs from now one, rather both types. My wuestions is only about the
default.

Comment 16

[Ben Bucksch (:BenB)]

Attachment: Fix, Step 1, Version 2

Comment 17

[Ben Bucksch (:BenB)]

[OT]
> [e-mail] you managed to change this bug from assigned to leaf to new for
> leaf. How did you do that? cc:tara

Reassign to the current owner?

Comment 18

[Stephan Niemz]

    timeless: sorry, I didn't mean to do that. That must be a bug in
Bugzilla.

    I wasn't logged in to Bugzilla when viewing this bug. Then I added my
comment, and logged in in another Mozilla window (see bug 20122 for the
reasons and my comments there for the workaround I'm using). Then I hit
commit.  Bugzilla complained that I tried to change the owner or something
from "[e-mail]" to "[e-mail] (Daniel" (without the
quotes).  So I hit the back button and deleted the " (Daniel" from the
"assign to" field. Unfortunately Mozilla forgot what I entered in the
textarea input field (another bug!), so I had to enter my whole comment
once again. I hit commit again and this time Bugzilla was happy with it.
Maybe Bugzilla doesn't handle the two pairs of parens in leafs id
correctly.

    Does anybody know the bug numbers of those two bugs, or if they are
open already? Thanks!

Comment 19

[Ben Bucksch (:BenB)]

Will attach new patch to bug 53462.

Comment 20

[gabriel]

Is anybody still looking into this or has it been forgotten ?

Comment 21

[Daniel (Leaf) Nunes]

bzip2 source for milestones has been happening for a few milestones, i've
updated the nightly source automation to do bzip, should start appearing
tomorrow morning.

Comment 22

[Jon Granrose]

verified.

Comment 23

[Ben Bucksch (:BenB)]

The nightlies are not yet availble as bz2. Testers also asked for a version with
jarred chrome only (bug 53462). Maybe we can combine both of that, to provide a
small download.
MOZ_PKG_FORMAT=BZ2 make dist
should do it (IIRC). leaf, should I file a new bug?

Comment 24

[Daniel (Leaf) Nunes]

sure... this bug is for the source drops, and that's fixed.

Comment 25

[Ben Bucksch (:BenB)]

bug 66358

Comment 26

[Asa Dotzler [:asa]]

*** Bug 181475 has been marked as a duplicate of this bug. ***