279570 - "compile" text files before .jar-ing them

Status: RESOLVED EXPIRED

(Firefox Build System :: General, enhancement)

Description

[Radek 'sysKin' Czyz]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

Javascript, css and xul files are kept in the .jar archives in exactly the same
form as in source tree. However, once they are part of the executable end-user
binary, they don't need to be human-readable - we can remove all the whitespaces
and comments while building the product.

The obvious advantage is less memory used, reduced stress on the parser and
zlib. Also, a smaller installer.

Reproducible: Always

Steps to Reproduce:
1. Imagine how zlib opens, say, browser.jar

Actual Results:  
zlib wastes memory and clock cycles to parse comments and whitespaces

Expected Results:  
waste is minimized

I made a test: I used w3compiler (by port80 software) to "optimize" the contents
of toolkit.jar, which is the biggest .jar in Firefox. It ignored .xul files and
only processed .css, .js and .html. In all cases, it saved 34% of their size by
removing whitespaces and comments.

Once I packed the files back ito a .jar, the jar was 20% smaller. Once I 7zipped
the jar, the result was 22% smaller (compared to 7zipped original file).

The only question is "how", and I guess the only method is to write a small
command-line utility that would remove the whitespaces, and compile & use this
utility when building firefox.

Comment 1

[Ben Fowler]

Your proposal was to save time (cycles) but your measurements were
of space (byte counts).

If this infrastructure is ever implemented either in whole
or in part (and I suspect that this will eventually be WONTFIXed)
could I make a side order that the eventual jar creator scans input
files for conflict markers ( ^<<<<< ), and warns or even stops
the build when these are found. Syntactically incorrect files
like that can cause a lizard to fail silently!

Comment 2

[:Gavin Sharp [email: [e-mail]]]

Compression is a no go, as the overhead required offsets any benefits. The files
are also already cached, so I'm thinking that this is about as optimised as it
will get already.

Comment 3

[Radek 'sysKin' Czyz]

(In reply to comment #1)
> Your proposal was to save time (cycles) but your measurements were
> of space (byte counts).

Yup, mu assumption is that the time needed to parse text files, or load them
from disk (at startup) is somehow proportional to their size (in bytes).

> If this infrastructure is ever implemented either in whole
> or in part (and I suspect that this will eventually be WONTFIXed)
> could I make a side order that the eventual jar creator scans input
> files for conflict markers ( ^<<<<< ), and warns or even stops
> the build when these are found. Syntactically incorrect files
> like that can cause a lizard to fail silently!

Cool idea but who will write that? :) As for simple whitespace callapse +
comment removal, I can volunteer to write such tool, once(/if) this bug is
confirmed.

Comment 4

[Ben Fowler]

(In reply to comment #3)
> (In reply to comment #1)
> > Your proposal was to save time (cycles) but your measurements were
> > of space (byte counts).
> 
> Yup, my assumption is that the time needed to parse text files, or load them
> from disk (at startup) is somehow proportional to their size (in bytes).

My assumption would be that the time required to skip N bytes of white
space is O(1) http://www.nist.gov/dads/HTML/bigOnotation.html and therefore
the value of N is unimportant. I also assume that removing comments
and other trivial analysis of the chrome files takes an insignificant
amount of time. I recollect seeing some figures put forward 5 or more years
ago tending to support this and also comment 2 . 

Of course if you had  a theory as to why this was a good idea and 
measurements (or even estimates) of timings 
http://c2.com/cgi/wiki?PrematureOptimization , then we would be out
of the realm of assumptions and have solid facts to work with. I am 
not saying that you are wrong - though you might be - merely that 
optimisation of an application needs to be preceded by some fairly 
astute information gathering: Why do you think that Gecko is spending a 
lot of time parsing XUL, and Why do you think that this proposal will 
help matters?

Comment 5

[Radek 'sysKin' Czyz]

Apologies if you consider this a bugspam, but I wanted to tell that I've made a
nice tool that lets me measure browser opening time, window opening time and tab
opening time. It's cross-platform and should be cross-browser, although my "best
so far" version doesn't work with IE (older did). Yell if you're interested.

As for the bug itself, it seems I'll need to code a "whitespace collapser"
myself in order to test anything. However, maybe *you* know a tool that would do
that? Just to let me do some measurments...

Comment 6

[Ben Fowler]

Performance issues in general are most certainly considered appropriate
for tracking as Bugs. Videlicet:

Bug 255703 "Scrolling performance has regressed since 1.8a1"
Bug 225085 "Firebird paint performance when scrolling page with flash is very
slow compared to IE 6"
Bug 21762 "tracking: poor performance using DHTML"

amongst many hundreds of bugs relating to performance, and 
http://www.mozilla.org/performance/

Comment 7

[Ben Fowler]

(In reply to comment #5)
> ... I've made a
> nice tool that lets me measure browser opening time, window opening time and tab
> opening time. 

Perhaps in Page Info there could be a 'Performance' Tab that would show
(wall) time taken to parse, and to render a page.

Comment 8

[Nickolay_Ponomarev]

XUL/JS are already cached (in compiled state) in XUL cache (xul.mfl or similar).
The only point of such change would be to decrease the size of installer (do you
have numbers for that?). That would make any quick-debugging/tweaking/writing
extensions a PITA.

Comment 9

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 10

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.