397522 - Optimized PNG files for icons

Status: RESOLVED FIXED

(Firefox :: Theme, enhancement, P4)

Description

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7

Based on #377554 and #386973, I had some time to further compress the png files in the source of Gran Paradiso Alpha 8.

Statistics on 511 PNG files:
Original size: 1,768,059 bytes
Final size: 1,378,590 bytes

This saved 389,469 bytes, or 380kB, translating into a 22% decrease without any loss in quality.

Method: see https://bugzilla.mozilla.org/show_bug.cgi?id=377554#c9

Reproducible: Always

Comment 1

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

The files are here:

http://file2upload.net/download/21626/pngfiles.zip.html

Comment 2

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

nominating...

this should land before other changes are made to the image files. if this drags on, new files might be added or existing ones changed.

I'll upload to another one-click hoster if the files are unable to be downloaded. (I've tested though, and it works)

Comment 3

[u88484]

Is there any performance hit to decompress these to render them?

Comment 4

[Alex Faaborg [:faaborg] (Firefox UX)]

I should note that we are planning on replacing nearly every png file during M9 or M10, but we can instruct the agency creating icons to follow the same process for brute force compression.

I think we need an answer on the possible perf hit before we are sure we want to do this for the new icon sets.

Are we completely absolutely positively sure this process is loss-less?  Moving all the bits around makes me nervous :)

Comment 5

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

PNGOut's compression is lossless, and as Wikipedia goes, "PNGOUT is used mainly to reduce the size of image data contained in the IDAT chunk." and for this case, "the resulting image will have exactly the same appearance as the source image."

PNGGauntlet is used just to enable its "brute force" option of choosing among all possibilities and taking the smallest file.

More info for everybody:

http://en.wikipedia.org/wiki/PNGOUT
http://advsys.net/ken/utils.htm
http://brh.numbera.com/software/pnggauntlet/

Comment 6

[Mike Beltzner [:beltzner, not reading bugmail]]

Let's get these compressed ones in for M9, assuming that there's no performance hit. I don't see why there would be, as we still have an unzipping step, right?

(cc'ing sayrer to help us understand potential performance hit)

Comment 7

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

As per Alex's comment #4 above, will new icon sets replace the existing png files? otherwise i'll have to come up with new compressed file sets based on the new files instead.

checking the existing ones in now might be double work for you guys if eventually new icon sets replace them in M9 / M10, IMHO.

Comment 8

[Alex Faaborg [:faaborg] (Firefox UX)]

>assuming that there's no performance hit. I don't see why there would be

Who wrote our png decoder?  It would be great to get their feedback on any possible performance hit.  Or we could just check the images in and see what happens.

>will new icon sets replace the existing png files?

Yes, nearly every png will be replaced.

>otherwise i'll have to come up with new compressed file sets based on
>the new files instead.

We will probably have the design firm replicate your steps when they are creating final files for us so we can just drop them in.

Comment 9

[Robert Sayre]

Why do we think this will speed us up?

Comment 10

[u88484]

(In reply to comment #9)
> Why do we think this will speed us up?
> 

I think he meant optimized as in size...smaller footprint.

Comment 11

[Dão Gottwald [:dao]]

(In reply to comment #10)
> (In reply to comment #9)
> > Why do we think this will speed us up?
> > 
> 
> I think he meant optimized as in size...smaller footprint.

'footprint' is about memory use, not file sizes. Images are decoded anyway, so this shouldn't be a win.

Compressing images png images more aggressively could make decoding more expensive, which is why beltzner added 'perf', I think.

Comment 12

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

thought I'd add this well, since new PNG sets are coming in:

an example is that in mozilla\toolkit\themes\winstripe\mozapps\profile there is a file called profileicon.png

its original size is 826 bytes and is originally encoded as RGB. Recompressing this using PNGGauntlet doesn't bring about any size reduction initially.

On doing a colourcube analysis in GIMP, it has less than 256 colours (<200 IIRC) so the mode can be changed to "indexed". On saving with GIMP it becomes 689 bytes, and using the PNGGauntlet tool the size is brought down to 644 bytes.

22% decrease again, this time using "indexed" mode once the total number of colours is below 256. the method can thus be applied in conjunction with PNGGauntlet's "brute force" option as well.

Not sure about the performance hit for if both are used together though.

Comment 13

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: Search-provider-bkgnd.png (1074 bytes)

example. original size of file in M8 is 1225 bytes.

Comment 14

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

another example:

mozilla\browser\themes\pinstripe\browser called Search-provider-bkgnd.png has original size 1225 bytes. it can be brought down to 1074 bytes using the 2 methods in comment #12.

the 2 methods in conjunction would only affect files that were originally RGB and had less than 256 colours. Most PNGs aren't, so only the PNGGauntlet option can be used.

Comment 15

[Robert Sayre]

(In reply to comment #12)
> 
> On doing a colourcube analysis in GIMP, it has less than 256 colours (<200
> IIRC) so the mode can be changed to "indexed".

This sounds like an optimization that would not slow down the decoder, but make the file smaller.

I would be interested in trying this.

Comment 16

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

marking depends on bug #399398

Comment 17

[Glenn Randers-Pehrson]

(In reply to comment #11)

> Compressing images png images more aggressively could make decoding more
> expensive

Aggressive compression using a higher zlib level does not make decoding more expensive.  If the file is smaller there is less of it to decode, so that is a win.  On the other hand, aggressive compression using the PNG filters, especially the Paeth filter, does increase decoding time somewhat.

Comment 18

[Mike Connor [:mconnor]]

Very interesting, but not blocking at this point.

Comment 19

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: PNGs, part 1 of 3

the PNGs (from the /browser directory only) are recompressed from beta 3 (M11) source, using PNGGauntlet.

This archive comprises of files from part of the winstripe folder, all hierarchy has been retained.

Comment 20

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: PNGs, part 2 of 3

This archive comprises of files from the other part of the winstripe folder, the full pinstripe folder and part of gnomestripe folder; all hierarchy has been retained.

Comment 21

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: PNGs, part 3 of 3

This archive comprises of files from part of the gnomestripe folder and the remaining folders; all hierarchy has been retained.

Comment 22

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Some stats:

282 PNG files in the /browser directory and subdirectories.

Before: 1,008,523 bytes
After: 878,325 bytes
(130,198 bytes saved, or ~13% decrease)

Comment 23

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Attachment: PNGs, addendum

(In reply to comment #15)

I have converted 19 applicable PNGs (in their respective directory hierarchies) to indexed mode in GIMP after ensuring they have less than 256 colours, then used PNGGauntlet on them, and there was an additional decrease in size:

Before: 13,812 bytes
After: 12,030 bytes
(1,782 bytes saved, or a further ~13% decrease)

These files are also found in the previous patches (but without conversion to indexed), but they are separated so a choice can be made.

Comment 24

[Alex Faaborg [:faaborg] (Firefox UX)]

I would like Dolske to weigh in on if there could be any possible perf regression for decoding a PNG that has gone through this process.  Saving the user a second of download time obviously isn't worth messing up Txul.

Comment 25

[Justin Dolske [:Dolske]]

I wouldn't expect a problem here (also see comments from others above).

Conventional wisdom seems to be that a lot of startup time is due to disk IO, so smaller files would be a win in that respect... Optimize away!

(In reply to comment #23)

> I have converted 19 applicable PNGs (in their respective directory 
> hierarchies) to indexed mode in GIMP after ensuring they have less than 256
> colours

Just be careful doing that; some images can get *bigger*, due to color table overhead. Saving a few hundred bytes per image isn't going to have a measurable perf impact, and probably isn't a good use of your time (unless you have this all automated or are bored :-).

Comment 26

[[:Aleksej]]

Have you tried using pngcrush? http://en.wikipedia.org/wiki/pngcrush

Comment 27

[Alex Faaborg [:faaborg] (Firefox UX)]

>Conventional wisdom seems to be that a lot of startup time is due to disk IO,
>so smaller files would be a win in that respect... Optimize away!

Sounds good.

All of the Firefox 3 themes will be having a lot of new files landing over the next several weeks as we receive new icons and try out new designs.  It would be the most efficient to declare some type of "icon freeze" shortly before RC1 when we go ahead and optimize every file, working on the assumption that every icon is final.  After that we'll need to catch all the late-icon (late-i2n?) additions.

To get updates on the progress of the theme go to the icon inventory and select project status:
http://people.mozilla.com/~faaborg/files/granParadisoUI/icons/iconInventory.html

Even icons that are listed as complete on all 4 platforms might still be receiving small changes if one of the designers decides to tweak something, so I think a freeze is the simplest way of making sure we optimize all of the files before shipping.

Comment 28

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

(In reply to comment #25)
> Just be careful doing that; some images can get *bigger*, due to color table
> overhead. Saving a few hundred bytes per image isn't going to have a measurable
> perf impact, and probably isn't a good use of your time (unless you have this
> all automated or are bored :-).
> 

Yeah, I got bored I guess, :)

(In reply to comment #26)
> Have you tried using pngcrush? http://en.wikipedia.org/wiki/pngcrush
> 

PNGOut, which PNGGauntlet makes use of, mostly bests pngcrush in my own personal tests.

(In reply to comment #27)
> 
> Even icons that are listed as complete on all 4 platforms might still be
> receiving small changes if one of the designers decides to tweak something, so
> I think a freeze is the simplest way of making sure we optimize all of the
> files before shipping.
> 

Will that freeze be known at a later date?

Comment 29

[Alex Faaborg [:faaborg] (Firefox UX)]

>Will that freeze be known at a later date?

Yep, I'll come back to this bug to keep people updated on the progress of the themes, Also if you keep an eye on the inventory you will be able to track how the various teams are progressing.

Comment 30

[Kai Liu]

(In reply to comment #28)
> PNGOut, which PNGGauntlet makes use of, mostly bests pngcrush in my own
> personal tests.
> 

pngout -rem alla -brute in.png out.png almost always beats pngout for me.  In a few cases, pngout wins, but that doesn't happen very often in my personal tests.

Comment 31

[Kai Liu]

(In reply to comment #30)
> pngout -rem alla -brute in.png out.png
> 

Er, s/pngout/pngcrush/  :)

Comment 32

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Unassigning myself, post discussion with faaborg, as there is a better and more automated solution in place.

Comment 33

[Alex Faaborg [:faaborg] (Firefox UX)]

The past 5 or 6 icon drops have included loss-less compression with optipng
0.5.5, and testing with pngcheck 1.99.3 as part of the automated process of
generating the icon drop.  Gary: thanks for letting me know about png
compression, before you filed the bug I didn't know that they could be
compressed more than the default export settings.