630845 - Compress uploaded images

Status: VERIFIED FIXED

(support.mozilla.org :: Knowledge Base Software, task)

Description

[Kadir Topal [:atopal]]

Most applications use inefficiant compression (for several reasons), so there is some potential to compress them even further. We should hook a sercice for that into our media gallery upload system.

There are several alternatives: PNGcrush runs locally, www.punypng.com is a hosted service with APIs.

Comment 1

[James Socol [:jsocol, :james]]

We should do something locally. We don't want to depend on an external service that may be slow or down when someone uploads a large image.

PNGCRUSH works for pngs but I don't think for other formats. Of course we can always save everything as png, then crush it.

Comment 2

[Kadir Topal [:atopal]]

Right, we should always convert gifs to pngs, and PNGCRUSH was the only thing I knew about, there are probably other tools for JPEG.

Comment 3

[Paul Craciunoiu [:paulc]]

FWIW, I've used pngcrush locally and it's easy to use. I imagine it would work ok with python.

Saving everything as PNG would be good practice, too. That would be bug 619629, I think.

Comment 4

[Frédéric Kayser]

Pngcrush is outdated (it writes every attempt to disk), optipng (http://optipng.sourceforge.net/) is a faster and more powerful alternative.

As I explained in bug 631392 Ken Silverman's PNGOUT (http://www.advsys.net/ken/utils.htm) is often used in optimization scripts (like pngslim for Windows http://people.bath.ac.uk/ea2aced/tech/png/), followed by Ben Jos Walbeehm's DeflOpt (http://www.walbeehm.com/download/).
But you'll probably not want to run an optimization tool for hours just to squeeze out a few more bytes, if you tell me your precise goal (time limit, optimization level), the operating system on which you intend to run it, the number of hardware threads available, I could write you a script tailored to your needs.

PNGOUT can also convert a single frame gif (not animations) to a png file.

Gifsicle (http://www.lcdf.org/gifsicle/) is the tool usually used to optimize gifs, jpegtran from libjpeg (http://en.wikipedia.org/wiki/Libjpeg) can be used to recompress jpgs without hurting visual quality.

Comment 5

[James Socol [:jsocol, :james]]

Frédéric, ideally what we want is something that can do a reasonable job compressing pngs (we'll convert to png with PIL if need be) in a reasonable time (let's say a cap of 2 seconds) that has a simple CLI interface (i.e.: `<binary> <filename>` would be good) or even better, Python bindings.

We'll be running on RHEL in production and RPMs would be a great help, though the ability to build an RPM is good enough, and it also will need to build on (as a minimum) OS X.

Thanks for the pointer toward optipng, will definitely look at that.

Comment 6

[Tanay Gavankar [:tgavankar]]

Maybe something like https://github.com/thebeansgroup/smush.py... we could fork it and replace the processing libs with others (maybe) - particularly replacing pngcrush with optipng or something.

Comment 7

[James Socol [:jsocol, :james]]

(In reply to comment #6)
> Maybe something like https://github.com/thebeansgroup/smush.py...

Interesting! My biggest concern with that is the dependence on a patched library (pngnq).

I think 1) convert to png, 2) run optipng is probably the process to go with, here. That probably gets us the best ratio of quality/size.

Comment 8

[Tanay Gavankar [:tgavankar]]

I have most of the compression set up for newly uploaded images, but are we going to try to do something about the already uploaded images (or just leave them as is)?

Comment 9

[Tanay Gavankar [:tgavankar]]

Also, do we want this compression to apply to all uploaded images (attachment to questions vs gallery uploads, etc)? I know that there are two separate views for uploads for those two examples at least, so what exactly do we want to force to PNG/compress?

Comment 10

[Tanay Gavankar [:tgavankar]]

Done in https://github.com/jsocol/kitsune/commit/2bb577.

I filed bug 704873 for IT to install OptiPNG (which is needed for any of this to work), so hopefully that'll be done.

In the meantime, since this has already landed on master, I'm going to mark it as RESO FIXED.

Comment 11

[Tanay Gavankar [:tgavankar]]

Steps to verify: since technically this should be a transparent patch (no visible changes) until optipng is installed (bug 704873), just test all possibilities for uploading images (gallery, wiki, questions attachment, etc) and make sure they are functioning correctly.

Comment 12

[Tanay Gavankar [:tgavankar]]

Also see bug 619629 as to why even with compression the resulting images may not be strictly smaller than the original input.

Comment 13

[Rebecca Billings [:rbillings]]

Verified all uploading functions work [and edit/cancel/delete].