Last Comment Bug 500500 - (JPEG-XR) Add support for JPEG-XR/HD Photo
(JPEG-XR)
: Add support for JPEG-XR/HD Photo
Status: REOPENED
[parity-ie]
: feature, student-project
Product: Core
Classification: Components
Component: ImageLib (show other bugs)
: Trunk
: All All
: -- normal with 24 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-06-25 13:31 PDT by Jeff Muizelaar [:jrmuizel]
Modified: 2016-02-18 18:04 PST (History)
59 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description Jeff Muizelaar [:jrmuizel] 2009-06-25 13:31:34 PDT
The first step in making this possible is to develop a free library for decoding JPEG-XR. Ideally it would be released under the same license as libjpeg.

Information on the format is available at: http://www.microsoft.com/whdc/xps/hdphotodpk.mspx
Comment 1 David Humphrey (:humph) 2009-06-25 13:49:47 PDT
See http://en.wikipedia.org/wiki/HD_Photo#Licensing for info on licensing, specifically related to the Device Porting Kit code and copyleft licenses.
Comment 2 Jeff Muizelaar [:jrmuizel] 2010-03-16 14:22:27 PDT
The developer preview for IE9 has support for jpeg-xr
Comment 3 Please Ignore This Troll (Account Disabled) 2010-05-12 06:27:47 PDT
(In reply to comment #1)
> See http://en.wikipedia.org/wiki/HD_Photo#Licensing for info on licensing,
> specifically related to the Device Porting Kit code and copyleft licenses.

(In reply to comment #2)
> The developer preview for IE9 has support for jpeg-xr
And?
Sorry that I get so upset, but you mentioning this seems to mean two things:
- Either you belong to the big horde of people which are too lazy to follow links (I mean, it's a click with the mouse, come on, that's unbelievable).
- Or you didn't understand that hard to read legal stuff - I mean: it's not thaaat hard. Although I admit it can be confusing.

So, here is my simplified explanation: Mozilla is published under a mix of a GNU/BSD like license. But the MS license explicitly forbids GNU like licenses to be used with the JPEG XR source kit.
So: Anyone who codes and or publishes a Mozilla Firefox with the JPEG XR kit coded in it, is a criminal and can be sued by MS. They would have every right to do so. That's it.

The real solution here is JPEG2000, which sadly isn't implemented either.

If you meant: developing a library from scratch, just by the specification. Why the effort? There are already ready-made JPEG2000 libs with compatible licenses. Why put so much work into a MS format that only a MS browser supports, which even isn't released yet?
Comment 4 WulfTheSaxon [:Wulf] 2010-05-12 11:31:38 PDT
(In reply to comment #3)
Or perhaps comment 2 was just suggesting the parity-IE9 tag. Also, AFAIK, no JPEG2000 library currently supports progressive decoding, which would be desired for use in a browser. (Note that I'm not a fan of JPEG-XR, and would love to see JPEG2000 support -- just being the "devil's advocate" here.)

See also: Bug 36351
Comment 5 Gregory Maxwell 2010-07-12 15:33:15 PDT
It's worth mentioning that XR has alpha channel support, which would address the need driving people to periodically beg for JNG.
Comment 6 Please Ignore This Troll (Account Disabled) 2010-07-13 10:05:50 PDT
(In reply to comment #5)
> It's worth mentioning that XR has alpha channel support, ...
Exactly as JPEG2000 has. Transparency _and_ alpha channels.
Comment 8 Please Ignore This Troll (Account Disabled) 2010-10-06 03:58:02 PDT
(In reply to comment #7)
> http://www.microsoft.com/interop/cp/default.mspx
This isn't really helping. You still have to maintain a function-wise exact copy of the MS lib.
Comment 9 myutwo33 2010-11-18 13:11:34 PST
I may be wrong here but it seems the Wikipedia article may be out of date regarding licencing problems.

ITU/ISO/IEC in July this year released source code for a JPEG-XR encoding/decoding library ( http://www.itu.int/rec/T-REC-T.835 ) with the following licence:


/*************************************************************************
*
* This software module was originally contributed by Microsoft
* Corporation in the course of development of the
* ITU-T T.832 | ISO/IEC 29199-2 ("JPEG XR") format standard for
* reference purposes and its performance may not have been optimized.
*
* This software module is an implementation of one or more
* tools as specified by the JPEG XR standard.
*
* ITU/ISO/IEC give You a royalty-free, worldwide, non-exclusive
* copyright license to copy, distribute, and make derivative works
* of this software module or modifications thereof for use in
* products claiming conformance to the JPEG XR standard as
* specified by ITU-T T.832 | ISO/IEC 29199-2.
*
* ITU/ISO/IEC give users the same free license to this software
* module or modifications thereof for research purposes and further
* ITU/ISO/IEC standardization.
*
* Those intending to use this software module in products are advised
* that its use may infringe existing patents. ITU/ISO/IEC have no
* liability for use of this software module or modifications thereof.
*
* Copyright is not released for products that do not conform to
* to the JPEG XR standard as specified by ITU-T T.832 |
* ISO/IEC 29199-2.
*
* Microsoft Corporation retains full right to modify and use the code
* for its own purpose, to assign or donate the code to a third party,
* and to inhibit third parties from using the code for products that
* do not conform to the JPEG XR standard as specified by ITU-T T.832 |
* ISO/IEC 29199-2.
* 
* This copyright notice must be included in all copies or derivative
* works.
* 
* Copyright (c) ITU-T/ISO/IEC 2008, 2009.
***********************************************************************/


I don't know if there's anything here that is incompatible with the Mozilla licence or makes developers wary of legal issues, but it seems much more usable than the licence bundles with the Device Porting Kit.
Comment 10 Mike Shaver (:shaver -- probably not reading bugmail closely) 2010-11-18 16:19:13 PST
This points to the concern, I believe:
> * Those intending to use this software module in products are advised
> * that its use may infringe existing patents. ITU/ISO/IEC have no
> * liability for use of this software module or modifications thereof.
Comment 11 WulfTheSaxon [:Wulf] 2010-11-18 16:58:47 PST
(In reply to comment #9)
> * Copyright is not released for products that do not conform to
> * to the JPEG XR standard as specified by ITU-T T.832 |
> * ISO/IEC 29199-2.

Not sure how well that'd fly, either.
Comment 12 myutwo33 2010-11-18 18:02:45 PST
(In reply to comment #10)
> This points to the concern, I believe:
> > * Those intending to use this software module in products are advised
> > * that its use may infringe existing patents. ITU/ISO/IEC have no
> > * liability for use of this software module or modifications thereof.

As far as I can tell this is just ITU/ISO/IEC covering their asses in case someone in the future decides they own patents that are used in JPEG-XR, as has happened in the past with jpeg. Microsoft have added jpeg-xr to their community promise ( http://www.microsoft.com/interop/cp/default.mspx ) so I wouldn't expect any issues from them.

(In reply to comment #11)
> (In reply to comment #9)
> > * Copyright is not released for products that do not conform to
> > * to the JPEG XR standard as specified by ITU-T T.832 |
> > * ISO/IEC 29199-2.
> 
> Not sure how well that'd fly, either.

As I understand it, the explanation for this is to stop people from trying to make their own fork of the standard, on top of making people confident that all JPEG-XR software can read all JPEG-XR images etc. I wouldn't worry about this one.
Comment 13 Jeff Muizelaar [:jrmuizel] 2010-11-18 20:33:41 PST
I don't really think the reference implementation is suitable for use in Firefox for technical reasons so its copyright license doesn't really matter here.
Comment 14 WulfTheSaxon [:Wulf] 2010-11-18 20:43:54 PST
(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #9)
> > > * Copyright is not released for products that do not conform to
> > > * to the JPEG XR standard as specified by ITU-T T.832 |
> > > * ISO/IEC 29199-2.
> > 
> > Not sure how well that'd fly, either.
> 
> As I understand it, the explanation for this is to stop people from trying to
> make their own fork of the standard, on top of making people confident that all
> JPEG-XR software can read all JPEG-XR images etc. I wouldn't worry about this
> one.
I can understand their reasoning, I just don't think it's GPL-compatible.

(In reply to comment #13)
> I don't really think the reference implementation is suitable for use in
> Firefox for technical reasons so its copyright license doesn't really matter
> here.
Well, it could assumedly provide a good starting point.
Comment 15 Please Ignore This Troll (Account Disabled) 2010-11-19 13:31:09 PST
Well, the reference implementation could be indeed used for a first good starting point.

Where I'm really unsure:
Does the posted part of the license covers the whole reference implementation? Or does it have other licenses included, or references other libraries with other licenses, which do not work together with GPL and BSD?
Comment 16 myutwo33 2010-11-19 14:43:37 PST
(In reply to comment #15)
> Well, the reference implementation could be indeed used for a first good
> starting point.
> 
> Where I'm really unsure:
> Does the posted part of the license covers the whole reference implementation?
> Or does it have other licenses included, or references other libraries with
> other licenses, which do not work together with GPL and BSD?

After having a look through the code myself, the only other library included with the reference code is my_getopt, which is under an MIT licence. The licence I posted earlier (which is the licence in it's entirety) covers all of the other code.
Comment 17 Kailas 2011-03-25 01:31:04 PDT
Jeff: Why the reference implementation is not suitable for FF. Could you put some light on technical reasons?
Comment 18 Jeff Muizelaar [:jrmuizel] 2011-03-25 06:28:05 PDT
(In reply to comment #17)
> Jeff: Why the reference implementation is not suitable for FF. Could you put
> some light on technical reasons?

Two main reasons:

1. Performace. From my initial tests, it looks like the reference JPEG-XR implementation is about 10x slower to decode than libjpeg. 

2. Security. I don't know that the reference implementation has been hardened against malicious inputs and from my quick look it appears that it has not.
Comment 19 Bobby Holley (PTO through June 13) 2011-10-07 16:04:52 PDT
I don't think this is a WONTFIX. We're not going to ship the reference implementation, but we may very well write our own implementation.

At the very list, I think more discussion is in order.
Comment 20 Doug Sherk (:drs) (inactive) 2011-10-07 17:04:33 PDT
My mistake, thanks! I was misusing WONTFIX.
Comment 21 myutwo33 2011-10-09 02:31:20 PDT
Just wondering if there's been any change in momentum for the case for having this bug resolved? Particularly in light of the latest version of Flash Player containing native support Jpeg-XR. Interestingly Jpeg-XR support was added to Flash Player primarily for the case of games made with the Molehill 3D API. This is due to the fact that it can compress textures more efficiently and that it supports alpha transparency natively. More can be read about their reasons for implementing the standard at: http://blog.kaourantin.net/?p=116

I can see this being highly relevant to HTML5 games and games which use OpenGL. I think moving forward there'll be more and more demand for an image format supports lossy compression and alpha transparency for use in these areas. Interesting I think too that Jpeg-XR is the format chosen for storing all of the textures in ID's Rage which uses the Tech 5 engine. I imagine the chose the format due to it's ability to decompress regions of a Jpeg-XR image without having to decode the entire file. This could potentially allow for some nice optimisations down the road for OpenGL games which render based off a large Jpeg-XR texture file... or perhaps not.

I'm sure there may be other work arounds to solve the problem of lossy textures that need alpha channels, without having to add support for a new image format. I'm curious though to hear if anyone has any more thoughts on the matter?
Comment 22 myutwo33 2011-10-09 05:48:57 PDT
(In reply to myutwo33 from comment #21)

One more thing about Jpeg-XR possibly presenting a benefit to OpenGL games and applications... The fact the Jpeg-XR supports many different bit-depths sounds like a great advantage. Lower bit depths such as RGB555 present an opportunity for a developer to save GPU memory (and download speeds) while higher bit depth such as float32/64bpp present opportunities for high dynamic range textures to be used in OpenGL applications. To me, these sounds like exciting scenarios that Jpeg-XR would be extremely well suited to enabling which would be very difficult to enable with existing supported image formats.
Comment 23 xo2yo2zo2 2013-04-14 23:22:09 PDT
(In reply to Jeff Muizelaar [:jrmuizel] from comment #0)
> The first step in making this possible is to develop a free library for
> decoding JPEG-XR. Ideally it would be released under the same license as
> libjpeg.

They released one the other day (or at least I found one the other day).

Blog post: http://hdview.wordpress.com/2013/04/11/jpegxr-photoshop-plugin-and-source-code/

BSD-licensed library: https://jxrlib.codeplex.com/
Comment 24 Jeff Muizelaar [:jrmuizel] 2013-04-15 01:01:08 PDT
(In reply to xo2yo2zo2 from comment #23)
> BSD-licensed library: https://jxrlib.codeplex.com/

Thanks for pointing this out. I'll take a look.
Comment 25 myutwo33 2013-06-16 04:44:46 PDT
Microsoft have posted some new metrics for JPEG XR vs WebP using jxrlib 1.1. Seems to show JPEG XR in a more favourable light. I'm not good enough at reading these kinds of metrics to know if they're badly skewed or not. Food for thought?

http://hdview.wordpress.com/2013/05/30/jpegxr-updates/
Comment 26 camerontnorman 2013-10-19 21:08:16 PDT
Are there any plans to support this codec? JPEG XR seems pretty stable, the patent issues seem to be sorted out with MS's Community Promise (http://archive.is/DZfS4), the licensing issues are null with the release of the BSD licensed library (https://jxrlib.codeplex.com/), and it seems to offer desired improvements to JPEG including:

* increased compression
* better color accuracy
* transparency, through an alpha channel

Are there any objections to JPEG XR, or is the issue just a lack of manpower?
Comment 28 Jonas Nordlund 2014-01-24 00:33:44 PST
JPEG XR is a codec that is about more than compression efficiency and I would love to have this to maintain e.g. dynamic range from RAW's. Photographers regularly deal with 12 and 14 bit per channel RAW photos, but common JPEG implementations really just seem to compress down to 8 bits per color channel. This is one of the core differences between a RAW photo and a JPEG photo. Of course, WebP also supports more bits per channel so that's another option here. JPEG or HEVC-MSP, however, aren't.
Comment 29 myutwo33 2014-01-24 04:04:57 PST
(In reply to Jonas Nordlund from comment #28)
> WebP also supports more bits per channel
WebP actually doesn't support more colour depth than JPEG. Like JPEG it only supports 8-bits per channel. I consider this to be one of the major flaws in the format.
Comment 30 Juan 2014-03-06 14:26:15 PST
JPEG XR library is BSD licensed.

I hope mozilla implement JPEG XR decoder soon.
Comment 31 Ted Kapustin 2014-04-16 23:49:51 PDT
A support for JPEG-XR should be added soon to Firefox. Stay tuned.
Comment 32 Chris Peterson [:cpeterson] 2014-07-23 23:28:01 PDT
See also:

Chromium Issue 56908: Add JPEG XR support
https://code.google.com/p/chromium/issues/detail?id=56908
Comment 33 Ted Kapustin 2014-07-24 10:04:40 PDT
(In reply to Kailas from comment #17)
> Jeff: Why the reference implementation is not suitable for FF. Could you put
> some light on technical reasons?

The reference implementation did not support the decoding of a partially available image. Nor did it support progressive decoding. If you use IE with a slow connection you will see that its JPEG-XR decoder is lame. Apparently, they modified it in order to do a progressive decoding (when the image is in a corresponding layout), but it does not works as it should. And when an image is in SPATIAL mode, or has a planar alpha layer, it simly downloads the entire image before decoding it.
I fixed hundreds of bugs in the reference implementation and supported the progressive decoding, as well as the decoding of images that are partially available. The code is now being tested by Microsoft Open Technologies Inc. and hopefully will become part of the Firefox build soon.
Comment 34 Ted Kapustin 2014-07-24 10:20:37 PDT
(In reply to Please Ignore This Troll (Account Disabled) from comment #15)
> Well, the reference implementation could be indeed used for a first good
> starting point.
> 
> Where I'm really unsure:
> Does the posted part of the license covers the whole reference
> implementation? Or does it have other licenses included, or references other
> libraries with other licenses, which do not work together with GPL and BSD?

Like I mentioned earlier, a Firefox decoder for JPEG-XR images is now being tested by MS OpenTech. I am not sure when/whether it will be handed to Mozilla Foundation. The decoder itself is about 2500 lines of code. A modified decoding part of JXRLib comes with it. Until the decision is made by Microsoft Open Technologies, I can't share the patch with anybody. But I can share the binaries (Win32, Linux64, Mac OS X and Android),
Comment 35 Chris Peterson [:cpeterson] 2014-09-23 01:13:44 PDT
(In reply to Ted Kapustin from comment #34)
> Like I mentioned earlier, a Firefox decoder for JPEG-XR images is now being
> tested by MS OpenTech. I am not sure when/whether it will be handed to
> Mozilla Foundation. The decoder itself is about 2500 lines of code. A
> modified decoding part of JXRLib comes with it. Until the decision is made
> by Microsoft Open Technologies, I can't share the patch with anybody. But I
> can share the binaries (Win32, Linux64, Mac OS X and Android),

Ted, is there anything Mozilla can do to help MS OpenTech move forward with your patch? Can you share contact information for someone at MS OpenTech?
Comment 36 Jeff Muizelaar [:jrmuizel] 2015-01-14 06:45:53 PST
Just to clarify, the current feeling is that JPEG-XR's compression compression performance is generally worse than JPEG's and thus even though JPEG-XR adds additional features it's probably not worth adding support for.

Note You need to log in before you can comment on or make changes to this bug.