1602220 - content-type: image/webp file is saved with jpeg extension if URL has `.jpeg` extension, `image/webp` is unknown and `jpeg` extension is known in handlers.json

Status: RESOLVED FIXED

(Firefox :: File Handling, defect, P3)

Description

[Jerry]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0

Steps to reproduce:

Hello,
I have a serious problem with your newest Firefox v70. If I try to save an image from a web site the image is saved as .jpeg instead of .webp. Such file can not be displayed by ACDSee 2019,2020 versions. Internal signature of the image file is webp. See here Greta image:

https://www.novinky.cz/koktejl/clanek/ekologicka-cestovatelka-casem-dvojnice
-grety-thunbergove-se-objevila-na-fotce-z-roku-1898-40304439?seq-no=1&dop-ab
-variant=100&source=article-detail

Actual results:

There is a total image file format disarray while images saving. Instead of correct .webp extension the files are saved with .jpg or .jpeg extension.

I have Windows 7 x64 Profi SP1 + all attainabe updates.

Expected results:

Can you please correct this problem ? I sent an apply to ACDSee company, but they replied me that the problem is inside the Mozilla Firefox. It is all image files with internal signature webp. have to be saved with .webp extension and all files with -jpeg/jpg extension have to be saved with jpg/jpeg extension.

Comment 1

[Gingerbread Man]

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0
20191207092617

Server responds with content-type: image/webp but doesn't specify the filename in the response headers.

• Release saves ft8hP9.jpeg
• Nightly saves ft8hP9.jpeg.webp
• Vivaldi 2.9 saves ft8hP9.webp

Comment 2

[Jerry]

Hi, yes, the "bug report" I created reports really existing problem in Firefox v70-v73.
Please correct the problem so as a new version of Firefox was capable to
save all webp files which have .webp signature in the image file header
with .webp extension and all files which have jpeg/jpg signature in their
file header as .jpeg/.jpg. The biggest problem is that ACSDee v 2019/2020
is not capable to display a file which has .jpeg/.jpg extension and webp
signature in the file header and contrary.
Many thanks
Jerry

Comment 3

[:Gijs (he/him)]

(In reply to Jerry from comment #2)

Hi, yes, the "bug report" I created reports really existing problem in Firefox v70-v73.

Comment #1 suggests that we correctly set a .webp extension in 73, and I see the same thing. Can you re-test to make sure we're seeing the same thing? Also, how exactly are you saving the image? Context menu and then "save image as", or something else?

It looks like the fix for this is on the nightly channel but not making its way to release. It probably has to do with the accept header for images and the general/default accept header, like bug 1526731. Bizarrely, it seems like the problem here is the opposite of what it is there: here, there's a webp image that we save as jpg. There, we get a png image and save it as webp.

I expect that the fix is the same as point 4 in bug 1526731 comment 15 - the webbrowserpersist code should know what kind of thing it's being asked to save.

Comment 4

[Jerry]

Hi, from where I can download the newest and unstable beta Firefox v73. I still have only v71.
So I am not able to re-test the newly inserted function of webp.
Jerry

Comment 5

[:Gijs (he/him)]

(In reply to Jerry from comment #4)

Hi, from where I can download the newest and unstable beta Firefox v73. I still have only v71.
So I am not able to re-test the newly inserted function of webp.
Jerry

https://nightly.mozilla.org/ :-)

Comment 6

[Jerry]

Hi there,

I tested the "Firefox 73.0a1 (2019-12-12) (64 bit)" version and, surprisingly :) the firefox is really capable
to save this image:

https://www.novinky.cz/koktejl/clanek/ekologicka-cestovatelka-casem-dvojnice
-grety-thunbergove-se-objevila-na-fotce-z-roku-1898-40304439?seq-no=1&dop-ab
-variant=100&source=article-detail

as a .webp image file. So, thank you very much for the bug repair/correction.
I hope that the v73.0a1 will become to be "RTM - release to marked" as soon as possible.

Jerry

Comment 7

[Gingerbread Man]

(In reply to :Gijs (he/him) from comment #3)

Comment #1 suggests that we correctly set a .webp extension in 73, and I see the same thing.

.jpeg.webp is not the correct file extension.

Comment 8

[:Gijs (he/him)]

(In reply to Gingerbread Man from comment #7)

(In reply to :Gijs (he/him) from comment #3)

Comment #1 suggests that we correctly set a .webp extension in 73, and I see the same thing.

.jpeg.webp is not the correct file extension.

Well, it's a lot more correct than just .jpeg, and at least on mac, I do also just see .webp, not .jpeg.webp.

Comment 9

[:Gijs (he/him)]

I wrote patches for some other webp-related issues and assumed that'd solve this, and then I ran into some issues with reproducing before/after the patches.

It seems that the real issue here is in some frontend code and handlers.json. The code in contentAreaUtils.js looks for a default extension passing both the extension determined from the URL (.jpeg in this case) and the mimetype from the content (image/webp). getDefaultExtension then ends up looking up the MIME information object based on both the content type and the extension.

Unfortunately, almost nothing knows about image/webp. We'll first ask the OS. But the OS is likely to know nothing. In that case, we're going to fall back to using the extension instead ( https://searchfox.org/mozilla-central/rev/cfd1cc461f1efe0d66c2fdc17c024a203d5a2fd8/uriloader/exthandler/nsExternalHelperAppService.cpp#2467-2481 ). This breaks, because the extension is bogus - we end up replacing the mimetype with image/jpeg based on the bogus extension and that's how the broken behaviour here happens.

On a clean profile, at least on macOS, this problem does not reproduce because there is no image/jpeg entry in handlers.json. That explains why the reporter couldn't reproduce on nightly - the clean profile meant the problem doesn't happen there.

I'm going to unassign for now because my patches aren't helping here.

But I'm also not really sure what to do here. Unfortunately a lot of the methods that access our mimetype database pass both mimetypes and extensions, and for most decisions, one is more important than the other. The problem is that which one is more important actually differs based on the consumer of the utility methods... if we're opening a file with the OS default handler, we really (on Windows, at least) want to use the file extension, because that's what Windows will use to pick an application to open with, and if we rely on mimetype information sent by the website, bad things could happen (think someone sending a .exe file with a text/plain mimetype or similar).
In this case, when we're determining a file extension, we should privilege the content type above a file extension derived from the URL which could be bogus (e.g. what if the URL ended in imageserver.php - using the .php extension would clearly make no sense).

I can see a few solutions:

1. make all consumers of the mime service pass some indication of whether to prefer mimetype or file extension
2. avoid passing file extension information if the file extension was derived from the URL (and there's a query string? Or maybe just never send it unless it's come from Content-Disposition ?)
3. ensure hardcoded webp mimetype information is available in the mime service (ie if found is false after checking the OS then check extraMimeEntries for the mimetype, and construct mime info from that).
4. all of the above?

Marco/Boris, what do you think, and is there some obvious issue I'm missing here? (Marco, I also seem to recall we had a more clear-cut version of this problem in another bug on file but I can't think of more details that'd help me find it right now... if you recall, I'm all ears.)

Comment 10

[Boris Zbarsky [:bzbarsky]]

Hmm. So the way I would expect the "image/webp + .jpeg" case to work is that we'd get the info for "image/webp", figure out what valid extensions are for it, discover that ".jpeg" really isn't, and then act accordingly.

That "figure out what valid extensions are for it" might need to look at extraMimeEntries if the OS knows nothing, maybe? Would that help here?

In general, this stuff is all a mess, servers send all sorts of random nonsensical stuff, and I see no way to get all the edge cases right even for a human armed with the file command... :(

Comment 11

[:Gijs (he/him)]

(In reply to Boris Zbarsky [:bzbarsky, bz on IRC] from comment #10)

Hmm. So the way I would expect the "image/webp + .jpeg" case to work is that we'd get the info for "image/webp", figure out what valid extensions are for it, discover that ".jpeg" really isn't, and then act accordingly.

That "figure out what valid extensions are for it" might need to look at extraMimeEntries if the OS knows nothing, maybe? Would that help here?

I believe so, yes. It might be the simplest option.

In general, this stuff is all a mess, servers send all sorts of random nonsensical stuff, and I see no way to get all the edge cases right even for a human armed with the file command... :(

:-(

Comment 12

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

(In reply to :Gijs (he/him) from comment #9)

1. make all consumers of the mime service pass some indication of whether to prefer mimetype or file extension

This looks scary to me, we are passing the problem to consumers, and we know some consumers will end up doing the wrong thing.

2. avoid passing file extension information if the file extension was derived from the URL (and there's a query string? Or maybe just never send it unless it's come from Content-Disposition ?)

Maybe we can add more arguments to getFromTypeAndExtension to allow the service to make better fallback choices, like where the extension was derived from?
Or we could have an hardcoded list of common defaults. extraMimeEntries, as suggested by Boris, looks like a good start.

Comment 13

[Jerry]

As far as I know, every image file has a header. in this header there are all necessary information
enabling to identify the file without regard actual extension. There is ONE big problem, that if
a image file is saved with use of wrong extension the ACDSee viewer is NOT capable to display it.
This was the main stumbling block - the reason why I created this thread.
Jerry

Comment 14

[Boris Zbarsky [:bzbarsky]]

in this header there are all necessary information enabling to identify the file without regard actual extension.

If you know specifically it's an image file, this is sort of true. If not, this is false (see e.g. XPM).

More to the point, this code needs to generically work for all files, and needs to work before we have the actual bytes available, in general. All it has to work with is the metadata (URL, HTTP headers).

Comment 15

[Jerry]

ok, life is cruel. :)

Comment 19

[Toddintr]

no moron with IQ of fizzy grass should do that ...

Not acceptable language, and not even nice language.

I also have the same problem, and I have downgraded to 73.0. You can do the same as a workaround, while waiting for a fix. Language like the above is not going to help, especially during the extraordinary times we are experiencing.

Comment 20

[Jerry]

ok, not nice language .. you can delete the message

Comment 21

[:Gijs (he/him)]

As a workaround, I wouldn't advise using an out-of-date version of Firefox. Here's an easier workaround:

1. open an up-to-date copy of Firefox (74 or above)
2. go to https://gijsk.com/temp/webp-fix.html (I just set this up on my own site, be nice).
3. click the link
4. You'll get asked what to do with the image (which, fwiw, is the rose from https://developers.google.com/speed/webp/gallery )
5. pick "open with" and select any program you like (other than the OS default, which might be Chrome if you're on Windows and have it installed). Doesn't matter, could be notepad (obviously the result won't be helpful in that case...).

Try the testcase from comment #0 (or from the duplicate) again, and it should work (it does for me, at least).

The reason is that by doing this you've saved some (correct!) information about .webp files and the image/webp mimetype to your handlers.json file. You can go to the Firefox Options, search for "applications" and you'll see a big list of extensions / file types, one of which is webp (again, if you have Chrome installed, it might be called "Chrome HTML Document", because Chrome is being "helpful" -- you can verify the entry is for webp by opening the dropdown, clicking "Use other..." and noting the mimetype under the label being "image/webp").
You can now change that to some other default action ("always ask" or "save to disk" or whatever). Just having the information there is enough to work around the issue.

Of course, this will mean that the webp images you download will still be webp images (but that was already happening, they were just getting the wrong extension). If you wanted to use them with an external tool, that won't necessarily help you. There are add-ons available that will do the conversion, or you can use an external tool that knows how to read webp images, like imagemagick or photoshop (with google's free plugin).

Comment 22

[:Gijs (he/him)]

I believe I fixed this in bug 1366645. That fix is currently in 78 beta, and should be released end of June.