1688306 - Firefox does not respect content-disposition filename

Reporter

Description

•

5 years ago

Attached image response.png — Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36 OPR/73.0.3856.344

Steps to reproduce:

Setup a webserver serving the wrong content-type but correct content-disposition
Attempt to download said file

Actual results:

Firefox will ignore the file extension present on content-disposition and replace it with the extension assumed/derived from content-type.

Expected results:

Firefox should have respected the filename with extension provided by the Content-Disposition header.

Fabricio

Reporter

Comment 1

•

5 years ago

Looking at the bug tracker, I noticed one recent bug that is very similar 1683581 (I'm assuming in that case the Content-Type was correct but firefox still didn't respect the Content-Disposition filename).

I believe this issue was introduced on Firefox 83 but I'm not entirely certain, this was caught by our internal test automation software and Chrome saves the file with the extension provided by the filename (as did older firefox versions).

The file is a XLS served by a government website so I can't link it, however this should be reproduced provided both Content-Type and Content-Disposition are defined. Filename should be demands_aguardando_resposta.xls yet firefox saves it as demandas_aguardando_resposta.pdf.

Fabricio

Reporter

Comment 2

•

5 years ago

One more update, now with some more concrete details as I can't edit the last comment it seems.

This was detected by our automated testing software (running Selenium) on January 7th, looking at the public changelog page there was an update (84.0.2) on January 6th for a security fix (I tried looking at the diff however I don't have permission to look at the bug details), so this may be related.

Looking at our pod rotation logs I can confirm we had a pod rotation on December 9th that also caused an update to whatever was latest at the time, however this ran for several weeks without triggering an issue, which indicates that the issue was not yet present on that release.

Daniel Bodea [:danibodea]

Comment 3

•

5 years ago

This issue is too complex for me to attempt reproduction/confirmation, but I will even attempt to find the regressor if I'll receive a test case or exact steps to reproduce.

I am setting the File Management component. If incorrect, please set a more appropriate one.

status-firefox84: --- → affected

Component: Untriaged → File Handling

Keywords: regression

Fabricio

Reporter

Comment 4

•

5 years ago

Hey Bodea,

I've set up a one-click repro for this issue, you can access it at https://notfab.net/firefox.html

Once clicking on any of the two anchors on the page, a file will attempt to download, it's a XLSX but you will be able to observe that firefox will attempt to save it as a PDF, it's also possible to check the headers on this page via the network tab on dev tools.

I've also set up the exact same thing but with pdf to mp4 (the second anchor), as a way to demonstrate that this is not linked to a xls/pdf issue but more broadly as how firefox is deriving the extension from the Content-Type.

The HTML page is pretty basic:

<html>
        <head>
                <title>firefox test</title>
        </head>
        <body>
                <a href="firefox.xlsx">click 4 bug (xlsx -> will download pdf)</a>
                <br/>
                <br/>
                <a href="firefox.pdf">click 4 bug (pdf -> will download mp4)</a>
        </body>
</html>

And the desired header effects are made by modifying nginx:

location ~ \.xlsx$ {
    add_header 'Content-Disposition' 'attachment;filename="firefoxx.xlsx"';
    add_header 'Content-Type' 'application/pdf;charset=UTF-8';
}

location ~\.pdf$ {
    add_header 'Content-Disposition' 'attachment;filename="firefoxx.pdf"';
    add_header 'Content-Type' 'video/mp4';
}

Tom S. (please needinfo tschuster)

Comment 5

•

5 years ago

This is basically what the intended behavior since bug 1667787 looks like. It's a lot more common for the file extension to be wrong than the Content-Type header. Not much we can do if the Content-Type is intentionally? wrong.

Regressed by: 1667787

BMO Automation

Updated

•

5 years ago

Has Regression Range: --- → yes

:Gijs (he/him)

Comment 6

•

5 years ago

How is this happening in practice? Why is the web server sending application/pdf when it is just not a pdf, and why can't you fix the web server to send the correct mimetype? That seems much more straightforward.

As comment 5 pointed out, it's much more common for the filename to be wrong than the content type...

Flags: needinfo?(git)

Fabricio

Reporter

Comment 7

•

5 years ago

Hey!

(In reply to :Gijs (he/him) from comment #6)

How is this happening in practice? Why is the web server sending application/pdf when it is just not a pdf, and why can't you fix the web server to send the correct mimetype? That seems much more straightforward.

In practice it's happening when downloading a spreadsheet on a healthcare-related government regulation website, which we don't even have hope of ever reaching someone who may want to fix this (sadly - we would love to have a good talk with them), we have however implemented a workaround by downgrading Firefox and will probably switch out the selenium driver in case this issue is not deemed worthy of a fix/change.

That seems much more straightforward.

While that is indeed true, one could also argue that it's just as straightforward for the user-agent to accept the file name proposed by the server, and process in-app validations based on Content-Type (like the PDF reader), and for the original issue proposed by the scenario, have the web server (reddit) presenting the wrong extension send the correct content-disposition. I do understand that RFC 6266 states that the filename can be ignored by user-agents, however doing so would technically make the example from section 5 meaningless and one can also argue that both servers are wrong for sending "invalid" headers.

As comment 5 pointed out, it's much more common for the filename to be wrong than the content type...

I understand that completely! However by deriving the extension you are pretty much ignoring the file name that was recommended/sent by the server for a file and assuming (whatever the user-agent has implemented) interpretation is correct (maybe mime-checking would be more appropriate than guessing based on A or B, however that requires downloading a bit of the file first - which is understandably not feasible).

I don't know what the course of action should be here though, I raised the issue as I noticed a problem - something that was working before is now broken and that same thing is not broken on other browsers. Looking at the linked issue that caused this one, seems like this change has already caused several other issues similar to mine, specially on situations where the same content-type can have multiple extensions (like tiff or mp4 or zip or jpeg), in which case the fix was pretty much rolled back/added an exception for those extensions(?).

Also, sorry for the wall of text! I just wanted to have the entire point come across in one go!

Flags: needinfo?(git)

Tom S. (please needinfo tschuster)

Comment 8

•

5 years ago

For your use case if you are actually controlling the client you can set browser.download.sanitize_non_media_extensions to false in about:config.

:Gijs (he/him)

Comment 9

•

5 years ago

So, some history here:

In bug 1440677 (and its many dupes) there are LOTS (and I mean lots) of examples of relatively high-profile pieces of server software (amazon's S3 stuff, heroku, large hardware manufacturers) that mis-encode filenames in Content-Disposition headers. The result is that a download with a correct mimetype but badly encoded content-disposition header would end up either without an extension or with the wrong one. On Windows and macOS this also means that ordinary users struggle to open the files (you have to put the extension back). We "fixed" our parsing of the header to do better on badly encoded filenames, but also, we figured, it'd be a good idea not to have to put users through this -- if we know it's a PDF/zipfile/whatever, why not just put the correct extension on it? (cf. https://bugzilla.mozilla.org/show_bug.cgi?id=1652520#c9 )

Combined with the issues around media content (both news media and sites like imgur) that likes to use https://example.com/foo.jpeg for images in their stories, and then set up example.com to return image/webp content if the browser's Accept headers indicate it can cope with that - to save on bandwidth costs. Those servers don't usually send any content-disposition header. We fixed up extensions based on the content-type in that case (ie when the user decides to save any of those images). We initially did that in the frontend image saving code, but ran into too many edgecases, so at some point some of us decided we should fix this more conclusively, which is what happened in bug 1667787 - there we corrected extensions everywhere (!).

I realized a little belatedly that this was probably regression-prone, and in bug 1652520 we limited the fixup to cases where the server sends image/*, audio/* and video/* mimetypes, as well as a fixed list of other content types, and the unlimited-correction version of that code never shipped in release (ie all releases that have the fix for bug 1667787 have the code from bug 1652520, too).

Now, to answer some of your questions in not necessarily the same order:

I don't know what the course of action should be here though, I raised the issue as I noticed a problem - something that was working before is now broken and that same thing is not broken on other browsers. Looking at the linked issue that caused this one, seems like this change has already caused several other issues similar to mine, specially on situations where the same content-type can have multiple extensions (like tiff or mp4 or zip or jpeg), in which case the fix was pretty much rolled back/added an exception for those extensions(?).

It was rolled back for zip and json (ie we no longer change extensions for those mimetypes), not for images (not sure where you saw the tiff case, can you link it?). As noted above, the jpeg vs webp case was in fact the original problem here, and was fixed by the changes that caused your regression (but bugzilla's dependency tracking is... complicated... so it may be hard to tell).

We already limit the extension correction to a set of mimetypes where we figured that we could/should be able to be confident that the mimetype is correct and authoritative (ie we don't ignore all file extensions). We thought that'd be the case for PDF, but apparently not in the case of your server...

AIUI Chrome also adds file extensions (for more mimetypes than we do) when absent, but it doesn't always replace them if they're "wrong". I don't know exactly what logic it uses for that.

All of this is further complicated by the fact that Firefox's file handling code is set up to decide what to do with downloads (save vs. open vs. ask what you want) based on the mimetype, not the filename (decisions from 20 years ago are fun sometimes...). If the filename was really more important we should convert all of that logic to use file extensions, which is obviously not a small project... If we keep it as it is but override the mimetype with the file extension, strange/bad things can happen (cf. bug 1568003).

(In reply to Fabricio from comment #7)

In practice it's happening when downloading a spreadsheet on a healthcare-related government regulation website, which we don't even have hope of ever reaching someone who may want to fix this (sadly - we would love to have a good talk with them),

:-(

we have however implemented a workaround by downgrading Firefox and will probably switch out the selenium driver in case this issue is not deemed worthy of a fix/change.

As a more positive workaround, if you upgrade to 85 (released this week, though updates may be throttled initially to ensure stability) you can set browser.download.sanitize_non_media_extensions to false in about:config / as an internal pref via selenium, and that should fix the issue without requiring a downgrade.

(as the pref name suggests, this would fix things for application/pdf content types, but not for image/audio/video mime types)

I'm still on the fence about flipping the default for that pref. It would fix your issue but reintroduce the potential for the other ones....

Marco, thoughts?

Flags: needinfo?(mak)

Marco Bonardo [:mak]

Comment 10

•

5 years ago

Your analysis is perfect.
The only alternative I could think of, is to actually check if our guess was correct by sniffing the first bytes of the file.

Flags: needinfo?(mak)

Fabricio

Reporter

Comment 11

•

5 years ago

Hey Gijs, thanks for the detailed look on how this all played out! Right now we are still using a downgraded version of Firefox since we had to lockdown changes to production as this break caused a lot of havoc (it took us quite a few days to figure out what was going wrong - the gov. website isn't the most stable piece of software), however, I've taken a bit of time to test your proposed workaround but it doesn't seem to work for me.

I've manually opened about:config and set the browser.download.sanitize_non_media_extensions to false, restarted the browser, confirmed the option was still false and went to my example website, however the issue persists (both on the xls->pdf as well as the pdf->jfif/mp4). This was done on firefox 85.0, maybe this switch was not merged into the stable branch?

Thanks!

Tom S. (please needinfo tschuster)

Comment 12

•

5 years ago

I've manually opened about:config and set the browser.download.sanitize_non_media_extensions to false, restarted the browser, confirmed the option was still false and went to my example website, however the issue persists (both on the xls->pdf as well as the pdf->jfif/mp4). This was done on firefox 85.0, maybe this switch was not merged into the stable branch?

Thanks for pointing this out. It seems like the switch indeed doesn't work. We will work on a fix soon. However it's very unlikely that we will be updating Firefox 85.0 for this. For a corporate deployment you should look into using Firefox ESR.

Tom S. (please needinfo tschuster)

Updated

•

5 years ago

Depends on: 1690051

:Gijs (he/him)

Comment 13

•

5 years ago

The browser.download.sanitize_non_media_extensions pref should work from Firefox 86 onwards (to be released next week). We've also removed a bunch more of the mimetypes for which we correct file extensions in Firefox 87 (but we've kept PDF for now, so this may not help with your original problem).

I think for now, we are unlikely to change the default for this pref or remove PDF, so I guess that makes this particular bug WONTFIX -- though I would still be interested if you could confirm that changing browser.download.sanitize_non_media_extensions on 86 (currently on beta if you don't want to wait for next week) works for you.

Status: UNCONFIRMED → RESOLVED

Closed: 5 years ago

Resolution: --- → WONTFIX

ekeek

Comment 14

•

5 years ago

(In reply to :Gijs (he/him) from comment #13)

The browser.download.sanitize_non_media_extensions pref should work from Firefox 86 onwards (to be released next week). We've also removed a bunch more of the mimetypes for which we correct file extensions in Firefox 87 (but we've kept PDF for now, so this may not help with your original problem).

I think for now, we are unlikely to change the default for this pref or remove PDF, so I guess that makes this particular bug WONTFIX -- though I would still be interested if you could confirm that changing browser.download.sanitize_non_media_extensions on 86 (currently on beta if you don't want to wait for next week) works for you.

Will this fix consider content-disposition filename when there is not filename.pdf inside the url?
As an example, on firefox 85 this file is renamed as "document.pdf" instead of "Get Adobe Reader.pdf"
https://www.maricopa.gov/DocumentCenter/View/6802/Get-Adobe-Reader-PDF?bidId=

:Gijs (he/him)

Comment 15

•

5 years ago

•

Edited

(In reply to ekeek from comment #14)

Will this fix consider content-disposition filename when there is not filename.pdf inside the url?
As an example, on firefox 85 this file is renamed as "document.pdf" instead of "Get Adobe Reader.pdf"
https://www.maricopa.gov/DocumentCenter/View/6802/Get-Adobe-Reader-PDF?bidId=

If I right click this link and click "Save link as...", I get the correct filename ("Get Adobe Reader.pdf"). I suspect PDF.js (our PDF viewer) is passing its own filename to our filepicker code, when you click the "download" or "save" button inside the PDF viewer (where I can reproduce it being named "document.pdf"). That would need its own fix; please file a separate bug for that (it's not directly related to the one being discussed here).

Fabricio

Reporter

Comment 16

•

5 years ago

Hey Gijs, thanks for the heads-up.

I've just verified again with Firefox Stable 86.0 and and the switch still doesn't work for me, I've double checked the browser.download.sanitize_non_media_extensions option is set to false on about:config however when going to my test website and clicking on the XLSX link it still saves as PDF.

Issue also happens on ESR (although I didn't see any indication that I was running ESR even after installing it?).

Thanks for all the effort so far!

Tom S. (please needinfo tschuster)

Comment 17

•

5 years ago

Fabricio unfortunately I still can't reproduce this problem. After setting the pref https://notfab.net/firefox.xlsx downloads as firefox.xlsx instead of firefox.pdf. Can you maybe try with a new profile?

Flags: needinfo?(git)

Tom S. (please needinfo tschuster)

Comment 18

•

4 years ago

I am assuming no answer is forthcoming.

Flags: needinfo?(git)

Fabricio

Reporter

Comment 19

•

4 years ago

Hey Tom,

Sorry for not replying, I somehow missed the last email. I just tested again and I can confirm that with browser.download.sanitize_non_media_extensions set to false on 91.0.2, the file preview feature/save window "format" field shows PDF but the file correctly downloads as a XLSX when the save option is selected/clicked.

Thanks, and sorry again!

natew

Comment 20

•

4 years ago

Thanks for the detailed explanation of the reasoning behind this. We are running into problems with WMV files being downloaded as ASF files. I think this might warrant another exception in Firefox.

video/x-ms-asf is a "parent" mimetype that can describe audio/x-ms-wma and video/x-ms-wmv. However, it is common for mimetype detection libraries to detect a .wmv file as video/x-ms-asf (examples in python and PHP).

Would it make sense for Firefox to not overwrite the filename with .asf when it already includes a .wma or .wmv extension?

:Gijs (he/him)

Comment 21

•

4 years ago

(In reply to natew from comment #20)

Thanks for the detailed explanation of the reasoning behind this. We are running into problems with WMV files being downloaded as ASF files. I think this might warrant another exception in Firefox.

video/x-ms-asf is a "parent" mimetype that can describe audio/x-ms-wma and video/x-ms-wmv. However, it is common for mimetype detection libraries to detect a .wmv file as video/x-ms-asf (examples in python and PHP).

Would it make sense for Firefox to not overwrite the filename with .asf when it already includes a .wma or .wmv extension?

If this is a recent change (ie broke in Firefox 93), which I suspect, then I believe this is tracked in bug 1734606, caused by bug 1704115.

Flags: needinfo?(natew)

natew

Comment 22

•

4 years ago

Thanks for the quick reply :Gijs! Indeed I am on Firefox 94 so this is probably that bug.

Flags: needinfo?(natew)