1465458 - Inconsistent MIME type handling across platforms, failing to open downloaded file on Windows

Status: VERIFIED FIXED

(Firefox :: File Handling, defect, P1)

Description

[grmat]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180518073241

Steps to reproduce:

0. Start Linux
1. Visit http://entropymine.com/jason/testbed/mime/ (or any web site that offers downloads for different content-types and file extensions; or set one up)
2. From the lower examples, where the column "what should happen" reads "offer to Save to disk" or "Save to disk or open application", select one file without a file extension, e.g. http://entropymine.com/jason/testbed/mime/foo/file (x-foo/x-bar; "file")
3. Select a program to open the file with
4. repeat on Windows


Actual results:

after step 3, Firefox on Linux succeeds to open the file with the selected program. On Windows, the exception "fileExtension is null" (at DownloadIntegration.jsm:602) is printed to the Browser Console. The function there checks whether the file is an executable and the user should be warned, checking the file extension, which is null.

As a result, Firefox on Windows will download the file to the temporary folder, but not open it with the selected program.

I was able to workaround this issue by manually editing the "handlers.json" in the profile folder and add an "extensions" array containing a file extension string to the specific mimeType object, e.g. like this:

"x-foo/x-bar": {
    "action": 2,
    "handlers": [
        {
            "name": "NOTEPAD.EXE",
            "path": "C:\\Windows\\system32\\NOTEPAD.EXE"
        }
    ],
    "extensions": [
        "txt"
    ],
    "ask": true
}

This way, Firefox will append ".txt" to the filename and not run into the fileExtension is null exception.


Expected results:

Consistent behaviour across the platforms. If the content-type is given, FF should not rely on the file extension under Windows. Also, the user is not notified about the error if they don't have the browser console opened.

Comment 1

[grmat]

To add to the confusion, another issue:
The above mentioned handlers.json seems to be only updated for a mime-type at first occurrence. E.g. if a file "foo.txt" with type "x-foo/x-bar" is downloaded in a fresh profile, firefox will add the "txt" extension to the json object. If the file is downloaded *without* the extension in a fresh profile, and later a file with the same mime-type but *with* a file extension, Firefox won't add the extension to the json. IMHO that means that it is cyrrently possible to "break" a user profile for each mime type.

Comment 2

[Daniel Bodea [:danibodea]]

I have reproduced the steps given successfully. The presumed issue is that the downloaded file is not opened if the user chooses to open the "x-foo/x-bar" type file with a specific program and/or the issue is that the user user is not warned that the file does not have an extension, so it won't be opened. 
Furthermore, the file can successfully be opened (with text readers, in this specific case) afterwards from the destination is was downloaded to.

I will set this issue's component on File Handling and will let the developer decide whether this is an issue or not.

grmat, please log the second issue in a separate bug. Only one issue can be addressed in one bug.

Thank you for your contribution!

Comment 4

[:Paolo Amadini]

Attachment: Bug 1465458 - Fix launching downloads without a file extension on Windows.

Review commit: https://reviewboard.mozilla.org/r/253122/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/253122/

Comment 5

[Marco Bonardo [:mak]]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

https://reviewboard.mozilla.org/r/253122/#review260366

how hard is to make a test for this?

Comment 6

[:Paolo Amadini]

Not trivial, but it can be done with mock objects. I'll file a bug.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8030bbd5967f
Fix launching downloads without a file extension on Windows. r=mak

Comment 8

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/8030bbd5967f

Comment 9

[Daniel Bodea [:danibodea]]

This issue is fixed in Nightly v63.0a1 (2018-06-29). Tested on Windows 10 x64.

Comment 10

[:Paolo Amadini]

Thanks for the quick verification!

Comment 11

[:Paolo Amadini]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:

This is a simple code correctness fix for an issue that was also reported a while ago in bug 1391608. A workaround exists by opening the folder that contains the document and launching it from there, which I guess is the reason why we had this reported only twice in a year.

ESR 52.9 is also affected since bug 1369771 was uplifted recently to support bug 1468217. I recommend taking this code correctness fix as a ride-along for a possible dot release.

User impact if declined:
Unable to open documents without a file extension on Windows.

Fix Landed on Version:
Nightly

Risk to taking this patch (and alternatives if risky): 
Low risk, it just restores the correct behavior.

String or UUID changes made by this patch: 
None

Comment 12

[Liz Henry (:lizzard) (relman/hg->git project)]

Regression since 55 with a pretty easy workaround, but, I can see the point of uplift for 62, and we're still early in 62 beta.

Comment 13

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

Verified in nightly, let's take this for beta 5.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Might be nice to add an automated test here too if possible.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

Approved as a ride-along for 61.0.1 too.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/cd20a5a8658b

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/6b6ded8230f1

Comment 18

[Anca Soncutean, Desktop QA]

Platform: Windows 10 x64
- I’ve reproduced this issue on Firefox 62.0a1 (2018-05-30.
- I’ve successfully managed to open a file without extension on Firefox 62.0b5 (20180702164905) and Firefox 61.0.1(20180704003137).

Comment 19

[:Paolo Amadini]

This is now verified on Release, can we land this as a potential ESR 52 ride-along too? It fixes a bug that is new in 52.9, not present in the previous 52.8 version. Even if 52 is supported for only two more months, it would be good to have this landed, just in case we make a dot release in the meantime.

Comment 20

[Julien Cristau [:jcristau]]

We can take this if and when we prepare a dot release.

Comment 21

[:Paolo Amadini]

I'm asking because I get release tracking alert e-mails, but if we don't plan to take any action here, I'll just ignore them.

Comment 22

[Julien Cristau [:jcristau]]

This should silence the alerts.

Comment 23

[:Paolo Amadini]

Cool, thanks! :-)

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

Approved for ESR 60.2.

Comment 25

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/bb310ef154f7

Comment 26

[Daniel Bodea [:danibodea]]

This issue is still reproducible in ESR60 (v60.1.0esr) while the change is in the build. The error "fileExtension is null" is displayed in the Browser Console. 
@Paolo Amadini, do you know why this happens?

Comment 27

[Julien Cristau [:jcristau]]

Which exact build are you testing?  The 60.1.0esr release predates that change.

Comment 28

[Daniel Bodea [:danibodea]]

I took the change set from comment 25 (bb310ef154f7), installed the latest ESR60 build (Build ID: 20180621121604) and I made sure that the fix change set is inside the browser's pushlog, and it seems like it is:
"user: [e-mail]
push date: Sun Jul 15 16:47:34 2018 +0000	
bb310ef154f7	Paolo Amadini — Bug 1465458 - Fix launching downloads without a file extension on Windows. r=mak, a=RyanVM"

Then I tried the STR from comment 0 on Windows 10:
1. Start broser (v60.1.0esr).
2. Open "http://entropymine.com/jason/testbed/mime/".
3. pen Browser Console (CTRL+SHIFT+J);
3. Click on the "file" links that coresponds to the "x-foo/x-bar" (without extension).
4. Choose the "Open" option, not the "Save" option.
5. Notice that the pop-up to choose a program to open with will not appear AND an error will be displayed in the Browser Console:
"fileExtension is null  DownloadIntegration.jsm:602
	launchDownload resource://gre/modules/DownloadIntegration.jsm:602:1
	InterpretGeneratorResume self-hosted:1257:8
	next self-hosted:1212:9
	launch resource://gre/modules/DownloadCore.jsm:677:12
	_succeed resource://gre/modules/DownloadCore.jsm:545:7
	InterpretGeneratorResume self-hosted:1257:8
	next self-hosted:1212:9
"

This issue does not occur in the case of the 3 main version of Firefox (Release, Beta, Nightly).
Is it the wrong build?

Comment 29

[Julien Cristau [:jcristau]]

A build from June 21 is definitely the wrong one, yes.

Comment 30

[Daniel Bodea [:danibodea]]

I will wait for the next ESR60 build to verify the fix.

Thank you. 

P.S. I will leave the NI? request on my name so I'll be notified.

Comment 31

[grmat]

:danibodea you pinged me, but I'm not sure what you need from me. Could you please clarify or is it settled already?

Comment 32

[Daniel Bodea [:danibodea]]

There was a confusion related to which ESR60 build contains the fix because I somehow thought that the lase one released had it. However, it appears I'll have to wait for the next ESR60 build release, since there isn't a working build in CI. Thank you.

I'll still keep the NI? request on my name until I verify the uplift.

Comment 33

[Ryan VanderMeulen [:RyanVM]]

There's tons of working CI builds. Just grab the most recent from Treeherder.
https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr60&filter-searchStr=build

Comment 34

[Daniel Bodea [:danibodea]]

I have used the following build:
https://queue.taskcluster.net/v1/task/B0NW1lxTQQSX1Wdm528pSg/runs/0/artifacts/public/build/target.zip
This issue is also fixed in ESR60 (v60.1.1esr). Uplift successful. 
Thank you!

Comment 35

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8987837 [details]
Bug 1465458 - Fix launching downloads without a file extension on Windows.

We're not going to be shipping a 52.9 dot release.