1749294 - File with file name ending in .mp4 has the .mp4 portion changed to .3gp when downloading

Status: NEW

(Core :: Audio/Video, defect)

Description

[Franpa_999]

Attachment: Snapchat-207719153_1_1.mp4

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0

Steps to reproduce:

1. Download this file https://media.discordapp.net/attachments/474510580191854593/929083614786895952/Snapchat-207719153_1_1.mp4 (A copy is also attached to the report)

Actual results:

the file extension is changed from .mp4 to .3gp

Expected results:

File extension should remain identical to what it was originally (.mp4).

Comment 1

[Franpa_999]

(In reply to Franpa_999 from comment #0)

Also affects Firefox 93.

Comment 2

[Masatoshi Kimura [:emk]]

Comment on attachment 9258316 [details]
Snapchat-207719153_1_1.mp4

The discord server sends video/quicktime for this file.

Comment 3

[:Gijs (he/him)]

(In reply to Franpa_999 from comment #0)

1. Download this file https://media.discordapp.net/attachments/474510580191854593/929083614786895952/Snapchat-207719153_1_1.mp4 (A copy is also attached to the report)

Clicking this link (or the bmo attachment) just shows the video in Firefox for me. Are you using right click + save link as? Or "save as" when the video is loaded? Or something else?

Comment 4

[:Gijs (he/him)]

(In reply to :Gijs (he/him) from comment #3)

(In reply to Franpa_999 from comment #0)

1. Download this file https://media.discordapp.net/attachments/474510580191854593/929083614786895952/Snapchat-207719153_1_1.mp4 (A copy is also attached to the report)

Clicking this link (or the bmo attachment) just shows the video in Firefox for me. Are you using right click + save link as? Or "save as" when the video is loaded? Or something else?

Bleh, seems that only happens for the attachment, but not the link. I don't know why.

Comment 5

[:Gijs (he/him)]

bug 1704115 regressed this. Before that bug, this just played in a tab.

Comment 6

[Masatoshi Kimura [:emk]]

(In reply to :Gijs (he/him) from comment #4)

Bleh, seems that only happens for the attachment, but not the link. I don't know why.

Maybe because of Content-Disposition: inline?

Comment 7

[Masatoshi Kimura [:emk]]

(In reply to :Gijs (he/him) from comment #5)

bug 1704115 regressed this. Before that bug, this just played in a tab.

But if I use "Save Link As", I got .mov extension. Moreover, the discord link played even after bug 1704115 for me.
If I use "Save Link As" for the discord link, I got .mov extension before bug 1704115 and got .3gpp extension after bug 1704115.

Comment 8

[Franpa_999]

(In reply to :Gijs (he/him) from comment #3)

(In reply to Franpa_999 from comment #0)

1. Download this file https://media.discordapp.net/attachments/474510580191854593/929083614786895952/Snapchat-207719153_1_1.mp4 (A copy is also attached to the report)

Clicking this link (or the bmo attachment) just shows the video in Firefox for me. Are you using right click + save link as? Or "save as" when the video is loaded? Or something else?

Attachment: Right Click > Save Link As > .mp4 changed to .3GP file extension
URL: Left Click > .mp4 changed to .3GP file extension

Comment 9

[Franpa_999]

Both those actions trigger this prompt https://imgur.com/wECn4mG

Comment 10

[Franpa_999]

Firefox is configured to "Always Ask" for the file, so left clicking the URL triggers the prompt but oddly left clicking the attachment loads the video for playback in a new tab which is why I need to Right Click and choose Save Link As for the attachment.

Comment 11

[Jon Bauman [:jbauman:]]

At least on macOS, I can't reproduce this behavior either on 95 or Nightly (currently 97.0a1). However, I do see somewhat different behavior between the two, neither changes the file extension from .mp4.

Franpa_999: do you still see this change to .3gp on nightly?

Comment 12

[:Gijs (he/him)]

(In reply to Jon Bauman [:jbauman:] from comment #11)

At least on macOS, I can't reproduce this behavior either on 95 or Nightly (currently 97.0a1). However, I do see somewhat different behavior between the two, neither changes the file extension from .mp4.

Franpa_999: do you still see this change to .3gp on nightly?

What behaviour you see is likely to depend on what file extension is associated (by the OS and/or Firefox) with the server-provided and/or sniffed mimetype, so can be OS-dependent. I reproduced on nightly on Win10, so this hasn't been fixed.

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1704115

Comment 14

[Jon Bauman [:jbauman:]]

Attachment: Screen Shot 2022-01-12 at 10.29.26 AM.png

(In reply to :Gijs (he/him) from comment #12)

What behaviour you see is likely to depend on what file extension is associated (by the OS and/or Firefox) with the server-provided and/or sniffed mimetype, so can be OS-dependent. I reproduced on nightly on Win10, so this hasn't been fixed.

Ah, that's helpful. I do see this indication of the file type when saving, but I see that same behavior on 95.

I don't have a Windows install handy; Paul, do you understand why this behavior changed with your revision for bug 1704115?

Comment 15

[Paul Adenot (:padenot)]

Sniffing now always happens, for media files. For some reason Firefox decides to change the extension of a file when downloading it, so media files are now affected.

Comment 16

[:Gijs (he/him)]

(In reply to Jon Bauman [:jbauman:] from comment #14)

Created attachment 9258714 [details]
Screen Shot 2022-01-12 at 10.29.26 AM.png

(In reply to :Gijs (he/him) from comment #12)

What behaviour you see is likely to depend on what file extension is associated (by the OS and/or Firefox) with the server-provided and/or sniffed mimetype, so can be OS-dependent. I reproduced on nightly on Win10, so this hasn't been fixed.

Ah, that's helpful. I do see this indication of the file type when saving, but I see that same behavior on 95.

To be clear, this regressed with 93...

I don't have a Windows install handy; Paul, do you understand why this behavior changed with your revision for bug 1704115?

(In reply to Paul Adenot (:padenot) from comment #15)

Sniffing now always happens, for media files. For some reason Firefox decides to change the extension of a file when downloading it, so media files are now affected.

So first of all, there is no Content-Disposition header here so there isn't really a strong indication of file extension. For things like download.php?fileid={uuid} and whatnot there would just be nothing to go on besides the mimetype. Here we'll take the file extension hint because there's no query parameters and because it's the best we have.

However, Firefox sets the extension differently from the URL, relying on the mimetype, for specific mimetype sets. This was originally implemented because servers like to do things like have foo.jpg files that are really webp/avif files - transcoded to webp/avif if the client advertises support. Saving those images then produced unreadable jpeg files, or webp files that were really jpeg files, or worse.
Similarly, there were cases of various other filetypes where we mis-determine a file extension from the URL. Relying on the content type header seemed like a better choice. See bug 1667787, bug 1602220, bug 1366645, and many others. In bug 1652520 that change was restricted to image/, video/ and audio/ and a handful of other mimetypes.

So there's a few things we can do here. What is still not clear to me is what mimetype we're sniffing, and what that means for the media file. If .mp4 is a reasonable extension for whatever the sniffed mimetype is, I believe adding it as the second/third extension in the lists at https://searchfox.org/mozilla-central/rev/435a77f1a1aaf1a78d30a2aaa81c6158a2f83dba/uriloader/exthandler/nsExternalHelperAppService.cpp#620-621 would help.
If that file extension isn't reasonable, then perhaps this bug should be wontfix, as we're now fixing an issue with the site-provided data? But it's not really clear to me. I also don't know what the downside of these 3gp file extensions is, or if they're widely supported or what - the entries in that list are 8 or 10 years old.

The other thing we could potentially do is stop overwriting file extensions for video/ or audio/ mimetypes.

The final thing we could do is not adjusting some of these mimetypes? If we're going to cause a download to happen, it's not clear to me why we're sniffing or why the sniffing is changing the mimetype.

Thoughts, Paul/Jon? I think at this point we've had enough reports of this issue (I expect bug 1734606 is the same issue?) that we should do something about this problem.

Comment 17

[Franpa_999]

(In reply to :Gijs (he/him) from comment #12)

(In reply to Jon Bauman [:jbauman:] from comment #11)

At least on macOS, I can't reproduce this behavior either on 95 or Nightly (currently 97.0a1). However, I do see somewhat different behavior between the two, neither changes the file extension from .mp4.

Franpa_999: do you still see this change to .3gp on nightly?

What behaviour you see is likely to depend on what file extension is associated (by the OS and/or Firefox) with the server-provided and/or sniffed mimetype, so can be OS-dependent. I reproduced on nightly on Win10, so this hasn't been fixed.

In Windows the file is associated with Media Player Classic BE.

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

The component has been changed since the backlog priority was decided, so we're resetting it.
For more information, please visit auto_nag documentation.

Comment 19

[Franpa_999]

I've also noticed similar behaviour with what seem to be PNG files (which are associated with Mozilla Firefox).

• The website URL for the file ends in a .png extension
• When downloading the file to your HDD it's saved as a .webp file
• Rename the file extension to .png and opened it in MSPAINT, now perform a Save or Save As, MSPAINT wants to save the file as .heic

Comment 20

[Paul Adenot (:padenot)]

But is it really a .png initially, if you look at the bytes? If you can attach one here, we can check.

Comment 21

[Jon Bauman [:jbauman:]]

(In reply to Franpa_999 from comment #19)

I've also noticed similar behaviour with what seem to be PNG files (which are associated with Mozilla Firefox).

• The website URL for the file ends in a .png extension
• When downloading the file to your HDD it's saved as a .webp file
• Rename the file extension to .png and opened it in MSPAINT, now perform a Save or Save As, MSPAINT wants to save the file as .heic

Is this a situation like this file mentioned in bug 1644950? The URL makes it appear to be a PNG, but the server returns content-type: image/webp and the file itself is indeed WebP.

Comment 22

[Jon Bauman [:jbauman:]]

(In reply to :Gijs (he/him) from comment #16)

Ah, that's helpful. I do see this indication of the file type when saving, but I see that same behavior on 95.
To be clear, this regressed with 93...

🤦 thanks; I see the different behavior now.

The original file (https://media.discordapp.net/attachments/474510580191854593/929083614786895952/Snapchat-207719153_1_1.mp4) is served with content-type: video/quicktime, which doesn't tell us much. In terms of behavior on macOS:

In Safari:

• If you select "Download Linked File" with that link, you get "Snapchat-207719153_1_1.mp4.mov"
• If you select "Download Linked File As…", the default name is "Snapchat-207719153_1_1.mp4"
• If you click the link, it plays the video inline, but if you select "Save As…" on the page, the name is "Snapchat-207719153_1_1.mp4.webarchive"

In Chrome:

• If you select "Save Link As…" with that link, you get "Snapchat-207719153_1_1.mov"
• If you click the link, it downloads the file (without prompt) as "Snapchat-207719153_1_1.mov"

In Firefox 92:

• If you click the link, it plays in browser, choosing "Save Page As…", you get "Snapchat-207719153_1_1.mp4"
• If you select "Save Link As…", you get "Snapchat-207719153_1_1.mp4", and in that dialog it says "Format: *.mp4"

In Firefox 93+

• Whether you click the link or select "Save Link As…", you get "Snapchat-207719153_1_1.mp4", and in that dialog it says "Format: 3GPP Video"

So there's a few things we can do here. What is still not clear to me is what mimetype we're sniffing, and what that means for the media file.

I'm pretty that the we're changing the file extension on Windows (and the "Format" part of the macOS save dialog), because we're sniffing the MIME type of this file as "video/3gpp" here. That's happening because the file itself lists its major_brand as 3gp5.

$ MP4Box -info Snapchat-207719153_1_1.mp4 
* Movie Info *
	Timescale 1000 - 2 tracks
	Computed Duration 00:00:10.240 - Indicated Duration 00:00:10.233
	Fragmented File: no
	File Brand 3gp5 - version 0
		Compatible brands: 3gp5 isom

If .mp4 is a reasonable extension for whatever the sniffed mimetype is, I believe adding it as the second/third extension in the lists at https://searchfox.org/mozilla-central/rev/435a77f1a1aaf1a78d30a2aaa81c6158a2f83dba/uriloader/exthandler/nsExternalHelperAppService.cpp#620-621 would help.

I'm not actually sure what that would do on Windows as long as we still have "3gpp,3gp" in there. And it's hard to say whether .mp4 is a reasonable extension or not since 3GPP is based on MP4, but has additional features. Typically this should be the role of the major_brand to indicate how the file should be consumed.

Since we recently added telemetry as part of bug 1725190 to see how often the different MP4 container types were each getting sniffed, I think I can say with some confidence that files that get sniffed as 3gpp are essentially non-existent. Based on the precedent in bug 1552194 where we changed from sniffing 3gp4-branded files from video/3gpp to video/mp4, we can do the same for 3gp5.

Paul, do you have any objection to that?

Comment 23

[:Gijs (he/him)]

(In reply to Jon Bauman [:jbauman:] from comment #22)

(In reply to :Gijs (he/him) from comment #16)

If .mp4 is a reasonable extension for whatever the sniffed mimetype is, I believe adding it as the second/third extension in the lists at https://searchfox.org/mozilla-central/rev/435a77f1a1aaf1a78d30a2aaa81c6158a2f83dba/uriloader/exthandler/nsExternalHelperAppService.cpp#620-621 would help.

I'm not actually sure what that would do on Windows as long as we still have "3gpp,3gp" in there. And it's hard to say whether .mp4 is a reasonable extension or not since 3GPP is based on MP4, but has additional features. Typically this should be the role of the major_brand to indicate how the file should be consumed.

FWIW, I have not tested it, but going pretty much off my memory, I believe the following is true:

• mimeinfo objects have a list of extensions that are associated with the mimetype
• we will adjust the file extension if we're overriding it based on the mimetype, iff the file extension is not in the list on the relevant mimeinfo object
• we will use the OS-specific mimeinfo objects + OS-specific logic in the external helper app service to determine the contents of the mimeinfo object, but we will supplement it with the data from the extraMimeEntries linked above, cf. https://searchfox.org/mozilla-central/rev/3de56eb5f266f523340e739ae1b53258e0a95dfe/uriloader/exthandler/nsExternalHelperAppService.cpp#2914-2938 .

As a result, I would expect that adding extensions to these lists would essentially "allow" those extensions to persist on the saved file. So we could add mp4 and/or mov to the list. To repeat, I have not tested this and I'm not 100% confident, but it should be easy to test...

For files without any extension, we would continue to pick the first item in the list (likely provided by the OS, or if none exists, then the first item in our own list).

What I don't know is whether that is "right" - effectively, I think based on the other comments here that Paul and you and me are agreed that if the file we download is a webp file, it should have a webp extension. I don't know enough about mp4/video formats, so I'm afraid that even after reading your comment, I'm not sure if mp4 (or mov like chrome, for that matter) would be "wrong" for this file.

Since we recently added telemetry as part of bug 1725190 to see how often the different MP4 container types were each getting sniffed, I think I can say with some confidence that files that get sniffed as 3gpp are essentially non-existent. Based on the precedent in bug 1552194 where we changed from sniffing 3gp4-branded files from video/3gpp to video/mp4, we can do the same for 3gp5.

That would also change the file extension to mp4, yes.

Comment 24

[Ardelean Oana, Desktop QA [:oardelean]]

This issue is still reproducible on Firefox 103.0b8(build ID: 20220712185923) and Nightly 104.0a1(build ID: 20220712093327) using the link from the Description. Reproducible on macOS 12, Windows 10 64-bits and Ubuntu 22.04.

Comment 25

[:Gijs (he/him)]

Guessing this wasn't supposed to be wontfixed for current nightly...