1772988 - File .mp2 is converted to .mpeg while downloading

Status: VERIFIED FIXED

(Firefox :: File Handling, defect, P2)

Description

[yulia.furta]

Attachment: MP2 - MPEG-1 Audio.mp2

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36

Steps to reproduce:

Download file (I did this way):
const blob = new Blob([data], { type: 'audio/mpeg' });
const uri = URL.createObjectURL(blob);
var link = document.createElement("a");
link.download = documentName;
link.href = uri;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
URL.revokeObjectURL(objectURL);

Actual results:

File downloaded with .mpeg extension

Workaround:
Provide 'application/octet-stream' type to blob object
const blob = new Blob([data], { type: 'application/octet-stream' });

Expected results:

The file is downloaded in original format .mp2

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::File Handling' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Hani Yacoub, Desktop QA]

I'm having difficulties trying to reproduce this bug and bug 1772989. Could you please provide more detailed steps on how to reproduce it?
Where exactly should I run the script you provided above?
Thanks.

Comment 3

[:Gijs (he/him)]

(And what version of Firefox is this, on what OS?)

Comment 4

[Monica Chiorean]

removed comment.

Comment 5

[:Gijs (he/him)]

Without a response from the reporter we are unlikely to be able to do anything here.

Comment 6

[Monica Chiorean]

I was able to partially reproduce the issue on 102.0b1 (20220530132252). Using a new profile and right click -> Save link as on the attached .mp2 from bug it gets saved as '.mp3' but not '.mpeg' like mentioned by reporter.

Same issue is reproducing on 103.0b8(20220712185923) and Nightly 104.0a1(20220713215226) on Win10.

Comment 7

[Adrian Florinescu [:aflorinescu]]

(In reply to Monica Chiorean from comment #6)

I was able to partially reproduce the issue on 102.0b1 (20220530132252). Using a new profile and right click -> Save link as on the attached .mp2 from bug it gets saved as '.mp3' but not '.mpeg' like mentioned by reporter.

Same issue is reproducing on 103.0b8(20220712185923) and Nightly 104.0a1(20220713215226) on Win10.

I would asume that in order to fully reproduce the reporter's behavior and maybe confirm/infirm on different OS'es (currently I see it only reproduces on Windows 10 post bug 1746052 and subsequent fixes do not improve behavior), we'd also need handlers set in a certain way + OS side filetype defaults. For this particular issue, to reproduce the W10 behavior you can also use https://codepen.io/adiiflorinescu/pen/RwMGERx:
upload an mp2 file (get the one originally posted in the bug) and then download it, on W10 it will save it as mp3.
I reckon the above should be enough to reopen this issue.

Comment 8

[Neil Deakin]

On Mac, the mime service function GetFromTypeAndExtension() returns 'mp3', 'mp2' and 'mpga' as valid extensions associated with audio/mpeg when we don't specifiy an extension to that function. But on Windows only 'mp3' is specified as being valid. I confirmed that mp2 is in fact associated with audio/mpeg in the registry however, and if we explicitly ask if 'mp2' is valid', GetFromTypeAndExtension() returns both 'mp2' and 'mp3'. (.mpa also works)

So the behaviour of GetFromTypeAndExtension() on Windows is different here.

We could either fix that or special case mpeg files, or possibly all image/media types as we currently do for binary and ogg files.

Gijs, does this sound like a bug to investigate?

Comment 9

[:Gijs (he/him)]

(In reply to Neil Deakin from comment #8)

On Mac, the mime service function GetFromTypeAndExtension() returns 'mp3', 'mp2' and 'mpga' as valid extensions associated with audio/mpeg when we don't specifiy an extension to that function. But on Windows only 'mp3' is specified as being valid. I confirmed that mp2 is in fact associated with audio/mpeg in the registry however, and if we explicitly ask if 'mp2' is valid', GetFromTypeAndExtension() returns both 'mp2' and 'mp3'. (.mpa also works)

So the behaviour of GetFromTypeAndExtension() on Windows is different here.

We could either fix that or special case mpeg files, or possibly all image/media types as we currently do for binary and ogg files.

Gijs, does this sound like a bug to investigate?

I'm not sure I understand - are you saying that on Windows, we sometimes claim mp2 is not valid for audio/mpeg, and sometimes that it is? (cf:

But on Windows only 'mp3' is specified as being valid.

vs

if we explicitly ask if 'mp2' is valid', GetFromTypeAndExtension() returns both 'mp2' and 'mp3'. (.mpa also works)

)

I don't really understand how that's possible, but that definitely sounds like a bug, yes.

Comment 10

[Neil Deakin]

I tested the behaviour of GetFromTypeAndExtension using a variety of inputs:

Windows:

  GetFromTypeAndExtension("audio/mpeg", "mp3")                   - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "mp2")                   - returns a mime info with both '.mp2' and '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "")                      - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "html")                  - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "unknown")               - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("application/x-javascript", "")        - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-javascript", "js")      - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-javascript", "html")    - returns a mime info with both '.html' and '.js'
  GetFromTypeAndExtension("application/x-javascript", "exe")     - returns a mime info with both '.exe' and '.js'
  GetFromTypeAndExtension("application/x-unknowntype", "")       - returns a mime info with no extensions
  GetFromTypeAndExtension("application/x-unknowntype", "html")   - returns a mime info with only '.html'
  GetFromTypeAndExtension("application/x-unknowntype", "other")  - returns a mime info with only '.other'
  GetFromTypeAndExtension("application/x-unknowntype", "js")     - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-unknowntype", "exe")    - returns a mime info with '.exe, '.com', and '.bin'

Mac:

  GetFromTypeAndExtension("audio/mpeg", "mp3")                   - returns a mime info with only '.mp3', '.mpga' and '.mp2'
  GetFromTypeAndExtension("audio/mpeg", "mp2")                   - returns a mime info with only '.mp2', '.mp3', '.mpga' (note: different order) 
  GetFromTypeAndExtension("audio/mpeg", "")                      - returns a mime info with only '.mp3', '.mpga' and '.mp2'
  GetFromTypeAndExtension("audio/mpeg", "html")                  - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "unknown")               - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("application/x-javascript", "js")      - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-javascript", "")        - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-javascript", "html")    - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-javascript", "exe")     - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-unknowntype", "")       - returns a mime info with no extensions
  GetFromTypeAndExtension("application/x-unknowntype", "html")   - returns a mime info with '.html', '.htm', '.shtml' and '.ehtml'
  GetFromTypeAndExtension("application/x-unknowntype", "other")  - returns a mime info with no extensions
  GetFromTypeAndExtension("application/x-unknowntype", "js")     - returns a mime info with only '.js'
  GetFromTypeAndExtension("application/x-unknowntype", "exe")    - returns a mime info with '.exe' and '.com'

Comment 11

[:Gijs (he/him)]

(In reply to Neil Deakin from comment #10)

I tested the behaviour of GetFromTypeAndExtension using a variety of inputs:

Windows:

  GetFromTypeAndExtension("audio/mpeg", "mp3")                   - returns a mime info with only '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "mp2")                   - returns a mime info with both '.mp2' and '.mp3'
  GetFromTypeAndExtension("audio/mpeg", "")                      - returns a mime info with only '.mp3'

Yeah, this seems like a bug. So we have several options here:

1. hardcode more things into the extra mime entries array so these options start showing up (probably wise to do an audit of the various interrelated bugs about formats and extensions currently on file...).
2. try and work out why mp2 does get detected as valid but doesn't get included by GetFromTypeAndExtension if not passed an extension. But not sure if this is fixable...

Neil, do you have time to investigate further?

Comment 12

[Neil Deakin]

As far as as I can tell, the windows implementation in nsOSHelperAppService only ever checks and returns one extension. If the extension is valid for the given content type, it returns a mime info with that extension. If the extension is not valid, it returns the default extension or nothing. In the cases where additional extensions are included, this is done outside the platform-specific code by consulting the table in extraMimeEntries.

The table in extraMimeEntries adds any extensions that are valid for the given content type. If the content type doesn't match anything in extraMimeEntries, but the extension does, it also adds the other extensions from the same entry (so sample.exe will allow exe, com, and bin) even though that content type isn't specified.

The mac and unix implementations can return multiple extensions for one content type. The extraMimeEntries table might add some others as well.

Comment 13

[:Gijs (he/him)]

On Windows we read the extension from the mimetype using the windows registry, which only lists 1 extension for each mimetype: https://searchfox.org/mozilla-central/rev/c0bed29d643393af6ebe77aa31455f283f169202/uriloader/exthandler/win/nsOSHelperAppService.cpp#453-457 .

I don't think there's a way around this on a Windows level (or at least, my web searches are not finding me a way around this that provides more than 1 mimetype.

So although I think it's wrong that Windows only provides 1 extension, that seems to be the reality we live with.

Unless I'm wrong about that (Molly, can you confirm?) I think to address this and bug 1782366 and friends, we should compile a list of valid extensions for the various media mimetypes and augment extraMimeEntries with that information. I think that should fix these bugs. Neil/Molly/Al, does that sound workable? Or am I missing a better idea?

Comment 14

[:Gijs (he/him)]

The other thing we could try is always leaving the file extension alone for audio/ and video/ mimetypes - but we've been burned by this for image/ ones because of server transcoding (ie the server URL would be example.com/foo.jpg but we'd be served a webp image because the Accept header indicated we supported that...)

Comment 15

[Molly Howell (she/her) (no longer active)]

(In reply to :Gijs (he/him) from comment #13)

So although I think it's wrong that Windows only provides 1 extension, that seems to be the reality we live with.

Well. It depends on how badly you want to avoid living with it.

You're right that HKCR\MIME\Database only maps MIME types to a single extension (multiple types are allowed to resolve to the same extension though), and as far as I can tell that's the only direct MIME type to extension mapping available at the OS level.

But, HKCR keys for file extensions can also have a Content Type value for that extension. So, if we wanted to collect all the extensions for one content type, we could loop through every single file extension in the registry and find all the ones that list the type we're looking for! Couldn't be simpler!

... so yeah, I think we should probably have our own list instead of that.

Comment 16

[Neil Deakin]

The following is a list of the additional type/extension mappings that the OS provides that aren't in the table that we might consider adding:

Mac:
application/octet-stream: dms, lha, lzh, class, so, dll, iso, fla
image/jpeg: jpe
image/svg+xml: svgz
text/plain: asc
text/html: jhtml
application/xhtml+xml: xhtm
audio/mpeg: mpga, mp2
audio/mp4: mp4
audio/x-midi: midi

Ubuntu:
application/postscript: epsi, epsf, eps2, eps3
application/msword: dot
application/vnd.ms-excel: xlb, xlt
text/plain: pot, brf, srt
text/calendar: icz
application/ogg: ogx
audio/ogg: ogg, opus, spx
audio/mpeg: mpega, m4a

Comment 17

[Neil Deakin]

Attachment: Bug 1772988, add some additional extensions for some media types as Windows doesn't provide some of them, causing valid extensions to be replaced when saving files, r=mhowell

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/517b6c2b1d57
add some additional extensions for some media types as Windows doesn't provide some of them, causing valid extensions to be replaced when saving files, r=mhowell

Comment 19

[Atila Butkovits]

Backed out for causing failures at test_filename_sanitize.js .

backout link: https://hg.mozilla.org/integration/autoland/rev/982d71c1173213d406f178b805665846b118077c

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=517b6c2b1d57342d093798589e920fe381cc10d9

Failure log: https://treeherder.mozilla.org/logviewer?job_id=388941310&repo=autoland&lineNumber=3304

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/92f305403fda
add some additional extensions for some media types as Windows doesn't provide some of them, causing valid extensions to be replaced when saving files, r=mhowell

Comment 21

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/92f305403fda

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

Since nightly and release are affected, beta will likely be affected too.
For more information, please visit auto_nag documentation.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

Seems like something that'd be nice to get uplifted to Beta and ESR also. Please nominate if you agree. It grafts cleanly.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:enndeakin, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox105 to wontfix.

For more information, please visit auto_nag documentation.

Comment 25

[Neil Deakin]

Comment on attachment 9292266 [details]
Bug 1772988, add some additional extensions for some media types as Windows doesn't provide some of them, causing valid extensions to be replaced when saving files, r=mhowell

Beta/Release Uplift Approval Request

• User impact if declined: The file extension of some media files will get set incorrectly when downloaded on Windows
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Low risk as it just adds some additional file extension mappings for some video/audio types.
• String changes made/needed: None
• Is Android affected?: No

Comment 26

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9292266 [details]
Bug 1772988, add some additional extensions for some media types as Windows doesn't provide some of them, causing valid extensions to be replaced when saving files, r=mhowell

Approved for 105.0b8 and 102.3esr.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/cf0fa691e3ef

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr102/rev/e1094621add1

Comment 29

[Adrian Florinescu [:aflorinescu]]

Solely from the point of verifying that now the mp2 file type is being preserved on W10 while downloading it, this is verified on Windows 10 with Nightly 106.0a1 (2022-09-07), 105.0b8 and Esr 102.3.0.

Neil, could you please compile a list of additional extensions that we ought to be also verifying as part of this fix/verification?

Comment 30

[Neil Deakin]

The intent of this bug was to allow the following additional mpeg audio/video related extensions to be properly handled on Windows:
mpega, mp2, mp4, m4b

mp3 and m4a should already always work, and the above may already work depending on how the system is configured, but now all six should always work.

Comment 31

[Cristian Baica [:cbaica], Release Desktop QA]

Verified the download for mp2, mp4, m4b extensions using the latest nightly (2022-09-09), beta 9 and the latest ESR 102.3.0 on Windows 10. Extensions are correctly downloaded and named.