Closed Bug 1523898 Opened 6 years ago Closed 6 years ago

NetEase cloud music (music.163.com) is not playable

Categories

(Web Compatibility :: Site Reports, defect, P1)

Product:
Web Compatibility
Component:
Site Reports
Type:
defect

Tracking

(firefox66+ fixed, firefox67+ fixed)

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox66 + fixed
firefox67 + fixed

People

(Reporter: xidorn, Assigned: miketaylr, NeedInfo)

References

Details

(Keywords: regression, site-compat, Whiteboard: [sitewait])

Steps to reproduce:

  1. you need a mainland Chinese IP to use NetEase cloud music
  2. open a music for example https://music.163.com/#/song?id=428350227
  3. click the "播放" blue button to play (the button is gray when outside mainlan China)

Expected result:
The music can play.

Actual result:
It cannot.

I use mozregression to track, and the pushlog is https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e4cbf560928e09c1d7003e5375a5601eabb79c61&tochange=5686d29392e8aecb57c0f75414bedebf54631447 so it's regressed by bug 1487416 which seems to be related.

I'll stay in China for about a week or so, so if there is anything I can help diagnosing, I'm happy to. I believe there are other Chinese community members willing to help as well.

Bryce, could you have a look at this issue?

Flags: needinfo?(bvandyk)

FWIW, this was initially reported in https://www.v2ex.com/t/531740 (Chinese).

Assignee: nobody → bvandyk
Status: NEW → ASSIGNED
Flags: needinfo?(bvandyk)
Priority: -- → P1

One person in the thread mentioned that this one is probably playable outside China: https://music.163.com/#/djradio?id=527648690

You can open this url and click the play button (small triangle inside circle) of any song in the list. It seems to have the same issue.

(In reply to Xidorn Quan [:xidorn] UTC+11 from comment #3)

One person in the thread mentioned that this one is probably playable outside China: https://music.163.com/#/djradio?id=527648690

You can open this url and click the play button (small triangle inside circle) of any song in the list. It seems to have the same issue.

Thanks! This should speed up my workflow, as I'd been having to tunnel and do some DNS remapping to get the original case to function in my location.

What I think is happening here:

  • From watching the network traffic: the site is serving MP3s, and I assume trying to use them with MSE . Firefox doesn't support this, so I imagine they're repacking them into fragmented MP4 on the fly in JS -- we've seen other sites use a similar strategy.
  • The site is serving MP4 track metadata using the forbidden track id 0, specifically in the tkhd and trex boxes. This is why bug 1487416 broke this, though we are since tolerant of using track id 0.
  • When the site serves us actual Moof data, the track fragment header uses track id 2. This doesn't match any of the metadata we received earlier (which used track id 0), so don't parse this info in Index's MoofParser, which is locked to a specific track. This is why Bug 1519617 didn't fix this case.

This used to work because the 0 track id from the metadata would trigger us (unintentionally) to create Index's MoofParser such that it would parse all tracks.

You can breakpoint here in current central (making sure that your stack is being called from Index.cpp) and compare the track ids from the trex and tfhd boxes involved to see that they never match. If you comment out these lines the site will work, though that is not a sane general purpose solution.

Chrome supports MP3 in mse, so I expect that's why this works there.

This is a nasty because it's relying on (IMO) an oversight we had in previous code. Ideally the site would fix their track ids, which would involve either using tack id 2 in their initial metadata or track id 0 in their trafs' tfhd boxes.

A fix in Firefox would be non-trivial. The only thing I can think of at this stage, and I'm not sure if it's a good idea, as well as it being a lot of work: we'd rework the MoofParser to be aware of all the tracks in a file. If we are given fragments that do not to correspond to a known track id from our trak headers (tkhd), and if we only have one track header in our metadata, we do a best effort to use our mismatched fragments.

Xidorn, I don't suppose you have any idea on if we can reach the site to see if they're able to change their handling?

:jya, I'd appreciate your thoughts and if you see another way for us to handle this?

Flags: needinfo?(jyavenard)

I've sent a feedback to NetEase via their official feedback channel. We'll see if that's going to work.

Note that, it's close to Chinese New Year now, so I wouldn't expect they are going to handle this any time soon. It's not very likely to get any response before 11 Feb, I suppose.

I've been hacking on the sites JS to see if I can get it to work in Firefox. It's obfuscated, and there's a lot, so I don't have a great handle on how it all works, but I've got far enough to have some insight into what's going on. The source is here, probably worth beautifying it for anyone else wanting to look at it.

There appears to be something going wrong between the way the c.trex + c.tkhd and c.traf are handling track ids -- note there are n. versions of these too, but they appear to be for some other path I'm not hitting, or are just obfuscation. If we change the id used in c.trex and c.tkhd to be 2, then the site works, or if we change c.traf to use id 0, it works too. So there's something wrong with how IDs are being passed around. I expect, something is calling c.generateInitSegment with a t.id == 0 and c.moof with an t.id == 2.

In case my methodology is helpful: this can be done by using something like Fiddler to intercept and replace the script with a locally modified copy.

Those are invalid files. Need to be fixed on their end.

Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Flags: needinfo?(jyavenard)
Keywords: site-compat
Resolution: --- → INVALID
Status: RESOLVED → REOPENED
Component: Audio/Video: Playback → Desktop
Product: Core → Tech Evangelism
Resolution: INVALID → ---

Bryce, is that a webcompat regression introduced by bug 1487416 in the end or is our current release also affected?

tracking-firefox67: --- → +
Flags: needinfo?(bvandyk)

(In reply to Pascal Chevrel:pascalc from comment #9)

Bryce, is that a webcompat regression introduced by bug 1487416 in the end or is our current release also affected?

We used to play them out of luck. They should have never played.
And it's not exactly a webcompat issue because such bad content will only be served to Firefox (as others have native MSE mp3 support)

(In reply to Jean-Yves Avenard [:jya] from comment #10)

(In reply to Pascal Chevrel:pascalc from comment #9)

Bryce, is that a webcompat regression introduced by bug 1487416 in the end or is our current release also affected?

We used to play them out of luck. They should have never played.
And it's not exactly a webcompat issue because such bad content will only be served to Firefox (as others have native MSE mp3 support)

So it worked before the patch and it no longer works after the patch but it works in our competitors browsers. Luck or not, that looks like a webcompat problem to me as some of our users are affected.

Xidorn, is that a popular player in China? Should our webcompat team be involved in contacting them? Thanks.

Flags: needinfo?(xidorn+moz)

(In reply to Pascal Chevrel:pascalc from comment #11)

(In reply to Jean-Yves Avenard [:jya] from comment #10)

(In reply to Pascal Chevrel:pascalc from comment #9)

Bryce, is that a webcompat regression introduced by bug 1487416 in the end or is our current release also affected?

We used to play them out of luck. They should have never played.
And it's not exactly a webcompat issue because such bad content will only be served to Firefox (as others have native MSE mp3 support)

So it worked before the patch and it no longer works after the patch but it works in our competitors browsers. Luck or not, that looks like a webcompat problem to me as some of our users are affected.

Yes.

Xidorn, is that a popular player in China? Should our webcompat team be involved in contacting them? Thanks.

Yes, it is a very popular player in China. For the second question, probably, if you have better channel...

Flags: needinfo?(xidorn+moz)

Thanks Xidorn!

NI Mike so as that he has that his team has this webcompat issue on his radar for 67.

Flags: needinfo?(miket)

[Tracking Requested - why for this release]: The fix that breaks the site landed in 66, requesting tracking for visibility there.

status-firefox66: --- → affected
tracking-firefox66: --- → ?
Flags: needinfo?(bvandyk)

Thanks. Will re-assign to myself so I get the emails about a tracked bug.

Xidorn, please let us know if the site gets in touch after Chinese New Years. If not, we can ask some Mozilla China folks to also attempt outreach.

Assignee: bvandyk → miket
Flags: needinfo?(miket)
Whiteboard: [sitewait]

Bingqing, do we have any contacts with NetEase?

Flags: needinfo?(bli)

(In reply to Mike Taylor [:miketaylr] from comment #16)

Bingqing, do we have any contacts with NetEase?

Sorry for the delay.
We are trying to reach NetEase, but last week was Chinese New Year, some people are still on PTO this week. Hope they can give an update next week.

Flags: needinfo?(bli)
Summary: NetEase cloud music is not playable → NetEase cloud music (music.163.com) is not playable
Product: Tech Evangelism → Web Compatibility

Chinese New Year period should have fully finished now, have we get any update from NetEase?

Flags: needinfo?(bli)

(In reply to Xidorn Quan [:xidorn] UTC+11 from comment #19)

Chinese New Year period should have fully finished now, have we get any update from NetEase?

We tried to ping NetEase this Wed, but still no updates. Keep trying, and will keep you in the loop.

Flags: needinfo?(bli)

hi Everyone:

thanks for attention：

I am the web developer of NetEase. I have fixed it by make the id of trex, tkhd and traf be same.

and now you can open music.163.com and play it successfully.

but you need to login to check MSE.

if you find bug or question any else, just reply me.

Thanks for fixing this and letting us know!

I tested the link in comment 0 and comment 3, both of them seem to work correctly now.

Closing this bug.

Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → FIXED

Thank you very much!

it doesnt work right on linux still,(In reply to ghj from comment #21)

hi Everyone:

thanks for attention：

I am the web developer of NetEase. I have fixed it by make the id of trex, tkhd and traf be same.

and now you can open music.163.com and play it successfully.

but you need to login to check MSE.

if you find bug or question any else, just reply me.

it can be reproduced still on linux　with firefox　66.0.1 debian testing.
https://music.163.com/#/playlist?id=988690134 as test link.
Attention: https://music.163.com/#/djradio?id=527648690 which mentioned in #c3 can not reproduce my issue

but if i change useragent to windows/chrome or linux/chrome,it works well.

and i noticed that on linux this website http://www.twoeggz.com/movie/5852987.html, if change UA to linux/chrome ,the sound get weird.have tested on windows ,nomatter what UA it is ,it works well.is it the same series issue or should i open new issue?

windows/chrome
Mozilla/5.0 (X11; Linux x86_64; ; x6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.47 Safari/537.36

linux/chrome
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.47 Safari/537.36

Yep. After a period of working time, it currently doesn't work right on Firefox 67.0b11, Arch Linux.

Somg songs play abnormally. e.g. https://music.163.com/#/song?id=428350227

Some songs play normally. e.g. https://music.163.com/#/djradio?id=527648690

They work well on Chromium 72.0.3626.122, Arch Linux, so I think maybe it's a problem of encode decision.

For the extra links given by FateRover, the Conan series on twoeggz.com are playing normally on my Firefox.

NI ghj for him to get notified.

Flags: needinfo?(guohjia)

(In reply to Outvi V from comment #25)

Yep. After a period of working time, it currently doesn't work right on Firefox 67.0b11, Arch Linux.

Somg songs play abnormally. e.g. https://music.163.com/#/song?id=428350227

Some songs play normally. e.g. https://music.163.com/#/djradio?id=527648690

They work well on Chromium 72.0.3626.122, Arch Linux, so I think maybe it's a problem of encode decision.

For the extra links given by FateRover, the Conan series on twoeggz.com are playing normally on my Firefox.

NI ghj for him to get notified.
just try set ua as windows/chrome
Mozilla/5.0 (X11; Linux x86_64; ; x6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.47 Safari/537.36
easy to reproduce on twoeggz.com ,it should be reprodued by opposite steps to 163.com,but the same problem

Hi, thanks for your feedback.

I can reproduce the bug partially in a fresh installed lubuntu and Debian, which instead of not playing, the music sounds like slower, or say it sounds like resample to a lower frequency.

All the audio files of the given links that cannot play are AAC, and from caniuse, it says:

Partial support in Firefox refers to only supporting AAC in an MP4 container and only when the operating system already has the codecs installed.

The AAC files are provided as M4A, which is MP4 wrapped AAC, so the file format should not be an issue, the problem maybe related to Firefox AAC codec.

However, as https://bugzilla.mozilla.org/show_bug.cgi?id=1523898#c24 says, if I change User-Agent to Chrome, the music sounds normally, the frequency and glitch problem is gone. Besides, I checked another site that also uses AAC (M4A) to play audio, and it works fine.

@ghj Is it possible that there are some hacks for Firefox specifically which makes Firefox cannot handle AAC correctly? As when User-Agent is Chrome, the requests in Network panel shows as media (so the file may load directly), but if it's Firefox, it shows as xhr. So maybe the bug is still related to MSE? And in a worst case, is it possible to fallback to MP3 for Linux Firefox users (thought in some Linux distributions like lubuntu, Firefox cannot support MP3)?

(In reply to ccloli from comment #27)

I can reproduce the bug partially in a fresh installed lubuntu and Debian, which instead of not playing, the music sounds like slower, or say it sounds like resample to a lower frequency.

Yes. That's also my case.

if I change User-Agent to Chrome, the music sounds normally, the frequency and glitch problem is gone.

I tried this, and it also works for me.

Thanks for everyone's feedback.There are two questions i want to confirm:

1、Its seems like the firefox on linux can reproduce this bug？

2、the play abnormally address is m801？

3、It may cause by Partial support in Firefox refers to only supporting AAC in an MP4 container?

Flags: needinfo?(guohjia)

(In reply to ghj from comment #29)

1、Its seems like the firefox on linux can reproduce this bug？

AFAIK, on a fresh installed Debian 9 and latest Firefox 66.0.3 (not ESR which included in Debain by default, you should go to https://www.mozilla.com/firefox to get the latest build).

2、the play abnormally address is m801？

Nope, as for me it's m701, so maybe it's not related to CDN.

3、It may cause by Partial support in Firefox refers to only supporting AAC in an MP4 container?

I don't think so, if you open the music link directly, it plays without any problem.

I prefer it is some decode bug of firefox ,for the Conan play abnormally if change ua to windows/chrome,reopen?

Flags: needinfo?(lhenry)

13:08.54 INFO: Last good revision: 9d343e5ad00b39fdd5d2d925393e1b985435ed6d
13:08.54 INFO: First bad revision: 3ff9e491c530a952c358362c11fdc2e0faaa66d8
13:08.54 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9d343e5ad00b39fdd5d2d925393e1b985435ed6d&tochange=3ff9e491c530a952c358362c11fdc2e0faaa66d8

for this song: https://music.163.com/#/song/1301409077/

cdn node is m801. With good versions it plays but has noise, With bad versions there are lots of repeating error messages in console:

RangeError: isContained, range.length is 0

Weird is that it plays correctly (without noise) once with my daily use nightly. But it didn't reproduce.

I'm on Arch Linux as always.

forgot to say the bisect range: mozregression --good 2019-02-22 --bad 2019-04-23

(In reply to lilydjwg from comment #32)

13:08.54 INFO: Last good revision: 9d343e5ad00b39fdd5d2d925393e1b985435ed6d
13:08.54 INFO: First bad revision: 3ff9e491c530a952c358362c11fdc2e0faaa66d8
13:08.54 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9d343e5ad00b39fdd5d2d925393e1b985435ed6d&tochange=3ff9e491c530a952c358362c11fdc2e0faaa66d8

I can see when requesting music on Firefox, the XHR response type is set to moz-chunked-arraybuffer in core_[hash].js call stack.

But it seems that Bug 1120171 is supposed to be landed on Firefox 68, so it shouldn't touch Firefox 66, which behavior is sounds like lower frequency and glitch, so we're not talking about the same bug.

With good versions it plays but has noise, With bad versions there are lots of repeating error messages in console:

RangeError: isContained, range.length is 0

However when I tested Firefox Nightly 68.0b1 on Windows 10, I can reproduce your bug with plenty of RangeError: isContaineed, range.length is 0 in console, and the music cannot play. Should cc @ghj to find a replacement of moz-chunked-arraybuffer.

Flags: needinfo?(guohjia)

BTW, by switching User-Agent to Chrome can resolve the bug on Firefox Nightly 86.0a1.

So is there something that detect the User-Agent and use different ways to load and play music? As it works fine when User-Agent is other browsers (Chrome, Safari, IE, Edge) on Firefox, is it possible to remove the specific code that only affected to Firefox?

Flags: needinfo?(lhenry)

ghj, ccloli, Outvi: I created https://bugzilla.mozilla.org/show_bug.cgi?id=1549353 concerning the RangeError: isContained, range.length is 0 issue, since it doesn't seem to be the same problem of this track id bug.

You need to log in before you can comment on or make changes to this bug.