850520 - [music][metadata] Support non-Unicode text in ID3 tags

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Music, defect, P1)

Description

[sync-1]

+++ This bug was initially created as a clone of Bug #407751 +++
 
 Created an attachment (id=347828)
 add picture 1
 
 CONTACT INFO (Name,Phone number):
 Huxia 6552
 
 DEFECT DESCRIPTION:
 Show mess code in music.
 
 REPRODUCING PROCEDURES:
 Show mess code in music.
 
 EXPECTED BEHAVIOUR:
 No mess code.
 
 ASSOCIATE SPECIFICATION:
 
 TEST PLAN REFERENCE:
 
 TOOLS AND PLATFORMS USED:
 v109
 
 USER IMPACT:
 
 REPRODUCING RATE:
 
 For FT PR, Please list reference mobile's behavior:
 
 ++++++++++ end of initial bug #407751 description ++++++++++
 
 
 
 CONTACT INFO (Name,Phone number):
 
 DEFECT DESCRIPTION:
 
 REPRODUCING PROCEDURES:
 
 EXPECTED BEHAVIOUR:
 
 ASSOCIATE SPECIFICATION:
 
 TEST PLAN REFERENCE:
 
 TOOLS AND PLATFORMS USED:
 
 USER IMPACT:
 
 REPRODUCING RATE:
 
 For FT PR, Please list reference mobile's behavior:

Comment 2

[Andrew Overholt [:overholt]]

What is mess code?

Comment 3

[ying.tan]

Attachment: mess code image

The page can only display utf-8 encoded string. However,the string reading from
the ID3 field of some mp3 files is based on GB2312 encoding.

Comment 4

[hu xia]

Please see mess cose image

Comment 5

[Andrew Overholt [:overholt]]

Thanks, I've clarified the summary a bit.

Comment 6

[David Flanagan [:djf]]

I've heard that MP3's with non-Unicode text are common in Asia, but are non actually compliant with the ID3 specification (I haven't checked this recently, though). Anyone know how to tell what the encoding of an ID3 tag is?  What's the trick to tell if it is valid utf8 or GB2312?

And if we can identify a string as GB2312, how do we then display it correctly? Do we have to convert it to Unicode, or can we declare a charset for just a single element within an HTML document?  I think Gecko has new string conversion utilities, but I don't know if they're in b2g18 or if they're newer than that.

I wrote the current ID3 parser code, but am not really sure I'm qualified to take on this bug.  Anyone who is should look at apps/music/js/metadata.js and shared/js/blobview.js

Seting needinfo on the reporter to ask them to attach an MP3 file with this sort of ID3 tag for use in testing.

Comment 7

[Andrew Overholt [:overholt]]

(In reply to David Flanagan [:djf] from comment #6)
> And if we can identify a string as GB2312, how do we then display it
> correctly? Do we have to convert it to Unicode, or can we declare a charset
> for just a single element within an HTML document?  I think Gecko has new
> string conversion utilities, but I don't know if they're in b2g18 or if
> they're newer than that.

Bug 764234 (String encoding/decoding API) was backported to b2g18 (from [1]):

  https://hg.mozilla.org/releases/mozilla-b2g18/rev/44ccc86fa232
  https://hg.mozilla.org/releases/mozilla-b2g18/rev/e94004b134a3

Not sure if that's sufficient for this purpose ...

[1]
https://bugzilla.mozilla.org/show_bug.cgi?id=814257#c43

Comment 8

[ChengLin.Wu]

Attachment: the ID3 noncoding UTF-8 MP3 file

Comment 9

[ChengLin.Wu]

Comment on attachment 730024 [details]
the ID3 noncoding UTF-8 MP3 file

the string of ID3 field is not encoded on Unicode.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

Please renom if this issue would block our v1.1 target markets - which are not in Asia - otherwise this is for a v-next.

Comment 11

[sync-1]

Created an attachment (id=390528)
 the song show mess code
 
 Other language also has mess code.
 
 And the mess code not only happen in chinese song, but also happen in English
 song.

Comment 12

[Daniel Coloma:dcoloma]

Based on comment 6 I think this is not blocking 1.0.1 relaase as:

1/ It does not affect target markets
2/ It is not compliant with the standard

Comment 14

[Cheng-An, XIONG]

Attachment: english song with mess code

Comment 15

[Cheng-An, XIONG]

(In reply to Daniel Coloma:dcoloma from comment #12)
> Based on comment 6 I think this is not blocking 1.0.1 relaase as:
> 
> 1/ It does not affect target markets
> 2/ It is not compliant with the standard

please refer to comment 11 and comment 14. english song also show mess code.

Comment 16

[Daniel Coloma:dcoloma]

My opinion is still the same, this is a corner case and we have bigger fish to fry. I would tef- but asking Moz for feedback.

Comment 17

[Jim Porter (:squib)]

(In reply to David Flanagan [:djf] from comment #6)
> I've heard that MP3's with non-Unicode text are common in Asia, but are non
> actually compliant with the ID3 specification (I haven't checked this
> recently, though). Anyone know how to tell what the encoding of an ID3 tag
> is?  What's the trick to tell if it is valid utf8 or GB2312?

I mentioned this at the work week, I think, but the way you'd tell if an id3 tag is using a non-standard encoding is to write a heuristic that detects the most likely encoding.

As far as I'm aware, this is really only an issue in Asian countries (Russia included). I believe this is very rare in European and American countries, so we can probably keep this a low priority until we launch a device in Asia.

Comment 19

[Jim Porter (:squib)]

I discussed some strategies for handling this with :djf on IRC, and we both agreed that when implementing the heuristic to detect non-standard encodings, we should have some kind of pref stored in the Settings API that carriers can configure for each market. That way, this can be turned off entirely in American and European markets, so that we don't over-aggressively use the heuristic on standards-compliant id3 tags.

Comment 24

[BMO Automation]

Firefox OS is not being worked on