246403 - Bad import bookmarks from Opera (non-ASCII characters garbled)

Status: RESOLVED FIXED

(Firefox :: Migration, defect)

Description

[Pavel Cvrcek [:JasnaPaka]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040608 Firefox/0.8.0+
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040608 Firefox/0.8.0+

I use File -> Import and import bookmarks and settings from Opera browser
(version 7.51). Problem is that national characters looks bad.

Opera 7.51: Look at "opera-bookmark.png"
Firefox 0.9 RC: Look at "firefox-bookmark.png"

Reproducible: Always
Steps to Reproduce:
1.
2.
3.

Comment 1

[Pavel Cvrcek [:JasnaPaka]]

Attachment: opera-bookmark

Comment 2

[Pavel Cvrcek [:JasnaPaka]]

Attachment: firefox-bookmark

Comment 3

[Asa Dotzler [:asa]]

I can confirm this with 0.9 on windows XP importing from Opera 7.5

Comment 4

[Pavel Cvrcek [:JasnaPaka]]

Example URL: http://www.idnes.cz/ (v. screenshots).

Comment 5

[Jure Repinc (JLP)]

I don't know if this is the same bug or I should open another one. When I
imported bookmarks from Mozilla all my bookmarks with apostrophs were imported
incorrectly. For example "JLP's" was imported as "JLP's". This is with
Firefox 0.9.1 on Windows XP.

Comment 6

[Pavel Cvrcek [:JasnaPaka]]

No, this isn't the same bug. 

Comment 7

[Tommi Mäkelä]

Same thing happens with Finnish characters ( ö,ä,Ö,Ä ). Bookmarks (or bookmark
folders) containing these characters are not properly imported from Opera. This
bug exists also in the Linux version of Firefox. All local characters ( ö,ä,Ö,Ä
) are shown as ~Ä (tilde followed by capital A with two dots over it).

Probably the same thing happens with Swedish ( å,Å ) characters, but I can not
confirm it.

Comment 8

[Jungshik Shin]

Can anyone upload an opera bookmarks file? I need to know how they store
non-ASCII characters in bookmarks file. 

Taking.

Comment 9

[Simon Montagu :smontagu]

Attachment: Opera bookmarks file

Here's an Opera bookmarks file with one Hebrew site and one Japanese. The
encoding of this file is UTF-8, but note that starts like this:

Opera Hotlist version 2.0
Options: encoding = utf8, version=3

Possibly the file could be in other encodings and we need to parse the
"Options" line?

Comment 10

[Jungshik Shin]

(In reply to comment #9)
Thanks. Somehow 'File | Import' doesn't work for me in Firefox when I choose an
opera bookmarks file. I ended up removing ~/.mozilla/firefox to make it believe
this is the first-run so that it prompts me to answer from which program to
import  bookmarks/preferences. 

> Options: encoding = utf8, version=3
> 
> Possibly the file could be in other encodings and we need to parse the
> "Options" line?

Maybe, that's the case. I didn't have any trouble importing (when I ran firefox
for the first time) opera bookmarks file in UTF-8. 

Pavel, can you take a look at what 'Options' line of your Opera bookmarks file
is like? You can upload it here if you want. 

Comment 11

[Pavel Cvrcek [:JasnaPaka]]

Attachment: Opera bookmark

Comment 12

[Pavel Cvrcek [:JasnaPaka]]

(In reply to comment #10)
> Pavel, can you take a look at what 'Options' line of your Opera bookmarks file
> is like? You can upload it here if you want. 

I upload example of Opera bookmars. With this file i have the same problem.

'Options' line
---------------
Options: encoding = utf8, version=3

Comment 13

[Nathan Boydston]

I just tried to import my Opera 7.54 bookmarks into Firefox 0.9.3, and the
import process stopped abruptly when it reached the folder that I will paste
below from Opera's bookmarks file.  When I went back into Opera and put in a
name for the folder, it worked perfectly.

---------------- Start here in Opera6.adr
#URL
    ID=314
    NAME=Top4x4Sites.com
    URL=http://top4x4sites.com/top.htm
    CREATED=1018320954
    VISITED=1018320954

-

#FOLDER
    ID=315
    CREATED=1093150353

-

--------------------------------------------- End Opera6.adr

Comment 14

[Pavel Cvrcek [:JasnaPaka]]

Firefox 1.0 RC1 has still this problem.

Comment 15

[Jungshik Shin]

I can reproduce this on Windows, but not on Linux. It's a mystery because I
can't find any platform-specific code in nsOperaProfileMigrator.cpp (especially
ParseBookmarkFolder())that can affect importing. 

On Windows, UTF-8 strings are interpreted as if they're in ISO-8859-1, but I
couldn't figure out where that happens by inspection. I need to build a debug
build on Windows and track it down. 

Comment 16

[Pavel Cvrcek [:JasnaPaka]]

I think that i know where is a problem. I will create a patch.

Comment 17

[Pavel Cvrcek [:JasnaPaka]]

Attachment: Patch

Patch

Comment 18

[Jungshik Shin]

Comment on attachment 163947 [details] [diff] [review]
Patch

r=jshin

Aha. that's why I didn't see it on my trunk build of firefox on Linux while
firefox 1.0PR on Windows has a problem. I thought the aviary-1.0 branch and the
trunk are are almost in sync as far as the profile migrator is concerned. The
trunk source doesn't have a single line of 'ASCIItoUTF16' which would have
turned up my alarm level immediately.  All I saw was 'CopyUTF8toUTF16'.

Comment 19

[Jungshik Shin]

this 1.0 only.

Comment 20

[Ben Goodger (use ben at mozilla dot org for email)]

Comment on attachment 163947 [details] [diff] [review]
Patch

since this is isolated, sr+a=[e-mail]. please land asap. freeze is
tuesday.

Comment 21

[Jungshik Shin]

fix checked into aviary-1.0 branch. thanks for sr/a and the patch.