1520949 - Logins imported from Chrome (on Windows) have the wrong character encoding

Status: VERIFIED FIXED

(Toolkit :: Password Manager, defect, P1)

Description

[Matthew N. [:MattN]]

+++ This bug was initially created as a clone of Bug #1510964 +++

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0

Characters that are not in the a-z range aren't imported properly from Chrome (on Windows), probably due to a character encoding problem somewhere along the migration process.

See attachment 9032392 [details] and bug 1510964 comment 8.

Comment 1

[Jared Wein [:jaws] (please needinfo? me)]

This is happening because the code at this point assumes characters are one byte each,
https://searchfox.org/mozilla-central/rev/9eb30227b21e0aa40d51d9f9b08bb0b113c5fadb/toolkit/components/passwordmgr/OSCrypto_win.js#78,91

The proper fix should be to use [...(new TextEncoder()).encode()] and (new TextDecoder()).decode(new Uint8Array(arr)) instead of the current code for stringToArray and arrayToString, respectively. However with that change made I'm getting "Error: decryptData failed: 0".

The attached patch allowed me to import the password of "新年快樂新年快樂" successfully but the patch isn't complete.

Comment 2

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Bug 1520949 - Properly decrypt multibyte character passwords from Google Chrome. r?MattN,agashlin

Comment 3

[Jared Wein [:jaws] (please needinfo? me)]

Hey Adam, this code previously doubled the size of the DATA_BLOB struct that is passed to CryptProtectData/CryptUnprotectData. However I'm not sure that it was actually necessary and I believe the second half of the bytes just went unused. Could you take a look at https://phabricator.services.mozilla.com/D18800#C591672NL136 and let me know what you think?

Comment 4

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

I'm not familiar with this code or js-ctypes in general, but:

I assume it's because entropyArray.length is a number of array elements, and the elements are 2 byte WORDs, and DATA_BLOB's cbData field needs to know the size in bytes.

Comment 5

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

To clarify without resorting to a comment, you could use entropyData.elementType.size instead of 2, that's similar to the sizeof(entropyData[0]) that you'd use in C. (If I've understood this correctly.)

Comment 6

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Bug 1520949 - Remove unused third argument to OSCrypto.decryptData. r?MattN

Comment 7

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Bug 1520949 - Update Chrome 'Login Data' file using Chrome 72.0.3626.109. r?MattN

Comment 8

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Bug 1520949 - Use TextEncoder/TextDecoder to handle multibyte characters in passwords. r?MattN

Comment 9

[Jared Wein [:jaws] (please needinfo? me)]

(In reply to [away 2/21-2/26] Jared Wein [:jaws] (Regression Engineering Owner for 65) (please needinfo? me) from comment #8)

Created attachment 9045546 [details]
Bug 1520949 - Use TextEncoder/TextDecoder to handle multibyte characters in passwords. r?MattN

This is a minimal patch which fixes the issue reported in this bug.

Try push, https://treeherder.mozilla.org/#/jobs?repo=try&revision=95c008afba3e2b3a63a63db8ed06a83d965939ad

I'll move the other code cleanups to bug 1529495.

Comment 10

[Jared Wein [:jaws] (please needinfo? me)]

(In reply to [away 2/21-2/26] Jared Wein [:jaws] (Regression Engineering Owner for 65) (please needinfo? me) from comment #9)

I'll move the other code cleanups to another bug.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/85b22be06489
Use TextEncoder/TextDecoder to handle multibyte characters in passwords. r=MattN

Comment 12

[Matthew N. [:MattN]]

QE should verify that Logins are imported successfully from Chrome on Windows and IE on Windows 7 (especially ones with accents/non-ASCII).

IE on Windows 8 and Microsoft Edge use different storage and encryption so it's important to test on Windows 7. It may be useful to confirm that accented characters and imported fine from Edge and/or Windows 8+ but that would be for a separate bug.

Comment 13

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/85b22be06489

Comment 14

[Daniel Bodea [:danibodea]]

build is not yet available. NI to myself to verify later.

Comment 15

[Daniel Bodea [:danibodea]]

I have tested all the necessary scenarios as requested:

Windows 7:

1. Importing passwords from IE11 v11.0.9600.19266 to Nightly v67.0a1 (2019-02-25)

• the import could not be performed. The Nightly browser's feature to import passwords from Internet Explorer is apparently performed as intended, but the passwords are not being imported at all. This article confirms this behavior:
  https://www.techwalla.com/articles/how-to-move-cookies-and-autocomplete-passwords-to-a-new-computer

2. Importing from Chrome v72.0.3626.119 to Nightly v67.0a1 (2019-02-25)

• The password was successfully and correctly imported.

Windows 8:
3. Importing passwords from IE11 v11.0.9600.19130 to Nightly v67.0a1 (2019-02-25)

• As opposed to the behavior on Windows 7, on Windows 8 the passwords were successfully and correctly imported.

4. Importing passwords from Chrome v72.0.3626.119

• The password was successfully and correctly imported.

Windows 10:
5. Importing passwords from IE11 v11.590.17134.0 to Nightly v67.0a1 (2019-02-25)

• The password was successfully and correctly imported.

6. Importing passwords from EDGE v42.17134.1.0 to Nightly v67.0a1 (2019-02-25)

• The passwords were successfully and correctly imported.

7. Importing passwords from Chrome v 72.0.3626.119 to Nightly v67.0a1 (2019-02-25)

• The passwords were successfully and correctly imported.

Considering all of the above, I can definitely say that the issue is now fixed, but take into consideration the following:
a. All these situations were tested using the credentials of Facebook being saved with the password "MåäöÅÄÖ1".
b. On Windows 7, the passwords would not get imported (probably intended, but not sure).

Should I also test this with more credentials at once? Are there different types of credentials so that it would make sense to try different saved passwords to import at once?
Is it intended that passwords would not be imp[orted on Windows 7 from IE11?

Comment 16

[Matthew N. [:MattN]]

(In reply to Bodea Daniel [:danibodea] from comment #15)

I have tested all the necessary scenarios as requested:

Windows 7:

1. Importing passwords from IE11 v11.0.9600.19266 to Nightly v67.0a1 (2019-02-25)

• the import could not be performed. The Nightly browser's feature to import passwords from Internet Explorer is apparently performed as intended, but the passwords are not being imported at all. This article confirms this behavior:
  https://www.techwalla.com/articles/how-to-move-cookies-and-autocomplete-passwords-to-a-new-computer

This article doesn't say anything about Firefox not importing IE passwords. Our compatibility table is at https://wiki.mozilla.org/QA/Firefox_migrators#Supported_data_types which says it should work for form passwords. It was implemented in bug 682069.

It would have been helpful if you included the Browser Console output. Does importing of IE history work on that computer? The login you want to import has to have been visited in your IE history. Can you test again with a form password and make sure that its in your IE history and that that IE history is successfully imported. Please also include Browser Console output if it doesn't work.

Comment 17

[Liviu Seplecan]

I've tested this issue on Windows 7 & Windows 10. On Windows 7 IE -> I couldn't import any data to Firefox Nightly even if I perform the following steps: Go to a site, create an account and click on "Save" (in order to save password and data in browser history) -> Open Firefox nightly -> go to Menu->Options-> Privacy & Security->Login & Passwords -> Click on Saved Login -> on the pop-up opened click on Import button-> select IE and -> Click twice on the "Next" button then notice that no data will be imported.
On Windows 10 I wasn't able to reproduce the issue. As well, from IE or Chrome the data will be imported and the password used "MåäöÅÄÖ1" is rendered correctly.

Comment 18

[Jared Wein [:jaws] (please needinfo? me)]

(In reply to Liviu Seplecan from comment #17)

I've tested this issue on Windows 7 & Windows 10. On Windows 7 IE -> I couldn't import any data to Firefox Nightly even if I perform the following steps: Go to a site, create an account and click on "Save" (in order to save password and data in browser history) -> Open Firefox nightly -> go to Menu->Options-> Privacy & Security->Login & Passwords -> Click on Saved Login -> on the pop-up opened click on Import button-> select IE and -> Click twice on the "Next" button then notice that no data will be imported.
On Windows 10 I wasn't able to reproduce the issue. As well, from IE or Chrome the data will be imported and the password used "MåäöÅÄÖ1" is rendered correctly.

Did import fail on Windows 7 IE before this patch as well?

Comment 19

[Daniel Bodea [:danibodea]]

After further investigation, I have observed the following:

1. bug 1191175: Password Import from IE not available for HTTP basic authentication

• I don't know if this is the issue that is blocking Firefox from importing passwords saved in IE11

2. The error displayed in the Browser Console when attempting to import passwords saved in IE11:
   "We failed to decrypt and import some logins. This is likely because we didn't find the URLs where these passwords were submitted in the IE history and which are needed to be used as keys in the decryption."

3. Windows 7 and/or IE11 does not have the features that Windows 8 and 10 have with IE11/EDGE.

• More exactly, when going into Control Panel/Credentials Manager, there's no separate section for Web Credentials, only for Windows Credentials, which does not cover the credentials saved for the web.
• Furthermore, when going into IE11/Internet Options/Content tab/AutoComplete section, Settings button, there is no Password Manager button to click and check the saved web credentials.
  -> So basically, there's no other way to see the saved credentials then go into the webpage and see that the credentials are being autocompleted into the input boxes.

4. The complete testing process I followed is this:
   a. Firstly, I have saved the passwords into IE11 using the auto-save feature ("User names and passwords on forms" is CHECKED and "Ask me before saving passwords" is CHECKED, settings foundin IE11 at Internet Options, Content tab, Auto-Complete Settings button), logged into some websites and chose to save the credentials in the automatic pop-up.
   b. I have confirmed that the passwords were saved by logging out and reloading the page to see whether the credentials were actually saved, then again after a browser restart.
   c. I have confirmed that the login attempt was saved in the IE11 browser history.
   d. I have attempted to import the credentials into Firefox using Firefox's import feature.
   -> Did not work. Also, History was also not imported from IE11.

5. To answer your remaining questions from comment 16:

• "Does importing of IE history work on that computer?" -> Yes, it worked correctly when importing passwords from Chrome to Firefox on the same machine and Windows 7 OS.
• "The login you want to import has to have been visited in your IE history. Can you test again with a form password and make sure that it's in your IE history and that that IE history is successfully imported." -> Does the process in point 4 above cover this?

6. The only way to manage passwords saved in IE11 on Windows 7 is like described in this Microsoft forum answered question:
   https://social.technet.microsoft.com/Forums/en-US/5a85b30f-035f-49e8-97c6-0398386518d2/manage-passwords-for-internet-explorer-11-on-windows-7-?forum=w7itprogeneral

7. Per request from [:jaws], I have also attempted to import the passwords from IE11 on an older Firefox build (Build ID: 20190215045130), prior to the patch, and the reproduction was the same. No passwords imported, no history imported, error (point 2) in Browser Console.

I hope this covers everything. If not, feel free to NI me to test other scenarios. Thanks.

Comment 20

[Liz Henry (:lizzard) (relman/hg->git project)]

Do you want to uplift this as it seems to improve the current import problems ?

Comment 21

[Jared Wein [:jaws] (please needinfo? me)]

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #20)

Do you want to uplift this as it seems to improve the current import problems ?

We should let this ride the trains. We haven't gotten many bug reports for it, it's not a recent regression, and there are remaining questions about Windows 7 that we are still trying to understand.

Comment 22

[Daniel Bodea [:danibodea]]

Considering that all the scenarios but one are working as expected, should we mark it as verified?
The only remaining scenario is the one where the user attempts to import from IE11 on Windows 7. Is this scenario actually fixable on our side?

Comment 23

[Matthew N. [:MattN]]

(In reply to Bodea Daniel [:danibodea] from comment #19)

After further investigation, I have observed the following:

1. bug 1191175: Password Import from IE not available for HTTP basic authentication

• I don't know if this is the issue that is blocking Firefox from importing passwords saved in IE11

That's not relevant since I was asking you to test form logins, not HTTP auth dialogs. As I said in comment 16, import of those was implemented in bug 682069.

2. The error displayed in the Browser Console when attempting to import passwords saved in IE11:
   "We failed to decrypt and import some logins. This is likely because we didn't find the URLs where these passwords were submitted in the IE history and which are needed to be used as keys in the decryption."

Thank you, can you please file a bug blocking bug 682069 with this information so we can fix it.

3. Windows 7 and/or IE11 does not have the features that Windows 8 and 10 have with IE11/EDGE.

• More exactly, when going into Control Panel/Credentials Manager, there's no separate section for Web Credentials, only for Windows Credentials, which does not cover the credentials saved for the web.
• Furthermore, when going into IE11/Internet Options/Content tab/AutoComplete section, Settings button, there is no Password Manager button to click and check the saved web credentials.
  -> So basically, there's no other way to see the saved credentials then go into the webpage and see that the credentials are being autocompleted into the input boxes.

Right, that's expected and why we need to test there separate from the other IE/Windows versions.

4. The complete testing process I followed is this:
   a. Firstly, I have saved the passwords into IE11 using the auto-save feature ("User names and passwords on forms" is CHECKED and "Ask me before saving passwords" is CHECKED, settings foundin IE11 at Internet Options, Content tab, Auto-Complete Settings button), logged into some websites and chose to save the credentials in the automatic pop-up.
   b. I have confirmed that the passwords were saved by logging out and reloading the page to see whether the credentials were actually saved, then again after a browser restart.
   c. I have confirmed that the login attempt was saved in the IE11 browser history.
   d. I have attempted to import the credentials into Firefox using Firefox's import feature.
   -> Did not work. Also, History was also not imported from IE11.

If history isn't imported then that's the problem (as mentioned in the error message above). Please include that in your new bug.

5. To answer your remaining questions from comment 16:

• "Does importing of IE history work on that computer?" -> Yes, it worked correctly when importing passwords from Chrome to Firefox on the same machine and Windows 7 OS.

Sounds like a no to me.

• "The login you want to import has to have been visited in your IE history. Can you test again with a form password and make sure that it's in your IE history and that that IE history is successfully imported." -> Does the process in point 4 above cover this?

Yes, that's likely the reason it doesn't work.

6. The only way to manage passwords saved in IE11 on Windows 7 is like described in this Microsoft forum answered question:
   https://social.technet.microsoft.com/Forums/en-US/5a85b30f-035f-49e8-97c6-0398386518d2/manage-passwords-for-internet-explorer-11-on-windows-7-?forum=w7itprogeneral

7. Per request from [:jaws], I have also attempted to import the passwords from IE11 on an older Firefox build (Build ID: 20190215045130), prior to the patch, and the reproduction was the same. No passwords imported, no history imported, error (point 2) in Browser Console.

I hope this covers everything. If not, feel free to NI me to test other scenarios. Thanks.

Thanks. I think we can call this verified (even though we can't test the IE portion which is already broken due to not importing history).

I would also like to understand how QA never reported broken IE history import before? We can discuss that in your new bug.

Thanks

Comment 24

[Daniel Bodea [:danibodea]]

I have logged this issue to cover the remaining scenario: bug 1533288.
This bug will be marked as Verified. Also removing the "qe-verify+" tag.

Thank you!