1729930 - Browser hangs for 5-6 seconds when an auto-fill field is focused on the website

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Gopi Krishna]

Attachment: autofill-hang.mp4

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:93.0) Gecko/20100101 Firefox/93.0

Steps to reproduce:

Opened a webpage with a signup or sign in form.
Firefox already remembers the credentials to login.

Actual results:

When the input field to enter credentials comes into focus, the browser hangs for 5-6 seconds.
later a dropdown with list of available credentials opens below the input field.

Expected results:

as soon as the input field comes into focus, a dropdown with list of available credentials should popup below the input field almost immediately.

Comment 1

[Gopi Krishna]

I've tried this in "troubleshoot mode" and even after "refreshing" firefox. Same result.

Comment 2

[Andrei Purice]

Hey Gopi,
I tried reproducing this issue on the latest version of Firefox Nightly 94.0a1 (2021-09-10), beta 93.0b2 and release 92.0 but it doesn't occur for me.

Could you try with a fresh new profile? You can find more about creating a new profile here : https://support.mozilla.org/en-US/kb/troubleshoot-and-diagnose-firefox-problems#w_6-create-a-new-firefox-profile .
If possible, you can test this issue on the nightly build as well. Download the build from : https://www.mozilla.org/en-US/firefox/nightly

Comment 3

[Gopi Krishna]

Hey Andrei,

I've created a fresh new profile. Unable to reproduce this issue in the new profile.

I've installed a nightly build 94.0a1 (2021-09-12) (64-bit). Unable to reproduce the issue there as well.

I've launched the nightly build with the old profile where I was facing the issue. I am able to reproduce the issue there.

This could probably be because of some setting or some add-on I have installed in my old profile.

Let me know if you need any more details.

Comment 4

[Gopi Krishna]

On debugging further, I've also noticed that the about:logins page takes 30 seconds or more to show the list of logins in the faulty profile, on First time load and on consecutive refreshes as well.

So, I have copied the logins.json file and key4.db file from the faulty profile to a fresh profile. I was able to reproduce the issue in the fresh profile.

So, this issue may have everything to do with the credentials I've saved in my browser and nothing else.

I was able to work around this issue by syncing the credentials to my Firefox account and then, restoring them back into a fresh profile.

Comment 5

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Toolkit::Form Autofill' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 6

[Tim Giles [:tgiles]]

Hi Gopi, thanks for filing this. It seems like the issue lies in the logins file itself. 30 seconds is way too long to be decrypting these logins. Can you let us know how large the logins.json file is? Unless it's an extremely large file, I'm not sure why we would take more than 5 seconds or when navigating to about:logins. Do you happen to have some kind of anti-virus running? anti-virus can cause slowdown issues in situations like this. How old is the old profile?

Comment 7

[Gopi Krishna]

Hi Tim,

The logins file is not so big. I have 610 logins saved in the profile. File size is 347 KB on disk. I don't have any antivirus running. I have a master password set. I have been using this browser/profile for almost a year now. But I have synced it to my Firefox account, which has been active for more than 5 years.

I feel that one or more of the encrypted username/password strings might have been corrupted. The issue goes away if I remove or change the master password.

Comment 8

[Dimi Lee [:dimi]]

This looks like related to Bug 1726022, which has been fixed in 93. Not sure if there is still unsolved problem in it.

Hi Gopi,
Could you help check again whether you can still reproduce this bug with the 610 logins entries in Nightly 94?
If the bug is still reproducible it 94, it will be helpful if you can also test 92 with mozregression . So we can know if this bug is a regression of Bug 1724869.
Thank you!

Comment 9

[Gopi Krishna]

Attachment: mozregression.txt

Yes, I'm able to reproduce this issue in 94.0a1 (2021-09-27) (64-bit).

I have tried to run the mozregression. Attached the Logs.

It narrowed down to this commit. Hope this helps.

Comment 10

[Dimi Lee [:dimi]]

Hi Gopi, That is really helpful! Thank you for testing with mozregression!

Hi Beurdouche,
From Comment 9, it looks like this is related to Bug 1724869, could you help take a look at this issue? Thanks!

Comment 11

[Tim Giles [:tgiles]]

Moving this over to NSS due to mozregression pointing out a particular commit. Feel free to kick it back this way if things change.

Comment 12

[Benjamin Beurdouche [:beurdouche]]

What I suspect is happening here is that the problem is indeed related to https://bugzilla.mozilla.org/show_bug.cgi?id=1726022.
However, I am very surprised because the behaviour you mentionned for auto-fill should have been fixed at the same time as the about:logins one.

@Gopi Can I once more ask you to check if you still have trouble with a recent nightly, please?

Comment 14

[oneofthedamons]

I was the OP of Bug 1734503. I downloaded the current Nightly build, synced with my account, added a Primary Password, and restarted so that I needed to enter the Primary Password to view about:logins . At each stage I am unable to replicate this bug on the Nightly release, but confirm it in the current Production release. Please let me know if I can assist any further.

Comment 15

[Martin Thomson [:mt:]]

From what I can see, the fix is present in Firefox 93 (release) and all later versions. If this is still happening for you (not oneofthedamons, thanks for that), then it would be really helpful if you could gather a profile (https://profiler.firefox.com/) that includes the long hang. That will help us greatly in identifying whether this was an incomplete fix or a regression in other parts of the code.

Comment 16

[Gopi Krishna]

Hi Martin,

The issue is still persistent on 95.0a1 (2021-10-10) (64-bit) Nightly build.

Also, as I have mentioned before, If I change/remove the primary password or If I sync the passwords to Firefox account and back to nightly build, this issue doesn't come up.

I downloaded the current Nightly build, synced with my account, added a Primary Password, and restarted so that I needed to enter the Primary Password to view about:logins . At each stage I am unable to replicate this bug on the Nightly release

@oneofthedamons has confirmed the same above.

Most users will not be doing this when they upgrade from FF92 to FF93.

I have performed profiling of about:logins page here. Also, for the site with autofill form here.

Comment 17

[Valentin C.]

Hi,

I still have this issue on Firefox 93.0 release (Linux, build 20210927210923). For ~1000 password with a master password, it takes about 70s.
I tried to generate a profile : https://share.firefox.dev/3iQde1u (I'm not sure about parameters to use, tell me if you need another !)

Comment 18

[Benjamin Beurdouche [:beurdouche]]

@Bob, @MT. It looks like this is related to Bug 1726022. Any chance you can have another look?

Comment 19

[Valentin C.]

My bad, I wanted to comment on this linked bug. Auto-fill is working correctly for me, about:logins is not.

Comment 20

[Robert Relyea]

Looking at the description, It looks like something isn't quite right about the key4.db.

We can see this because the reporter says changing the password resolves the problem.
I would have to see the failing key4.db to debug this (we don't know how old the key4.db is, it could be from some quite old firefox release, or from something newer), unfortunately the reporter is unlikely to want to give us his key4.db (nor should he).

I suspect the easiest mitigation is to mark the database for 'update', which will do an internal 'change password' once you first log into the database. To do this we would need a sample failing key4.db (preferably one which doesn't have real peoples keys and passwords in it).
bob

Comment 21

[Martin Thomson [:mt:]]

I just looked at the profile and it's pretty clearly stuck in nsspkcs5_ComputeKeyAndIV. Bob, is there anything you can think of that might cause the depth 2 cache to not work as intended? I'm looking at Bug 1720226, which included a fallback for a bug. If the database has that bug, could we be cycling through more than 2 potential keys? If that's the case, would doubling the cache size (which is still tiny) help smooth this over for people?

Maybe the best thing is to have the database password "changed", either for people having trouble (if we can detect that somewhat reliably) or just for everyone with a master password. Perhaps we can just do that from Firefox unconditionally after a master password is provided.

I noticed a LOT of IPC in the profile (15% of the time during the jank). That might be worth looking into from the perspective of the front end code.

Comment 22

[oneofthedamons]

Confirming changing the Primary Password resolved the issue in the Production release for me, too. Indeed, I didn't even have to change the password to something different — I just typed the same password into Change Primary Password and it reduced the time taken to show about:logins from 20+ seconds to around 1 second (291 logins).

Comment 23

[Gopi Krishna]

Do you guys have any idea why the issue disappears once we change the primary password?

And, is there anything I can try and check locally without sharing the key4.db file?

Comment 24

[Robert Relyea]

If someone can find a test version of the failing key4.db, it could help narrow this down. I suspect it has something to do with bad macs (I wonder if the failover case is again swamping the cache). A simple test would be to bump KDF2_CACHE_COUNT from 2 to 4. When I made the previous patch, I made the count configurable with the #define in lowpbe.c, so increasing the cache count could have an effect.

Changing the password updates the mac records to the correct value and location, so everything falls nicely into the cache again.

bob

Comment 25

[Dennis Jackson]

I've managed to reproduce this locally and test that bumping the KDF2_CACHE_COUNT from 2 to 3 does largely mitigate the problem. I've attached a minimal example database with 1500 logins and a primary password 'example2'. I created this database by starting a new profile on Firefox 91, adding some logins and changing the primary password, then opening it in Firefox 93.

Looking at the changesets, I believe this issue is impacting all users who have set a primary password and changed it between Firefox 87 (when the KDF iterations were increased) and prior to Firefox 93. The impact is likely worse for users who first set a password since Firefox 87 and changed it at least once. Users that have never set or set but never changed their primary password don't seem to be affected, based on my testing.

Note that increasing the kdf2 cache does not entirely fix the performance regression, its still a few seconds slower than previous releases, but it fixes the worst of the impact.

mozregression is pretty handy for launch older Firefox versions, as it fetches the builds automatically.

mozregression --launch RELEASE_NUMBER  --profile-persistence reuse --profile TEST_DIRECTORY

Comment 26

[Dennis Jackson]

Attachment: minimal-failing.zip

Comment 27

[Dennis Jackson]

Attachment: Bug 1729930 - Increase KDF cache size to mitigate perf regression in about:logins r=bbeurdouche

Comment 28

[Robert Relyea]

Dennis, does bumping it to 4 help any?

Comment 29

[Benjamin Beurdouche [:beurdouche]]

https://hg.mozilla.org/projects/nss/rev/de3db3a55aef9c03deebee9a4db5b53276a1026d

Comment 30

[Benjamin Beurdouche [:beurdouche]]

@Bob, Dennis told me he will take another look at 4 but his preliminary testing showed no difference between 3 and higher values (not 4 specifically)...

Comment 31

[Dennis Jackson]

@Bob: For my test cases, caches larger than 3 don't result in any observable differences in performance. Can you think any scenario in which we'd be trying to compute more than three keys?

@Gobi, @oneofthedamons, @Valentin - Thank you for reporting this issue to us and helping us diagnose it! A fix is now shipping in the latest Nightly. I've tested it against databases I've created locally, but if you have time to run it against your older databases and confirm it resolves the performance issue, that would be very helpful. I'm also looking into how we can improve our automated tests to ensure this issue doesn't arise again with any future changes to the password database.

Comment 32

[Gopi Krishna]

Hi Dennis,

Just tested the about:logins and autofill in Firefox nightly 95.0a1 (2021-10-19) (64-bit). It resolved the issue. Thank you.

Comment 33

[Robert Relyea]

OK, then I think we'll keep the patch at 3. Thanks.
bob

Comment 34

[Dana Keeler (she/her) [:keeler]]

This landed, right?

Comment 35

[Benjamin Beurdouche [:beurdouche]]

@Dana, yes this landed, we kept the bug open for monitoring...
@Dennis, can you please close that bug when you think we had enough monitoring time? Thank you!

Comment 38

[Mathew Hodson]

Can this be closed now?