1573542 - Slow performance (switching tabs, interacting with map) on this map

Status: VERIFIED FIXED

(Core :: Security: PSM, defect, P1)

Description

[Andrew Overholt [:overholt]]

Interacting with https://trca.ca/conservation/flood-risk-management/flood-plain-map-viewer/#map leads to intermittent lack of ability to move the map and also the ability to switch tabs or interact with the browser. I can make it happen after restarting the browser, too.

Windows 10 on a 2017 Surface Laptop
Yesterday's Nightly
Intel GPU, WebRender enabled, External 4k monitor

My super quick look at the profile (https://perfht.ml/33GGsH8) seems to indicate long GC times?

Comment 1

[Andrew Overholt [:overholt]]

I discussed this briefly with Ehsan (who can't reproduce) and at the same time captured https://perfht.ml/33rnbcB. He suggested main thread I/O and it looks like it's coming from NSS. I don't seem to have any weird certificates or anything.

Thyla, do you know who'd be good to investigate?

Comment 2

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Oof. I guess we do run the client auth certificate discovery code on the main thread. Do you actually have/use a client auth certificate? Also, what antivirus software are you using? Additionally, can you capture the socket thread and ssl verification threads in addition to the main thread? Thanks!

Comment 3

[Andrew Overholt [:overholt]]

Attachment: viewcertificates.PNG

Comment 4

[Andrew Overholt [:overholt]]

Attachment: certs.PNG

Comment 5

[Andrew Overholt [:overholt]]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #2)

Do you actually have/use a client auth certificate?

I don't think so. Is this the dialog that would show it? https://bug1573542.bmoattachments.org/attachment.cgi?id=9085220 FWIW, when I click on "View Certificates" (https://bug1573542.bmoattachments.org/attachment.cgi?id=9085220), Firefox's main thread pauses and I get the Windows "Nightly is not responding" title bar modification thing.

Also, what antivirus software are you using?

Windows Defender.

Additionally, can you capture the socket thread and ssl verification threads in addition to the main thread? Thanks!

https://perfht.ml/2TyzMWW - is it just appending ",ssl" to the "Add custom threads by name:" box in the profiler UI?

Comment 6

[Dana Keeler (she/her) (use needinfo) [:keeler]]

https://bugzilla.mozilla.org/attachment.cgi?id=9085221 says you don't have any, but I guess the implementation looks for them anyway...
Can you attach the cert9.db file from your profile here? (it shouldn't have any private information in it - just cached public intermediate certificates and any certificates you have imported yourself (so those might be private if you have done that)) (another option is to email it to me).
Also, do you have a way of seeing if any other processes are opening that file in your profile while Firefox is running?

Comment 7

[Andrew Overholt [:overholt]]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #6)

Can you attach the cert9.db file from your profile here?

I emailed you my cert db.

Also, do you have a way of seeing if any other processes are opening that file in your profile while Firefox is running?

Watching Resource Monitor (and refreshing the search in the CPU tab's "Associated Handles" search field for |cert9.db|) while opening and interacting with https://trca.ca/conservation/flood-risk-management/flood-plain-map-viewer/#map seems to indicate only firefox.exe is accessing cert9.db.

Comment 8

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Andrew, if you try the build at https://treeherder.mozilla.org/#/jobs?repo=try&revision=cd4a882b8d85c0037b9b292498c00608ccf151de with the same profile you were using before, how is the performance?

Comment 9

[Andrew Overholt [:overholt]]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #8)

Andrew, if you try the build at https://treeherder.mozilla.org/#/jobs?repo=try&revision=cd4a882b8d85c0037b9b292498c00608ccf151de with the same profile you were using before, how is the performance?

Performance is great! Nice work 👍

Comment 10

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: bug 1573542 - be more efficient about finding client certificates r?jcj,kjacobs

Before this patch, Firefox would call CERT_FindUserCertsByUsage to gather all
known client certificates. This function enumerates all known certificates and
filters some of them out. When there are many certificates that are not client
certificates (e.g. roots and intermediates), this is inefficient. Since this is
likely to be the case for most users, this patch optimizes this task by instead
first searching for private keys and then gathering all certificates that have
corresponding public keys.

Comment 11

[Dana Keeler (she/her) (use needinfo) [:keeler]]

(In reply to Andrew Overholt [:overholt] from comment #9)

Performance is great! Nice work 👍

Thanks!

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c37a2bd30156
be more efficient about finding client certificates r=jcj,kjacobs

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/c37a2bd30156

Comment 14

[Daniel Bodea [:danibodea]]

I did not manage to reproduce this issue on an affected Nightly build (v70.0a1 from 32019-08-12) on a Microsoft Surface Pro 4 (1724) and on a Microsoft Pro 2 with an external monitor and webrender enabled. Furthermore, I have compared it to the fixed build (latest nightly) while dragging the map and interacting with the map's controls shows no performance differences were observed between the affected and fixed builds.

This being said, I cannot seem to properly reproduce the issue OR properly verify it.
@Andrew: Can you verify this fix? We need you to verify it on all the channels where the fix has reached. Do you have some time?

1. Nightly: http://archive.mozilla.org/pub/firefox/nightly/2019/12/2019-12-06-09-39-47-mozilla-central/
2. Beta: http://archive.mozilla.org/pub/firefox/releases/72.0b3/
3. Release: http://archive.mozilla.org/pub/firefox/releases/71.0/

Thank you for your contribution!

Comment 15

[Andrew Overholt [:overholt]]

I can verify (seems I could in comment 9 and can still) that it at least is fixed on Nightly :)

Comment 16

[Daniel Bodea [:danibodea]]

It was fixed in Firefox 71, which is now the latest release version. so it should be fixed in al the 3 channels (Release, Beta and Nightly).
Are you saying that it appears to be fixed in Nightly only? Did you check a build for each channel?

Comment 17

[Andrew Overholt [:overholt]]

I don't have time to check a build for each channel but it was fixed on nightly back when I tested it and it's still fixed in nightly.

Comment 18

[Daniel Bodea [:danibodea]]

Considering the fact that it wasn't uplifted but rode the trains and also the fact that I cannot reproduce it myself, I will consider this verification sufficient. Thank you.

Comment 19

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.