1689796 - Slow page loading in Nightly after restart browser.

Status: RESOLVED FIXED

(Core :: Security: PSM, defect, P1)

Description

[Alice0775 White]

Steps to reproduce:

1. Create new profile
2. Restart several times
3. Click Wikipedia button from Top Sites

Actual Results:
Page loading is slow after click the button.
Profiler log Nightly: https://share.firefox.dev/39t5H4y

Expected Results:
Page loading is fast after click the button.
Profiler log Firefox85: https://share.firefox.dev/3cpPFKw

Comment 1

[Alice0775 White]

Attachment: about:support

Comment 2

[Alice0775 White]

Setting Home to "Blank Page" does not fix the slowness in Nightly.
Profiler log Nightly: https://share.firefox.dev/36qtErg

Comment 3

[Mike Conley (:mconley) (:⚙️)]

A quick look at the profile - I do agree that Core :: Networking is probably the right component: it looks like almost 10s is spent trying to establish a TLS connection.

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Thank you for the report, Alice.
It's not clear why TLS connection takes so much time. Could you also record some HTTP logging (new profile if possible - logs may contain private info)
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Thanks!

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Also, since it works in 85 but not in nightly, do you think you could try to get a regression range?
https://mozilla.github.io/mozregression/quickstart.html

Comment 6

[Valentin Gosu [:valentin] (he/him)]

Dragana suggested that it might be the HTTP3 experiment which is enabled on Nightly.
Indeed, given that it shows up in about:support, it's a probable cause.
Could you check if setting the network.http.http3.enabled pref to false fixes the issue for you?

Comment 7

[Alice0775 White]

Attachment: HTTP_LOG.zip

I attach HTTP LOG using the method "Logging HTTP activity by manually setting environment variables".

FYI, I can't seem to reproduce the problem, if I tried using the method "Using about:networking".

Comment 8

[Alice0775 White]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #6)

Dragana suggested that it might be the HTTP3 experiment which is enabled on Nightly.
Indeed, given that it shows up in about:support, it's a probable cause.
Could you check if setting the network.http.http3.enabled pref to false fixes the issue for you?

Setting network.http.http3.enabled = false did not fix. the issue.

Comment 9

[Alice0775 White]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #5)

Also, since it works in 85 but not in nightly, do you think you could try to get a regression range?
https://mozilla.github.io/mozregression/quickstart.html

I found a regression window, but it is old...

Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ae43921b1fe66df9bc8059b97232a030fc53c687&tochange=cf1159646b4577b0e7ed5589689a378c49d75626

Comment 10

[Dragana Damjanovic [:dragana]]

I looked at the log. We are spendnig a lot of time in TLS handshake.

the regression range is also about TLS handshake.
I will move it to the tight component.

Comment 11

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Can you capture a profile with the socket thread and the SSL threads included? (to capture the latter, add SSL* to the Add custom threads by name: field. Thanks!

Comment 12

[Alice0775 White]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #11)

Can you capture a profile with the socket thread and the SSL threads included? (to capture the latter, add SSL* to the Add custom threads by name: field. Thanks!

profiler: https://share.firefox.dev/3pEa1nd

Comment 13

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Oh - my mistake. It looks like you need just SSL (no *) to capture the SSL threads.

Comment 14

[Alice0775 White]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #13)

Oh - my mistake. It looks like you need just SSL (no *) to capture the SSL threads.

profiler: https://share.firefox.dev/2MOHLzx

Comment 15

[Alice0775 White]

FWIW,
Delete all files in folder profile\security_state, then the issue is fixed.

Comment 16

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Thanks! Do you still have the files crlite.filter and crlite.stash from that folder?

Comment 17

[Alice0775 White]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #16)

Thanks! Do you still have the files crlite.filter and crlite.stash from that folder?

Yes, I have these file in backup profile.

Comment 18

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Can you attach those to this bug? (there's nothing private in them)

Comment 19

[Alice0775 White]

Attachment: crlite.zip

Comment 20

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Thanks! I'm fairly confident I know what the issue is, but can you capture another profile and add BackgroundThreadPool to the custom list of names?

Comment 21

[Alice0775 White]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #20)

Thanks! I'm fairly confident I know what the issue is, but can you capture another profile and add BackgroundThreadPool to the custom list of names?

Profiler log : https://share.firefox.dev/2MAOYDB

Comment 22

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Thanks! How does this build perform? https://treeherder.mozilla.org/jobs?repo=try&revision=13ffa944bd99805402a2dbb4c9dd817bf1202058

Comment 23

[Alice0775 White]

(In reply to Dana Keeler (she/her) (use needinfo) (:keeler for reviews) from comment #22)

Thanks! How does this build perform? https://treeherder.mozilla.org/jobs?repo=try&revision=13ffa944bd99805402a2dbb4c9dd817bf1202058

Yes, the try-build fixed this problem!

Comment 24

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: Bug 1689796 - asynchronously load crlite stashes r?mbirghan!,bbeurdouche!

Loading an accumulated set of crlite stashes can take some time. To address
this, this patch dispatches an asynchronous background task to read the
accumulated set of crlite stashes in a way that doesn't block certificate
verification. Of course, this means that the stash information won't
necessarily be available for the first few verifications. This shouldn't be a
security concern as long as the crlite filter is no more than 10 days out of
date (the maximum lifespan of an OCSP response, which is what Firefox relies on
currently in release). Note that currently crlite filters as published by
remote settings regularly end up being more than 10 days old, which will be
addressed in https://github.com/mozilla/crlite/issues/153. Note further that
crlite is currently not being enforced by default on any channel, so making
this change now is not a security concern.

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b085cd0cb191
asynchronously load crlite stashes r=mbirghan,bbeurdouche

Comment 26

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/b085cd0cb191