1932369 - Investigate https-first downgrade timeout and adjust current value

Status: RESOLVED FIXED

(Core :: DOM: Security, task, P3)

Description

[Frederik Braun [:freddy] (OOO - back August 10th)]

We have numbers, so obviously we should do better than guess that 3 seconds is an interesting value :-)

https://mozilla.cloud.looker.com/explore/sql__s7r9tvqvpygkgv/sql_runner_query?qid=1PnKy5kIPNNz0zFVStXA0z&toggle=dat,pik,vis

Comment 1

[Simon Friedberger [:simonf]]

:maltejur do we actually have the right numbers here? My understanding is that the graph shows general load time not really load time discrepancy for HTTP vs HTTPS.

The graph seems to have very few numbers on actual downgrades. The highest bar is at 2 seconds with 10 occurrences. Is there any way to increase the scope of this or does it just happen extremely rarely?
And does this imply that at 2 seconds some kind of server side timeout was hit and we then downgraded because of that?

Comment 2

[Malte Jürgens [:maltejur]]

GLAM is a much better way for looking at our existing data for this: https://glam.telemetry.mozilla.org/fog/probe/httpsfirst_downgrade_time_schemeless/explore?aggType=avg&app_id=release&normalizationType=non_normalized (right-click on a point with a lot of clients and select "Distribution Comparison" for a histogram). I am also attaching an image of that histogram here for reference.

Generally, we can expect the following load times from HTTPS-First. The column "Downgrade Time Telemetry" is what we can see in the histogram, so times from both 2. and 3.

	Wait time for HTTP URL of site which	Without HTTPS-First	With HTTPS-First	Downgrade Time Telemetry
1.	supports HTTPS	D	A,	-
2.	fails (server error) on HTTPS but succeeds on HTTP	D	C + D,	C
3.a	fails (timeout) on HTTPS but succeeds on HTTP	D	3s + B + D	3s + B
3.b	supports HTTPS but succeeds > 3s faster on HTTP	-"-	-"-	-"-
4.	fails on HTTPS and HTTP	E / timeout	C + E / 3s + E / timeout	-

A - Load Time Successful HTTPS
B - Load Time Successful HTTP (without path)
C - Load Time Failed HTTPS
D - Load Time Successful HTTP
E - Load Time Failed HTTP

Additionally, from the other schemeless HTTPS-First telemetry we know that:

• 1. makes up about 93% of loads (excluding 4.)
• Of the successful downgrades (2. and 3.), 2. makes up about 66% and 3. makes up about 34%

We have two approaches to further reduce the load time which HTTPS-First adds to insecure loads. Either, we reduce the amount of potential upgrades (and thus downgrades) that we need to do. Bug 1884921 is something that already goes in that direction. Or, for the downgrades that eventually happen, we try to reduce the downgrade time (third column above) as much as possible.

The main way to potentially do that is to modify the timer after which we start the racing HTTP request, which this bug is about. This means reducing the extra load time introduced by 3.a, but trading that off with more loads being 3.b instead of 1.

We can't yet directly measure the amounts of downgrades we do "wrongly". We know the amount of 1. in relation to 3. as a whole, but not to 3.b specifically. Potentially, we could add telemetry for that. Instead of directly canceling the loading HTTPS request when the HTTP request from the timer succeeds, we could somehow let it continue in the background, and record whether it eventually failed or succeeded later. But architecturally, I am not sure how easily this is doable, and/or if this has any unforeseen side effects.

Still, there should be an optimum of average extra load time introduced by HTTPS-First, where the load time impact of more "wrong" downgrades at some point equals out the gain from the reduced timeout. The extra load time can be read off quite nicely from our existing downgrade time telemetry (whose specific goal of measuring the average extra time introduced by HTTPS-First). This means, instead of measuring 1. vs 3.b directly, we could do a small-scale experiment for figuring out that optimum. But we should also consider that:

• Only optimizing for load time in this case leaves out the impact on the percentage of successful upgrades we actually do. Generally, the more we lower the timeout, the more we decrease the percentage of upgrades we are able to do (which we don't want). We should be able to read this off our existing telemetry for the amount of successful upgrades (1.) compared with the amount of downgrades on the timer (3.).
• There is a lot of variance in the load time users experience. Some users may on average have a lot higher load time than the averages we are looking at here. The two racing requests will probably still both take more or less equally longer, but I would nonetheless expect somewhat more "wrong" downgrades for people with specifically bad network conditions.

So TLDR: I will look into if more telemetry to identify "wrong" downgrades is possible. If it is possible, we should be able to calculate an "optimal" average value with it pretty well already. Otherwise, we should do an experiment varying the timeout, and looking at our average downgrade time and percentage of downgrades on a timer we do.

Comment 3

[Malte Jürgens [:maltejur]]

Attachment: Screenshot_20241204_154633.png

Comment 4

[Frederik Braun [:freddy] (OOO - back August 10th)]

Simon made a release experiment with 5% of our users and we believe the timeout could be lowered. The impact of the timer is generally low, as it is only experienced once per incompatible site every 7 weeks (Firefox remembers when a site can not support HTTPS).

We're going to adjust it to be at 1500ms for now. This is subject to wider release experiments.

Comment 5

[Frederik Braun [:freddy] (OOO - back August 10th)]

Attachment: timing-study-january-2025-image.png

Comment 6

[Frederik Braun [:freddy] (OOO - back August 10th)]

Attachment: Bug 1932369 - reduce https-first/only downgrade timer to 1.5s r=ckerschb

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be419509b32c
reduce https-first/only downgrade timer to 1.5s r=ckerschb

Comment 8

[Sandor Molnar[:smolnar]]

Backed out for causing dt failures @ devtools/client/netmonitor/test/browser_net_waterfall-resize.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/2ea0e69d7c1839c5e3a0d244985fc41a220643bf

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | devtools/client/netmonitor/test/browser_net_waterfall-resize.js | The request timing element should be updated

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/75410e6f6777
reduce https-first/only downgrade timer to 1.5s r=ckerschb CLOSED TREE

Comment 10

[Sandor Molnar[:smolnar]]

Backed out by mistake, relanded.
sorry

Comment 11

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/75410e6f6777

Comment 12

[Frederik Braun [:freddy] (OOO - back August 10th)]

Thank you. :)