1911111 - Alt-Svc header for Http3 is sometimes ignored in 128.0.3, unknown protocol h3

Status: UNCONFIRMED

(Core :: Networking, defect, P2)

Description

[Luke]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0

Steps to reproduce:

In the current patch release of Firefox, I can't get a site running nginx to upgrade from Http2 to Http3. The logging seems to claim h3 is an unknown protocol, however Http3 is working with google.com. It works in previous releases of Firefox and Nightly, as well as in other browsers. I suspect it's potentially a regression from https://bugzilla.mozilla.org/show_bug.cgi?id=1909666

This isn't my thread, but someone else has seemingly encountered a similar problem very recently: https://www.reddit.com/r/firefox/comments/1e9zmw4/how_does_firefox_decides_what_version_of_http_to/

Nginx version: 1.26.1
Directives:

server {
listen 443 quic reuseport default;
listen 443 ssl default_server;

listen [::]:443 quic reuseport default;
listen [::]:443 ssl default_server;
http2 on;

quic_retry on;
ssl_early_data on;
add_header alt-svc 'h3=":443"; ma=2592000'; # Also tried h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

...

}

Actual results:

Firefox 127.0: h3 (browserstack)
Firefox 128.0: h3 (browserstack)
Firefox 128.0.3: h2 (fresh install)
Firefox Nightly: h3 (fresh install)
Edge 127: h3 (old install)

Logging from Firefox 128.0.3:

2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp nsHttpHandler::NotifyObservers [this=267338d7200 chan=26700aa6040 event="http-on-examine-response"]
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp nsHttpChannel: No Strict-Transport-Security header, continuing load.
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp AltSvcMapping::ProcessHeader: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp Alt-Svc Response Header h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp Alt Svc unknown protocol h3, ignoring
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp Alt Svc unknown protocol h3-29, ignoring
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp nsHttpChannelAuthProvider::CheckForSuperfluousAuth? [this=26740ea4280 channel=26700aa6850]
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp continuation state has been reset
2024-08-01 12:19:16.937000 UTC - [Parent 44892: Main Thread]: D/nsHttp nsHttpChannel::ContinueProcessResponse3

Expected results:

Non-nightly versions of Firefox >= 128.0.3 should also upgrade to h3 instead of staying on h2.

Comment 1

[BugBot [:suhaib / :marco]]

The Bugbug bot thinks this bug should belong to the 'Core::Security: PSM' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Kershaw Chang [:kershaw]]

I think this means that Firefox failed to establish an HTTP/3 connection to the origin previously, so we put that origin into an excluded list.
Luke, could you confirm if this is the reason? Maybe you could also send the full http log to [e-mail], so I can take a look.
Thanks.

Comment 3

[Luke]

(In reply to Kershaw Chang [:kershaw] from comment #2)

I think this means that Firefox failed to establish an HTTP/3 connection to the origin previously, so we put that origin into an excluded list.
Luke, could you confirm if this is the reason? Maybe you could also send the full http log to [e-mail], so I can take a look.
Thanks.

Both Firefox 128.0.3 and Nightly were downloaded on my colleague's computer only after we configured HTTP/3 on the server and validated it worked in Edge. Unfortunately, the full logs from those tests are gone. How long is it before entries in the excluded list expire, and can that list be manually emptied?

However, this week we've pushed the configuration from an internal-only server to a public server, and this is what we're seeing for requests by user-agent using HTTP/3 vs HTTP/2:

Browser	Monday	Tuesday	Today*
Chromium 127.x	78.5%	69.0%	70.4%
Firefox 128.x	35.9%	17.7%	29.9%
Firefox 129.x	-	21.3%	33.0%

That's well above zero for Firefox but still lower than Chrome. I've also sent you a log from my own browser collected yesterday if you're still interested, but it's not a fresh install/profile like last week.

Comment 4

[Luke]

Hey just a heads up incase you look now, for the next couple of hours (approx) we are running with h2 only, for an unrelated reason

Comment 5

[Kershaw Chang [:kershaw]]

I've looked the log again and it still shows the same thing.

2024-08-06 16:43:00.088000 UTC - [Parent 8868: Main Thread]: D/nsHttp AltSvcMapping::ProcessHeader: h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400
2024-08-06 16:43:00.088000 UTC - [Parent 8868: Main Thread]: D/nsHttp Alt-Svc Response Header h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400
2024-08-06 16:43:00.088000 UTC - [Parent 8868: Main Thread]: D/nsHttp Alt Svc unknown protocol h3, ignoring
2024-08-06 16:43:00.088000 UTC - [Parent 8868: Main Thread]: D/nsHttp Alt Svc unknown protocol h3-29, ignoring
2024-08-06 16:43:00.088000 UTC - [Parent 8868: Main Thread]: D/nsHttp Alt Svc unknown protocol h3-27, ignoring

I assume the reason is that the domain name is on the excluded list, so Firefox refuses to use the alt-svc header. However, it might be worth giving it another try by creating a speculative connection. If the new connection works, we can remove the domain from the excluded list.

Comment 6

[Max Inden]

Given that Happy Eyeballs v3 always races H3 and H2, should we remove the alt svc exclusion list with happy eyeballs? I assume it does more harm (especially through added complexity) then it does good.

Comment 7

[Kershaw Chang [:kershaw]]

(In reply to Max Inden from comment #6)

Given that Happy Eyeballs v3 always races H3 and H2, should we remove the alt svc exclusion list with happy eyeballs? I assume it does more harm (especially through added complexity) then it does good.

Agreed. I think we should just remove that exclusion list.
I'll work on this.