2034178 - Facebook does not open in Firefox 150.0 (HTTP/3 stalled by endpoint protection?)

Status: VERIFIED FIXED

(Core :: Networking, defect)

Description

[DoctorWho]

Attachment: Browser Agent.jpg

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0

Steps to reproduce:

This occurred after the update to Firefox 150.0.
I am able to open Facebook in the Brave browser.

Comment 1

[Krzysztof]

There's multiple reports on SUMO.

Lowering security.tls.version.max to 3 helps, but Firefox for Android is also affected.

Comment 2

[Benjamin Beurdouche [:beurdouche]]

@DoctorWho I am sorry to ask but where do you access Facebook from ?
(We are multiple mozillians that connot reproduce from our respective location)

Comment 3

[Kershaw Chang [:kershaw]]

Hi Reporter,

It would be really helpful if you could also capture a networking log for this issue. Thanks!

Comment 4

[dbert721]

Hack that worked for me:
about:config, search for security.tls.version.max and change it from 4 to 3.
This worked for a day, then got reset back to 4 the next day, so I had to re-apply the fix.

Comment 5

[Kershaw Chang [:kershaw]]

(In reply to dbert721 from comment #4)

Hack that worked for me:
about:config, search for security.tls.version.max and change it from 4 to 3.
This worked for a day, then got reset back to 4 the next day, so I had to re-apply the fix.

Hi, if you are still able to reproduce, could you try to record a http log?
Please send the log or profiler to [e-mail].
Thanks.

Comment 6

[dbert721]

(In reply to Kershaw Chang [:kershaw] from comment #5)

(In reply to dbert721 from comment #4)

Hack that worked for me:
about:config, search for security.tls.version.max and change it from 4 to 3.
This worked for a day, then got reset back to 4 the next day, so I had to re-apply the fix.

Hi, if you are still able to reproduce, could you try to record a http log?
Please send the log or profiler to [e-mail].
Thanks.

Log file sent

Comment 7

[MIKÓSDI Balázs]

(In reply to DoctorWho from comment #0)

Created attachment 9572579 [details]
Browser Agent.jpg

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0

Steps to reproduce:

This occurred after the update to Firefox 150.0.
I am able to open Facebook in the Brave browser.

Comment 8

[Oskar Mansfeld [:omansfeld]]

(In reply to dbert721 from comment #4)

Hack that worked for me:
about:config, search for security.tls.version.max and change it from 4 to 3.
This worked for a day, then got reset back to 4 the next day, so I had to re-apply the fix.

Hey, sorry to bother even more.

If you can still reproduce the issue could you check if setting security.tls.version.max=4, so back to it's default, while also setting network.http.http3.enable=false solves the issue?

That would help us narrow down the cause.

Thanks a lot!

Comment 9

[dbert721]

Yes- it looks like these settings allow Facebook to load normally:

security.tls.version.max=4
network.http.http3.enable=false

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

I've built a browser with a patch to verify if a specific change in neqo (QUIC/http3 support) caused it (SCONE support). This change could cause a server with a bug to fail (or a proxy like bitdefender likely uses).

Here's the build: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K_qzFw4BRZ2MyI3oN_B9NA/runs/0/artifacts/public/build/target.installer.exe
I would make sure you run it with a new profile, or clone your profile (when firefox isn't running) and rename it; i would go with a new profile unless the bug won't reproduce that way. "firefox.exe -P" will let you create and use a new profile.

First verify the bug still happens with this build. If it does, then in about:config flip network.http.http3.scone_indication to false, and try again. I would try normally first, and then try shift-reload, or clear the cache and retry, just to make sure we aren't being tricked by cached files.

Hopefully flipping the pref will make the problem go away. If so, it's almost certainly a bug in a proxy or the server, and we're pretty sure that Meta has fixed the SCONE bug they had some time ago, which would leave bitdefender. Even if disabled, it may well stay "in the loop" and continue to proxy connections, but just not enforce anything.

Thanks!!!!

Comment 11

[Tanek DK]

With Firefox 150.0 the network.http.http3.enable=false made Facebook load like normal in the Facebook Container extension, and no other extensions added at all.
Someone in r/Firefox asked what specific anti-virus people have installed, who still can reproduce this error, and for me that is Bitdefender Total Security.

Comment 12

[Tanek DK]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #10)

First verify the bug still happens with this build. If it does, then in about:config flip network.http.http3.scone_indication to false, and try again. I would try normally first, and then try shift-reload, or clear the cache and retry, just to make sure we aren't being tricked by cached files.

Bug verified in this Nightly build. Facebook will not load.
Flipping network.http.http3.scone_indication to false made Facebook load again, but this build doesn't have the Facebook Container extension.

Comment 13

[s3n0]

(In reply to Tanek DK from comment #11)

With Firefox 150.0 the network.http.http3.enable=false made Facebook load like normal in the Facebook Container extension, and no other extensions added at all.
Someone in r/Firefox asked what specific anti-virus people have installed, who still can reproduce this error, and for me that is Bitdefender Total Security.

Hi.

Same here.

I use Bitdefender (antivirus in a free edition). When I turn off all options in the Protection -> Online Threat Prevention, then Facebook starts working. So... in my case, on the path between the web browser and the Facebook server, there is this Bitdefender. But everything was working until now. It was only after updating to v150.0 that Facebook stopped working.

Comment 14

[Kershaw Chang [:kershaw]]

(In reply to s3n0 from comment #13)

(In reply to Tanek DK from comment #11)

With Firefox 150.0 the network.http.http3.enable=false made Facebook load like normal in the Facebook Container extension, and no other extensions added at all.
Someone in r/Firefox asked what specific anti-virus people have installed, who still can reproduce this error, and for me that is Bitdefender Total Security.

Hi.

Same here.

I use Bitdefender (antivirus in a free edition). When I turn off all options in the Protection -> Online Threat Prevention, then Facebook starts working. So... in my case, on the path between the web browser and the Facebook server, there is this Bitdefender. But everything was working until now. It was only after updating to v150.0 that Facebook stopped working.

Could you also take a look at comment #10 and use the test build to see if the issue still reproduces?
Thanks.

Comment 15

[s3n0]

Hi.

Of course... no problem...

I tried it... I downloaded from comment #10 and installed Firefox Nightly. I launched it.

1. I tried to open the website Facebook but it didn't load. It showed a loading and blank screen (see image 1 at https://imgur.com/a/3foLGEC) and nothing loaded.

2. Then I stopped (cancel) loading of the website Facebook, in Firefox Nightly.

3. I went to Bitdefender settings... and this time I tried disabling individual options one by one (in the "Protection -> Online Threat Prevention" menu). At each step, I stopped + changed the setting + let the Facebook.com website load again in Firefox Nightly. Facebook managed to load only when I disabled the second option in Bitdefender, called "Search advisor", see image 2 at https://imgur.com/a/3foLGEC .

4. If I re-enable the second option "Search advisor", then Facebook only works for about 10 minutes. Then the entire Facebook website refuses to load again. Apparently some security key is being created somewhere, which is only created for about 10 minutes... and that's why it only works for about 10 minutes. Of course... if I turn off the mentioned second option in Bitdefender again, Facebook starts working again.

I also tried disabling the other options in "Bitdefender > Protection > Online Threat Prevention" one by one (only one option at a time, while I always left the others enabled - as part of testing). The only option that allows the Facebook website to load, in this Firefox Nigthly, is the mentioned second option "Search advisor".

BTW, I'm an IT guy... with secondary or higher professional experiences. So feel free to ask if you need anything more (LOG - for example). You just have to describe to me how to get LOG. I don't know the web environment at all... unfortunately. As for programming, I'm currently more of a Python developer (github.com/s3n0) :-).

Comment 16

[Max Inden]

Attachment: Bug 2034178 - update to Neqo v0.24.2 in FF v150

https://github.com/mozilla/neqo/pull/3573

Comment 17

[Kershaw Chang [:kershaw]]

Hi, thanks a lot for your help.

If you disable the network.http.http3.scone_indication preference in the test build (from comment #10) and keep all Bitdefender settings at their default values, are you still able to reproduce the issue?

Comment 18

[Frederik Braun [:freddy]]

@s3n0, given your stated experience as an IT professional -- If you could produce a wireshark trace just for the facebook packets would be perfect.

Comment 19

[Valentin Gosu [:valentin]]

(In reply to Frederik Braun [:freddy] from comment #18)

@s3n0, given your stated experience as an IT professional -- If you could produce a wireshark trace just for the facebook packets would be perfect.

And ideally you could follow these steps to include the decryption keys too:
https://wiki.wireshark.org/TLS

Feel free to put the dumps in a google drive and send the link to [e-mail]

Comment 20

[[e-mail]]

For basic users like me - can you tell me where to go to change settings to lower security.tls.version.max to 3

Comment 21

[s3n0]

(In reply to [e-mail] from comment #20)

For basic users like me - can you tell me where to go to change settings to lower security.tls.version.max to 3

In the address bar of FF (as URL), type: about:config. Then find the necessary configuration item (just type it into the search box), which is written about here. And change its value from 4 to 3.

Comment 22

[Frederik Braun [:freddy]]

All user reports indicate issues for people with BitDefender Total Protection installed.

People who are affected can try and see if this helps:

• Go to Settings, Protections, Online Threat Prevention, Web Protections
• Find "Seach Advisor" and disable that.
• Make a post-it note to remember that you may want to re-enable that in a few weeks.

If disabling this BitDefender setting fixes this, it would greatly appreciated if people comment here.

Aside: The issue is not with Firefox configuration and there are some side-effects to downgrading our TLS connection security. Please do NOT go into the deep internal bits of Firefox.

Comment 23

[Max Inden]

Suspected root cause

An HTTP/3 feature (SCONE, draft-ietf-scone-protocol) is the likely culprit.

The SCONE connection parameter defaults to off (neqo #3492), but that flag only gates advertisement of the SCONE transport parameter. The Initial-packet padding still emits the SCONE indicator bytes (0xc8 0x13) unconditionally.

So every Firefox 150 client signals SCONE on the wire without advertising it. Suspected chain:

1. Facebook's edge sees the indicator and alters its response.
2. The QUIC handshake still completes (SCONE is on-path signaling, not part of the handshake), so H3→H2 fallback never triggers.
3. Bitdefender's HTTP/3 inspection then trips on the altered server-side frames.

Note that the padding contents are an endpoint's prerogative, QUIC endpoints are free to pad Initial packets with arbitrary bytes, and HTTP/3 inspectors are expected to tolerate that. The SCONE indicator is a legitimate padding choice. That said, pragmatically it's tripping Bitdefender here, so backing it out for now is the least disruptive mitigation.

@jesup's try build disabling the indicator (pref network.http.http3.scone_indication=false) fixes Facebook for at least one confirmed Bitdefender user.

Proposed fix

Gate the Initial padding on the existing scone_enabled(), so default config pads with zeros — identical to pre-SCONE wire behavior.

• neqo patch: mozilla/neqo#3573 (against the v0.24 branch; v0.24.2 release to follow)
• Firefox-side integration: phabricator D296306

Comment 24

[Max Inden]

New Test build

Users affected can test a Windows build that includes the above patches (no SCONE padding emitted):

https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RYSRkesLRpOWyS4ueYQ2Cg/runs/0/artifacts/public/build/target.installer.exe

Please report back whether Facebook loads normally with this build.

Comment 25

[s3n0]

(In reply to Kershaw Chang [:kershaw] from comment #17)

Hi, thanks a lot for your help.

If you disable the network.http.http3.scone_indication preference in the test build (from comment #10) and keep all Bitdefender settings at their default values, are you still able to reproduce the issue?

I disabled the configuration option you mentioned, i.e. I switched the value from True to False, in Firefox Nightly (test build). Firefox then loaded the Facebook website.

But not immediately after switching the configuration option. It took about 15-30 seconds for Facebook to start working, from the moment I switched the configuration option.

When I switched this option back to the original state, to True, Facebook then worked (I could load the Facebook.com website) for about 10 minutes. Then after 10 minutes, loading Facebook stopped working.

Comment 26

[s3n0]

(In reply to Max Inden from comment #24)

New Test build

Users affected can test a Windows build that includes the above patches (no SCONE padding emitted):

https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RYSRkesLRpOWyS4ueYQ2Cg/runs/0/artifacts/public/build/target.installer.exe

Please report back whether Facebook loads normally with this build.

Will there be any problem with the user profile created from the previous Firefox Nightly, from the previous test build ?
If I uninstall and install this new Firefox Nightly - test build ?
Or is it necessary to also delete the profile from Firefox Nightly, after uninstalling it ?

Comment 27

[Jan Brasna [:janbrasna]]

There should be nothing in the profiles affecting the connection results per se — so feel free to install the new build without any extra cleanup of the previous tests, and just see how the results are with your security software features enabled fully. Thank you!

Comment 28

[s3n0]

Hi again.

I uninstalled the old Firefox Nightly test build. To be sure, I also deleted its user profile. Then I installed this newer test build from comment 24.

Facebook loaded fine. No problem. I didn't have to disable anything in my Bitdefender settings.

To be sure, I tried waiting 10-20 minutes and reloading the Facebook website again and this time it always worked fine.

I only waited in case some encryption key, probably located somewhere on Facebook's servers, would be deleted within 10 minutes. So that I could be sure that the key was deleted (if any existed on Facebook's servers). Although... this fact is probably irrelevant now, since I actually deleted the entire previous profile - from the previous Firefox Nightly :-D. Just to be sure that the new Firefox Nightly test build is clean. I don't know. I am not an expert on web interfaces or web traffic :-).

BTW, I am using Bitdefender Antivirus Free edition.

Comment 29

[Ciprian Georgiu, Desktop QA]

We were able to reproduce the issue using Firefox 150.0 on Windows 11 with Bitdefender Total Security installed. However, I could only reproduce it on a single Firefox profile where Facebook failed to load properly. I wasn’t able to determine why the issue did not occur on other clean profiles.

I installed the build from bug 2034178 comment 24, and opened it using the same affected profile. The issue appears to be resolved, no problems were encountered when accessing facebook.com with the fixed build.

Comment 30

[Max Inden]

Attachment: Bug 2034178 - update to Neqo v0.26.1 in FF v151

Update to Neqo v0.26.1 more specifically include the following fix.

• fix: gate SCONE Initial padding on connection parameter

See details in:

• https://github.com/mozilla/neqo/releases/tag/v0.26.1
• https://github.com/mozilla/neqo/pull/3574
• https://github.com/mozilla/neqo/pull/3573

Comment 31

[Phabricator Automation]

firefox-release Uplift Approval Request

• User impact if declined/Reason for urgency: Firefox v150 Windows users with specific Bitdefender versions will not be able to access facebook.com. See Bug for details.
• Code covered by automated testing?: yes
• Fix verified in Nightly?: no
• Needs manual QE testing?: yes
• Steps to reproduce for manual QE testing: See previous QE testing:

https://bugzilla.mozilla.org/show_bug.cgi?id=2034178#c29

• Risk associated with taking this patch: low
• Explanation of risk level: Reverts to the Firefox v149 behavior.
• String changes made/needed?: no
• Is Android affected?: yes

Comment 32

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/e1594fbc174e
https://hg.mozilla.org/releases/mozilla-release/rev/4c6427b9bda0
update to Neqo v0.24.2 in FF v150

Comment 33

[dbert721]

(In reply to Max Inden from comment #24)

New Test build

Users affected can test a Windows build that includes the above patches (no SCONE padding emitted):

https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RYSRkesLRpOWyS4ueYQ2Cg/runs/0/artifacts/public/build/target.installer.exe

Please report back whether Facebook loads normally with this build.

Confirmed working normally on Windows 11 with BitDefender Total Security running.

Comment 34

[Ernesto Sánchez Benitez]

Hi

Confirmed working again with BitDefender Total Security running.

Comment 35

[s3n0]

Hi.

Do you need the Wireshark debug LOG ? Or is it no longer needed ?

It would take me some time... but if you want... I can try it - get the network log - as mentioned in comment 18 (Wireshark usage) and in comment 19 (+ SSL decryption keys). I would have to download Wireshark + install + run Firefox 150.0 there.

Anyway, anyone with Windows 11 + Wireshark + Firefox 150.0 can also do this. Even developers directly. But if it would help... then I will generate the debug log... for the good of the matter .-).

Comment 36

[Max Inden]

Thanks @s3n0. @valentin has been able to reproduce since, thus no pcap needed. Appreciate the help!

(In reply to Ernesto Sánchez Benitez from comment #34)

Hi

Confirmed working again with BitDefender Total Security running.

@Ernesto can you clarify whether it is "working again" with the above Firefox test build, or due to a BitDefender update?

Comment 37

[Ernesto Sánchez Benitez]

(In reply to Max Inden from comment #36)

Thanks @s3n0. @valentin has been able to reproduce since, thus no pcap needed. Appreciate the help!

(In reply to Ernesto Sánchez Benitez from comment #34)

Hi

Confirmed working again with BitDefender Total Security running.

@Ernesto can you clarify whether it is "working again" with the above Firefox test build, or due to a BitDefender update?

I'm using the Firefox Nightly version shared above in a post and the updated Bitdefender: Total Security, and Facebook is working again.

Comment 38

[Ciprian Georgiu, Desktop QA]

Marking this verifed as fixed on Firefox 150.0.1 (20260424153551) since I cannot repro it anymore with Win 11.

Comment 39

[Tanek DK]

(In reply to Ciprian Georgiu, Desktop QA from comment #38)

Marking this verifed as fixed on Firefox 150.0.1 (20260424153551) since I cannot repro it anymore with Win 11.

If this means release, I'm still stuck on 150.0.

Comment 40

[s3n0]

Because version 150.0.1 hasn't been released yet... I think. We have to wait.

https://www.firefox.com/en-US/releases/

Comment 41

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/120caa2496f4
https://hg.mozilla.org/integration/autoland/rev/77011063307f
update to Neqo v0.26.1 in FF v151 & v152 r=valentin

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/296d7935e0ba
https://hg.mozilla.org/integration/autoland/rev/986be31955da
Revert "Bug 2034178 - update to Neqo v0.26.1 in FF v151 & v152 r=valentin" for causing rust vendor failures

Comment 43

[Sandor Molnar[:smolnar]]

Backed out for causing rust vendor failures

Backout link: https://hg.mozilla.org/integration/autoland/rev/986be31955da416c3c0b10af48ea9da82af3446d

Push with failures

Failures log ->Errors occurred; new rust crates were not vendored

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/050516663ddc
https://hg.mozilla.org/integration/autoland/rev/991c6ab3bf3a
update to Neqo v0.26.1 in FF v151 r=valentin

Comment 45

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined/Reason for urgency: Incident response issue to fox loading of certain sites
• Code covered by automated testing?: unknown
• Fix verified in Nightly?: no
• Needs manual QE testing?: yes
• Steps to reproduce for manual QE testing: See https://bugzilla.mozilla.org/show_bug.cgi?id=2034178#c29
• Risk associated with taking this patch: low
• Explanation of risk level: Incident Response
• String changes made/needed?: None
• Is Android affected?: unknown

Comment 46

[Dianna Smith [:diannaS]]

Attachment: Bug 2034178 - update to Neqo v0.26.1 in FF v151

Update to Neqo v0.26.1 more specifically include the following fix.

• fix: gate SCONE Initial padding on connection parameter

See details in:

• https://github.com/mozilla/neqo/releases/tag/v0.26.1
• https://github.com/mozilla/neqo/pull/3574
• https://github.com/mozilla/neqo/pull/3573

Original Revision: https://phabricator.services.mozilla.com/D296371

Comment 47

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/991c6ab3bf3a

Comment 48

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/f7a264eac768
https://hg.mozilla.org/releases/mozilla-beta/rev/6e767352c294

Comment 49

[Jan Brasna [:janbrasna]]

For anyone affected, v150.0.1 is now out: https://www.firefox.com/firefox/150.0.1/releasenotes/ — please update and try with all the preferences switched back to their original state. Thanks everyone! 🚀

Comment 50

[Ciprian Georgiu, Desktop QA]

I also verified as fixed this bug on latest Nightly 152.0a1 and Beta 151.0b5 under Win 11.