1882839 - HTTPS-Only Mode fails to detect https websites

Status: RESOLVED WORKSFORME

(Core :: DOM: Security, defect)

Description

[singingotter]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:122.0) Gecko/20100101 Firefox/122.0

Steps to reproduce:

visiting a website via searchbar, link or typed address often results in the block and message "no https site available"

Actual results:

Clicking thru (ignoring the warnings) to get to the site it DOES have https://

Expected results:

HTTPS-only mode should detect the site does have https:// and not obstruct te address or issue a warning.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Security' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Daniel Veditz [:dveditz]]

I've visited lots of websites and had https-only work fine; are there any more details you could give?

1. "often results in": Do you mean that it works on some sites and not others, or do you mean that it's inconsistent on the same site?
2. Could you give us a few example sites that aren't working for you?

Comment 3

[Daniel Veditz [:dveditz]]

3. When this happens, does it often end up on a slightly different site than you typed in? For instance

• you type in "somesite.example" and get the error that https://somesite.example doesn't exist
• you say "fine, load the http: site"
• you end up on https://www.somesite.example

If that's the scenario it's quite possible https://somesite.example really doesn't exist, but the http: version does a redirect from "somesite.example" to "www.somesite.example", and then we're able to upgrade "www.somesite.example" because the https version of that one does exist.

We could probably special-case "try again with www. on the front" if that's the only kind of site you see having this problem. But "www." is just an old dot-com era tradition with no special significance to internet standards. It's always a guess whether that might work or even take you to an unrelated site.

Comment 4

[Daniel Veditz [:dveditz]]

We have an alternate implementation that we're trying called "https first" instead of "https only", that does try alternatives like this before it gives up. It is enabled in Private Browsing mode at the moment so you could try that on a few of your problem sites and see if that handles it better. If it does then you'll be happy when we make https first available for regular browsing.

Comment 5

[singingotter]

Attachment: Screenshot from 2024-03-05 09-56-04.png

have uploaded one screenshot example. This is for the local school district, a frequently visited site that has https but gets flagged as not.

Comment 6

[Thorin [:thorin]]

for me: both www.collegeboard.org and in a new session with everything sanitized collegeboard.org both open https://www.collegeboard.org/ without any interstitials - nightly125 (windows 11)

Comment 7

[singingotter]

Attachment: website with https but flagged as not

here's another one. I didn't catch the 'https unavailable' flag screenshot but this is the website address bar that clearly shows https.

Comment 8

[Daniel Veditz [:dveditz]]

OK, another guess (since these sites work fine for me): do you have a slow or inconsistent internet connection? Maybe you connect through a security proxy that has to create new MITM certificates each session? This might be a traffic scanning feature of an anti-virus product running on your own machine, or in a corporate or academic environment it might be part of a corporate internet firewall product.

People won't put up with waiting for the full connection timeout before getting the error page, so after a little bit we send an HTTP connection request and if that returns successfully and we still haven't heard back from the HTTPS connection then we show the error saying we couldn't reach that site. If your connection to HTTPS sites has additional delays you could try turning off your anti-virus traffic monitoring feature to see if that helps, or you could increase the time we wait before giving up.

For the latter you'd want to open the "about:config" page, and use the search box at the top to look for dom.security.https_only_fire_http_request_background_timer_ms. It should currently have the default value 3000 (ms, or 3 seconds). Change that to 5000 or 4000 and let us know if the problem goes away.

Comment 9

[singingotter]

The internet connection is solid (cable comcast), streaming etc works fine. Connection is via d-link dual channel router (usually wifi) with firewall and Opendns DNS, reserved IPs for regular devices, otherwise autoassigned. Laptops/phones have Protonvpn, with filtering for malicious sites, similar with Firefox browser , enhanced tracking protection, privacy badger, Disonnect. I don't know about MITM certificates.
Most machines are Linux or android.
I've tried changing each of the above security settings but none seem to make any difference.
One thing I have noticed is that, after clicking through to a site marked 'no https' that does have https, visiting that site again in the same session but with a new tab or window does not bring up the warning flag.
Is this because firefox is learning? If I visit the site on another day it may or may not generate the flag.
Mostly I'm using the same laptop, same browser awoken from suspend without quitting the browser, so I expected the permissions to persist over sessions but this is not the case. ie one day it will flag once then not again, but next day it might flag again.
This is why I suspect the https-only extention is faulty, not any of the other parts of the system.

Comment 10

[singingotter]

Attachment: Screenshot example htttps2024-03-08 08-33-07.png

Comment 11

[singingotter]

Attachment: another example 2024-03-08 08-18-02.png

Comment 12

[singingotter]

Attachment: https site flagged as not 2024-03-07 15-00-36.png

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:freddy, could you have a look please?

For more information, please visit BugBot documentation.

Comment 14

[Frederik Braun [:freddy]]

Sorry, singingotter. I'm not sure what could be different for your setup and ours.
I tried again and can not reproduce.

Did you try typical Firefox troubleshooting steps - use a different profile, try again without add-ons etc.?
If it reproduces for you and without extensions, then I think we could try capture a network log for you that you would then send to us. It might. contain sensitive information and it's not something I'd suggest you send here in public. So let's start with the troubleshooting steps from above.

(Assigning a low severity until we can reproduce)

Comment 15

[Daniel Veditz [:dveditz]]

ni? myself to answer comment 9

Comment 16

[Raphael]

I checked the website http://www.nahscollege.org and found that it has an incorrect SSL certificate. Another website, https://www.collegeboard.org/, displays correctly.

Could the error have occurred due to incorrect SSL settings on the website?

Comment 17

[Daniel Veditz [:dveditz]]

About comment 7: There is no such site as httpS://atlantapublicschools.us— If you typed "atlantapublicschools.us" or "http://atlantapublicschools.us" then the HTTPS-only warning would be correct. If you allow the site to continue to http then http://atlantapublicschools.us redirects to http://www.atlantapublicschools.us, and since there IS a secure version of that site (as seen in your screenshot) then it can upgrade to https. If you typed the full "www.atlantapublicschools.org" and got the interstitial then I still don't have an answer.

comment 10: likewise, https://www.atlanta.k12.ga.us does not exist and the HTTPS-only page is correct. If you connect insecurely it redirects to http://www.atlantapublicschools.us which does have an https site so it "works" while that exception is remembered.

comment 11: Can't explain this one. I don't get the error, and the site does exist on a secure connection. Maybe they upgraded the site?

comment 12: https://nahscollege.org/ does not exist; the error page is correct. If you continue through it redirects to www.nahscollege.org, and gets another HTTPS-only page because a secure version of that doesn't exist either. Only insecure http://www.nahscollege.org/ exists.

One thing I have noticed is that, after clicking through to a site marked 'no https' that does have https, visiting that site again in the same session but with a new tab or window does not bring up the warning flag.

Choosing to continue to the HTTP site anyway disables HTTPS-Only for that site temporarily. So your other windows are silently connecting to "atlantapublicschools.us" insecurely, then getting redirected to the secure "www." site. You can see this exception by clicking the lock icon, and you can change "Off Temporarily" to "On" to get HTTPS-only behavior for that site again, or you can turn if "Off" to permanently stop Firefox bothering you about that site (but you will briefly be connected insecurely before getting to the real site). You can also go into the HTTPS-Only section of Settings and "Manage Exceptions..." for many sites at once.

Mostly I'm using the same laptop, same browser awoken from suspend without quitting the browser, so I expected the permissions to persist over sessions but this is not the case. ie one day it will flag once then not again, but next day it might flag again.

I can't explain that. temporary permissions are stored EXPIRE_SESSION and as far as I can tell are kept in memory until you restart the browser.

As far as I can tell these examples show HTTPS-Only mode working correctly, except for https://www.tishmurtha.co.uk/. Today that site shouldn't get an error. Maybe the site added https in the the past 6 months? If it has always had it then I have no explanation, but also no clues on what to fix if Firefox was handling it badly back then.

HTTPS-Only mode is very strict and unforgiving, inspired by a tool the EFF designed for people in oppressive regimes who couldn't afford to leak anything about their browsing. Privacy activists love it, but we've discovered it's strictness is too annoying for most people. That's why we are working on HTTPS-first instead, which you can try out in Private Browsing mode. You'll have to turn off HTTPS-only mode because otherwise you get the stricter behavior. Eventually HTTPS-First will be available in regular browsing too.