1799457 - https is being forced even though "Don’t enable HTTPS-Only Mode" is selected (new bug).

Status: RESOLVED INVALID

(Firefox :: Address Bar, defect, P3)

Description

[donald.topsecret]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

Tried to access:

http://micropython.org/webrepl/.....

Actual results:

Firefox replaced the http/ with https/

Expected results:

Firefox should have left the http/ alone. I have "Don’t enable HTTPS-Only Mode" selected in the Privacy options. WebREPL (which I use to talk to a local Esp32 microprocessor) won't work over https. This bug has only appeared since about 4 November 2022, a few days ago. The last Firefox update was:

Details:
Firefox 106.0.5 (20221104133228)
Installed on: 5 November 2022 at 11:10:12
Status: The Update was successfully installed

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Security' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Daniel Veditz [:dveditz]]

Are you using Private Browsing, or have you set the preferences to "Never remember history"? If so then this is the effects of "Http-first" mode. You can either use normal browsing for that insecure site, or you can turn off the preference dom.security.https_first_pbm

Comment 3

[donald.topsecret]

Thanks, I think I solved it. I deleted all my "Cached Web Content" (but not "Cookies and Site Data") and it now works.

In answer to your queries:

1. Are you using Private Browsing? No

2. Have you set the preferences to "Never remember history"? No

Thanks for your help - it made me explore the Settings and try a few things, including deleting the Cached Web Content. However, I note I have received a new Update to Firefox as of 16 Novemeber (Firefox 107.0 (20221110173214) and I do wonder if the fix was included in this, and by clearing the Cache, I cleared the HTTPS entries which the cache produced. I don't know ? ?

Comment 5

[Daniel Veditz [:dveditz]]

In bug 1806493 the reporter says the problem is back.

Did you try turning off the preference dom.security.https_first_pbm just to be sure that wasn't the problem?

Other reasons for changing schemes:

• the site redirected you from http: to https: in the past and we cached the 301 redirect. This would explain why clearing the cache made it work. You could try clearing the cache again as a test.
• You visited the site at a https: url before and it served a Strict-Transport-Security header (which means "always connect using https"). This value is stored in your profile but wouldn't be cleared by clearing the cache. But they do have an expiration date so maybe it was coincidental timing?
• the site is in the strict-transport-security pre-load list. I think we can dismiss this one because then it would never have worked for you

Comment 6

[donald.topsecret]

Thanks for re-opening this bug, and apologies for the delay (Christmas etc).

I have just cleared my Cached Web Content (but not Cookies and Site Data) and it works again. No I didn't set "dom.security.https_first_pbm" to False, but I now have. However the problem has just returned. I re-started Firefdox as well.

I very much doubt I visited the REPL url using https, but it works via Mictropython.org (the url is http://micropython.org/webrepl/?# followed by my internal IP address for the ESP32 and a port id). I use the Micropython website quite often, using https, so that may have something to do with it.

I tried to use http on other websites (old ones) but they don't like it any more. I found I could use http on another in-house server though, without it being changed to https.

I am no expert on the internal workings of browers, by the way.

Comment 7

[Frederik Braun [:freddy]]

Funny. I could now reproduce the issue somewhat clearly.

Steps to repeat:

1. Go to http://micropython.org/webrepl
2. Change the address bar to go to the HTTPS version
3. Open a couple of tabs with the HTTPS site and open it a couple of times. Use the address bar for this.
4. Close all the open HTTPS tabs. Open a new tab and try to to back to the HTTP version using the address bar.

What should happen:

• Firefox should go back to HTTP, as instructed through the address bar.

What actually happens:

• Firefox prefers the HTTPS version.

I feel like this might be an address bar thing too?
Needs more investigation.

Comment 8

[Frederik Braun [:freddy]]

This must happen way above our networking/https stack. I think this could be address bar related.
In fact, it might just be our frecency algorithm kicking in.

Normally, I suppose people would be able to "teach" the address bar that one page is undesired by just going to the preferred variant more often.
However, that's tricky with this issue here where going to the preferred variant becomes almost impossible?

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:adw, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 10

[Andy]

Just for info, as of 115.0.3 it's still very easy to be caught out by the issue mentioned at https://bugzilla.mozilla.org/show_bug.cgi?id=1799457#c2 above - I just have been.

Essentially, in about:preferences#searchResults when I search for "https" I'll see "HTTPS-Only Mode" (and that I have that not enabled). I won't see any references to private windows or "Https-first" mode there at all. The "learn more" page https://support.mozilla.org/en-US/kb/https-only-prefs?as=u&utm_source=inproduct# also does not mention "https-first" or "dom.security.https_first_pbm".

Regardless of any code issues above, there's clearly an issue with the documentation when there is no mention that Firefox will "seemingly randomly"** change http connections to https ones, despite the UI suggesting ("Don’t enable HTTPS-Only Mode") that it won't.

** it's not random of course - the window must be a private one and the site must support https.

Comment 11

[Mandy Cheang [:mcheang][she/her]]

I was able to reproduce the problem from Comment 7. When I visited http://micropython.org/webrepl once, then visited the HTTPS version multiple times, I cannot go back to the http version again. When typing http://micropython.org/webrepl and pressing enter in the address bar, it now always goes to https://micropython.org/webrepl. Also to clarify, it's not recommending the HTTPS result in the result menu, so I don't think it has to due with frecency.

This problem needs further investigation. Perhaps investigating in _getURIFixupInfo

Comment 12

[Marco Bonardo [:mak]]

Afaict, it's SiteSecurityState (HSTS) kicking in and forcing micropython.org to load as https. Indeed I see and entry for it in SiteSecurityServiceState.bin.
The urlbar is asking to load http, but network obeys HSTS.

If I Forget About Site on any micropython.org url, then the entry in SiteSecurityServiceState is removed and I can again load the http version.

It looks like a server misconfiguration to me?

Comment 13

[Andy]

Just for completeness re my comment above at https://bugzilla.mozilla.org/show_bug.cgi?id=1799457#c10 , that wasn't related to HSTS. The site I noticed it on was http://map.atownsend.org.uk/maps/ (in a private browser). With dom.security.https_first_pbm as the default (true) it redirects to https://map.atownsend.org.uk/maps/ , but with dom.security.https_first_pbm set to false it does not - it honours what I type in.

Comment 14

[Frederik Braun [:freddy]]

Please see comment 13 from Andy.

Comment 15

[Marco Bonardo [:mak]]

I'm sorry but I'm still confused.

The original bug here from comment 0 is on http://micropython.org, and that bug is invalid, as I said in comment 12

Then there is a broader discussion around discoverability of dom.security.https_first_pbm, I don't see how that is related to the Address Bar.

I'm going to mark this as invalid, and the discussion around dom.security.https_first_pbm, that is surely valid feedback, should be filed as a new bug in DOM Security.
Andy, could you please file your feedback about discoverability of private browsing mode https first as a new bug in DOM Security?

Comment 16

[Andy]

Andy, could you please file your feedback about discoverability of private browsing mode https first as a new bug in DOM Security?

Done, https://bugzilla.mozilla.org/show_bug.cgi?id=1859850 , although I can't see a way to "file that in DOM Security" - presumably someone will need to explicitly categorise it?