1528603 - The user should be able to view and clear CORS cache

Status: NEW

(Core :: Networking, defect, P3)

Description

[Danilo B]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

I work on a webapp where we recently split up the front- and backend to separate domains.

The new frontend was already deployed, so I tried to log in, which generated a CORS preflight request that failed because CORS hasn't been set up yet on the old backend.

A day later we have deployed the new backend with proper CORS headers. But my Firefox still cannot log in. In the debug console, I'm getting the following warning:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://<redacted>/api/token/. (Reason: CORS request did not succeed).

The problem is that no CORS request is sent. This is probably a cached negative CORS reply.

From research with search engines, I found several questions on how to clear the CORS cache, but no real answers:

• https://stackoverflow.com/questions/28539849/clearing-the-cached-preflight-response-on-firefox
• https://stackoverflow.com/questions/10815842/how-to-check-content-of-preflight-result-cache-in-firefox

One of the answers suggests closing and re-opening Firefox, but that does not help. I also tried clearing the cache, which removed all my history, cookies and stored data but did not clear the CORS cache. Creating a new Firefox profile does help.

There should be an interface for the user to view and clear the CORS cache. This could be added to the developer tools, or to the "Clear Data" UI.

Comment 1

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Danilo B from comment #0)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

I work on a webapp where we recently split up the front- and backend to separate domains.

The new frontend was already deployed, so I tried to log in, which generated a CORS preflight request that failed because CORS hasn't been set up yet on the old backend.

A day later we have deployed the new backend with proper CORS headers. But my Firefox still cannot log in. In the debug console, I'm getting the following warning:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://<redacted>/api/token/. (Reason: CORS request did not succeed).

The problem is that no CORS request is sent. This is probably a cached negative CORS reply.

From research with search engines, I found several questions on how to clear the CORS cache, but no real answers:

• https://stackoverflow.com/questions/28539849/clearing-the-cached-preflight-response-on-firefox
• https://stackoverflow.com/questions/10815842/how-to-check-content-of-preflight-result-cache-in-firefox

One of the answers suggests closing and re-opening Firefox, but that does not help. I also tried clearing the cache, which removed all my history, cookies and stored data but did not clear the CORS cache. Creating a new Firefox profile does help.

There should be an interface for the user to view and clear the CORS cache. This could be added to the developer tools, or to the "Clear Data" UI.

This sounds a UI feature request to me, moving to Firefox:Preferences.

Comment 2

[Mike Conley (:mconley) (:⚙️)]

I'm not convinced this is something we'd want to expose to normal users in about:preferences. This is something more developer-centric, so I'm going to see if the DevTools team working on the Networking panel wants to look at this.

Comment 3

[Danilo B]

Putting it in the developer tools would be appropriate.

On the other hand, I would also expect the "delete data / cache" option in the preferences to actually remove all cached data. (Couldn't this actually be a privacy issue if CORS domains are cached without an option to remove them?)

Comment 4

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Danilo: do you have any tips (from the user perspective) how the UI/UX that allows to view and clear the CORS cache should look like/behave?

Honza

Comment 5

[Danilo B]

@Jan: From a user point of view, my understanding would be that a "CORS cache" is part of "website data". Therefore I would search for an option in the settings under "Privacy & Security".

In that section, I can find an entry called "Cookies and Site Data".

https://tmp.dbrgn.ch/screenshots/20190312114126-9qqtt0i5.png

There's an option called "Clear Data":

https://tmp.dbrgn.ch/screenshots/20190312114237-10awgai0.png

I would definitely expect that either checking "Site Data" or "Cached Web Content" would include the CORS cache.

Doing another CORS lookup is only a question of performance, not of security. I don't see a reason not to treat that cache the same as other website caches and to clear it together with other caches.

---

From a developer point of view it would be amazing if I could see and modify the CORS cache entries in the developer toolbar under "Storage". But that's probably another issue, the primary goal should be to have a way to clear that cache at all.

Comment 6

[:Harald Kirschner :digitarald]

I agree with Disable Cache in DevTools to disable as many caches as possible.

Not clearing the CORS cache when Clearing data seems not great as the CORS cache can include private data and can probably be found in the profile folder? ni? Arthur to provide input from the privacy perspective.

Comment 7

[Lennart Grahl [:lgrahl]]

Sorry for bumping this one but the current situation is really annoying.

Comment 8

[Arthur Edelstein [:arthur]]

I agree we should be clearing CORS data, probably when the HTTP cache is cleared. Not sure if that is already the case. This seems like an issue not just for DevTools but for the cache itself.

Comment 9

[Michal Novotny [:michal]]

(In reply to :Harald Kirschner :digitarald from comment #6)

Not clearing the CORS cache when Clearing data seems not great as the CORS cache can include private data and can probably be found in the profile folder? ni? Arthur to provide input from the privacy perspective.

nsPreflightCache (https://searchfox.org/mozilla-central/rev/1097f59d0bc17f6f8f805325c2f607e60cf0d26b/netwerk/protocol/http/nsCORSListenerProxy.cpp#139) is in memory only cache, so no data can be found in profile folder. I'm not sure why restarting Firefox doesn't help, but it definitely clears the cache.

Comment 10

[Honza Bambas (:mayhemer)]

Michal, would you find time to verify this bug actually exists?

Comment 11

[Danilo B]

I don't know about the nsPreflightCache, but I have a page where I cannot log in since multiple months using my profile:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://[redacted]/api/token/. (Reason: CORS request did not succeed).

This is a POST request. According to the network tab, no OPTIONS preflight request is being done to verify whether the CORS request is actually valid or not, that's why I'm assuming a cached negative response. It works in an anonymous window (which doesn't share the caches I assume) and in other browsers. (In the anonymous window, I can see that the OPTIONS preflight requests are being done properly.)

Comment 12

[Honza Bambas (:mayhemer)]

It would be interesting to examine cache + anonymous cache (anonymous is not private browsing!!) if there is some unexpiring response to OPTIONS. But I honestly doubt that, and I also think we are not caching them. Also, cached OPTIONS would be visible in devtools as well.

Maybe a log would help: https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Comment 13

[Danilo B]

This happened to me again. It's a simple application that does a fetch call (GET request):

  const res = await fetch(`${API_URL}reservations/`, {
      mode: 'cors',
      cache: 'no-cache',
      headers: {
          'Authorization': `Token ${API_TOKEN}`,
          'Accept': 'application/json',
      },
  });

Yesterday I did some local testing with the network connection completely turned off, to ensure that the web application handles that correctly. This of course means that the preflight request will fail before the actual request is even attempted. That is expected and normal. However, once the connection is back, further preflight requests should not be prevented.

Today I cannot successfully query the API from localhost. The debug console contains this error message:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://reservations.coredump.ch/api/v1/reservations/. (Reason: CORS request did not succeed).

No request is actually being sent. I can open the network tab, check "disable cache", press ctrl+f5 as many times as I want, but the request still fails even though the network connection is present and the API server responds properly to CORS requests. Running the application on another port does not seem to help either. Opening the same page (http://localhost:8080) in Chrome or with Firefox Nightly (which uses a separate profile) works without any issues though. Opening the production website (https://reservations.coredump.ch/) which queries the same endpoint works as well.

This probably means that somewhere in my user profile, that negative CORS response is being cached and will not be cleared. Unfortunately I cannot reliably reproduce this issue with a reproducer script.

I also went to "Settings > Privacy & Security > Clear Data" and deleted "Cached Web Content". That did not resolve the issue either.

It would be interesting to examine cache + anonymous cache (anonymous is not private browsing!!) if there is some unexpiring response to OPTIONS.

Thanks, I didn't know about:cache yet. After clearing the "Cached Web Content", Anonymous cache is empty and the regular disk cache does not contain any entry for the API I'm querying. Yet, the request still fails with a CORS warning.

Maybe a log would help: https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Unfortunately I have >100 tabs open, a 2s log results in a logfile that's over 50k lines long, so that won't be very useful. Maybe I can somehow export the open tabs, close them and then attempt to create a logfile again...

Comment 14

[gattid]

I'm sorry but what is the status of this bug/feature request?

I have a cached negative preflight CORS request that persists between starts, multiple days and clearing data & cache. The website works in a private firefox window and any other browser. Firefox reports in the console that a negative preflight requests probihits the request, however no preflight request has been recorded in the network tab of the dev tool (not even as cached).

It's currently unclear to me why the negative preflight request still persists and no new request is being attempted by Firefox, even after clearing data & cache, across reboots and across days (it has come to my ears that Firefox caches a CORS request for maximum 24h, but I have yet to verify that - it's not important for this now).

Comment 15

[Novack]

Also having this issue, quite annoying! :)
I also have what seems like sorta oldie cached cors entries!

Workarounds:

1. Incognito Mode.
2. Disable Cache from Dev Console.

Comment 16

[Valentin Gosu [:valentin] (he/him)]

Hi Christoph,
Do you have any bandwidth to deal with this?
We've made it a P3 but maybe you think it should be higher?

Comment 17

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #16)

Hi Christoph,
Do you have any bandwidth to deal with this?
We've made it a P3 but maybe you think it should be higher?

Unfortunately I don't have the bandwidth to take this on at the moment. I guess it has to remain a P3 for now. However is working on this, I agree with Arthur and we should clear the CORS cache probably when the HTTP cache is cleared. as well.

Comment 18

[rhardy]

This is a very painful bug if it is in fact was is causing my current grief. I'm posting my experience here in the hopes of raising the priority of this bug.
There was something odd with Quick Books Online which resulted in the login page for the service not loading, instead showing a blank screen.
I have a screen shot but it doesn't add much.
It can be found here: https://www.intuit.com/ca/sign-in/ and the login page is here: https://c35.qbo.intuit.com/qbo35/login?locale=en_CA&sc=seq_intuit_qbo_click_signin

To my dismay the vendor initially pushed users to switch to a browser that worked (i.e. anything other than Firefox) which I really did not want to do.
Initially this only happened in Firefox, was OS, host, network, ISP and version independent. It happened I believe in three release versions: 89, 89.0.1, 89.0.2 and 90b12 from firefox-next PPA and on 3 flavours of Ubuntu Linux and the latest Windows 10 Pro release. Since Intuit "fixed" whatever they "fixed" it ONLY happens on my primary workstation but for the life of me I cannot get rid of the CORS error.

I debugged down the original problem to a single Browser console error i.e. when this CORS error appeared I did not get a login prompt:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://edge.experimentation.intuit.com/api/v2/assignments/businessUnits/INTUITPLATFORM/applications/IUX/users/a837724b-f0b5-4bce-ae6a-8d54c1e95306. (Reason: CORS request did not succeed).

Now the problem is fixed everywhere EXCEPT where I need it most. From the rather insane behaviour this clearly is NOT a in memory only cache.
The problem is persisting across all my attempts to get rid of it:

• CTRL reload of web page
• Cache clear
• Disable Enhanced Tracking Protections
• Attempting to disable CORS all together (I undid all of this after debugging.)
• Various extensions related to CORS debugging and unsafe extensions which cause all CORS requests to succeed (now removed)
• Disable all extensions
• Private window
• New profile
• Uninstall and Reinstall latest release Firefox
• 2 upgrades of Firefox from 89 -> 89.0.1 and 89.0.1 -> 89.0.2
• Upgrade Firefox to 90b12 from firefox-next PPA (I down graded again when it did not help.)

I don't have an easy way of testing a nightly build. Hopefully the firefox-next beta was good enough.
This should clearly be way past what is reasonable to expect from an end user.
There is no way a cache should exist outside of a profile and clearly some how there is one. This is making very little sense at this point.
Anything which would help me clear this would be appreciated.

Comment 19

[rhardy]

Attachment: Screenshot showing effect of persistent cached CORS error that isn't there anymore.

This is a screen shot showing the effect of a persistent CORS error which will not go away on a single host with four different versions of Firefox including the latest firefox-next PPA beta despite extreme efforts.

Comment 20

[rhardy]

While this isn't a crash, what seems to be persistent CORS cache corruption, which should be impossible. It is causing users to migrate to other browsers. Is there some way to get this bug taken seriously?

Emma is there some way to get this bug out of the unconfirmed P3 purgatory it has been stuck in for years?
Even if someone would tell me which file to delete to force a purge of the CORS cache data that would be huge help.
I believe my CORS cache has multiple corrupted entry in it which I cannot even see at this point.

Comment 21

[Emma's Evil Twin (don't assign me bugs or needinfo, I'm just here for testing.)]

@rhardy, I'm not currently involved with the Mozilla project so I'll refer this to the component's triage owner. Thanks.

Comment 22

[Emma H 🧞‍♀️☕ (she/they) Pacific Time]

Apologies, I had logged in to answer that from my old testing account by mistake.

Comment 23

[rhardy]

Sorry Emma, I thought I found you in a mozilla.org wiki somewhere as an escalation point for Crash bug not getting attention.
Of course I can't find it now...

Comment 24

[rhardy]

It was this document which I'm guessing hasn't been updated since 2016: https://wiki.mozilla.org/QA/Bug_Triage_Guidelines

Comment 25

[Emma H 🧞‍♀️☕ (she/they) Pacific Time]

(In reply to rhardy from comment #24)

It was this document which I'm guessing hasn't been updated since 2016: https://wiki.mozilla.org/QA/Bug_Triage_Guidelines

Thanks. I got that page in the wiki updated.

Comment 26

[(inactive) Nihanth Subramanya [:nhnt11]]

I spent a second trying to reproduce this but didn't make any headway. I could take a closer look when I'm back from PTO, but in the meantime, bumping it back into our triage queue.

Comment 27

[rhardy]

Thanks for trying. I wish we could leverage the system which is consistently failing so as to get enough detail to find the problem.
Having reproducible problems is ideal but when they only appear from random usage of a product that doesn't help much.
It keeps happening every day for me but only on the one system.
The accounting vendor closed my ticket instead of trying to figure out why Firefox is misbehaving.
Their solution was to use Google's spyware with my financial data which really isn't good enough.
I'm up to Firefox 90.0 and the problem has followed me through every upgrade.

Comment 28

[Valentin Gosu [:valentin] (he/him)]

An easy way to disable the CORS preflight cache is to set devtools.cache.disabled to true in about:config.
But since you report that a new profile doesn't work, I doubt this will fix much.

If you can record a HTTP log that might be useful: https://firefox-source-docs.mozilla.org/networking/http/logging.html
You can put it in a ZIP archive to make it smaller - since it's only text it compresses well.
Thanks!

Comment 29

[rhardy]

Unfortunately I seem to have lost my ability to assist with this.
Out of the blue, on the system which had the persistent CORS error for months, Firefox 90.0.2 (64-bit) started working again this morning.
The only obvious change, aside from regular security updates which pulled in Firefox 90.0.2, is this system no longer has Firefox Multi-Account Containers extension present.
Keep in mind while the problem was happening it persistent with a clean profile so I don't see how an extension is related.
I would have thought that devtools.cache.disabled would have helped but that and the workaround related to the dev console didn't help on the affected box. I really hope this is the last I see of this particularly annoying bug.

Comment 30

[Valentin Gosu [:valentin] (he/him)]

Can you still reproduce this bug?

Comment 31

[Danilo B]

valentin: Sorry for the late response, I haven't experienced this issue in a while now. I'd let you know if it happens again.

Comment 32

[François Villeneuve]

I can definitely reproduce this bug...

Comment 33

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thank you for the update, François.

Could you please provide detailed STRs (+ a page we should use for testing), so we can see the issue on our machines?
Also, Valentin was asking for HTTP log in comment #28 (If you can record a HTTP log that might be useful: https://firefox-source-docs.mozilla.org/networking/http/logging.html You can put it in a ZIP archive to make it smaller - since it's only text it compresses well.)
That would be great help!

Comment 34

[simon.campano]

Just adding some context here, developing on firefox, it's difficult to understand why firefox is NOT triggering a preflight & CORS error on a localhost request where it should. Checking the cache content should be an option to be able to rule out its implication. The error is properly triggered on chrome & firefox nightly, and on a colleague's ff, so it's definitely my firefox config, but I can't figure out what. I've deactivated extesions, tried resetting many about:config stuff etc etc. I'm in pain 😅

Comment 35

[François Villeneuve]

Unfortunately the site is IP fenced on a corporate network, and I don't feel like screening the data from 44mb of logs to see if there's anything that would violate my terms of employment by sharing it in public. (There probably is...)

Repro steps:

1. Publish site (app.domain.dev) with an associated assets bucket (assets.domain.dev) on AWS A3 with a specific CORS allowed headers policy (app.domain.dev)
2. Visit the site in my browser, assets load up fine.
3. Republish the site on a different subdomain (agent.domain.dev), with an updated CORS policy on the bucket, which is republished. (app.domain.dev, agent.domain.dev)
4. Visit agent.domain.dev
5. Assets coming from assets.domain.dev don't show up in Firefox, they should.

(repeat steps 1-4 in other browsers, assets show up)

Comment 36

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:dragana, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 37

[Daniel Hammer]

Hi Firefox gurus,

Like François Villeneuve from comment #35 , I too have been torpedoed by the inability to determine the root cause behind the console error:

Access to XMLHttpRequest at 'https://<REDACTED-server>/openid-connect/token' from origin 'https://<REDACTED-client>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

I changed identity-provider for the server-domain yesterday, essentially from domain a -> b, but have been unable to convince Firefox v113.0.1 on Fedora 37 to accept the new setup ever since and I'm at my wits end.
I've tried creating a new Firefox profile and disabling caching (browser.cache.disk.enable, browser.cache.memory.enable, browser.cache.offline.enable & devtools.cache.disabled) based on the many suggestions in here to no avail.
I've also tried adding a proxy in-between to catch the OPTIONS request, but none is present for the server-domain ~ visiting a new site like test-cors.org or https://httptoolkit.tech/will-it-cors/ shows up OPTIONS in the proxy-log.

I suspect the missing CORS request is either Firefox caching or a filtering rule based on the request signature ~ which in my case is an authorized POST from client-domain to server-domain.
(I can successfully request a CORS response from the server-domain using curl that contains the expected Access-Control-Allow-Origin, but alas Firefox doesn't seem to perform a similar request at all)

The take away from my ramblings, is that I miss support from Firefox to pinpoint the root cause of page-breaking error and subsequently apply a fix that's reflected in the browser deterministically :)
Any steps towards providing debugging insight going forward would be greatly appreciated. 🙏

Comment 38

[pasik]

I seem to be having this issue aswell. On one of my systems I'm not able to login to particular website in Firefox, due to "CORS Missing Allow Origin", no matter what I do. The problem happens on all the Firefox profiles on that system. Using another computer, in the same network, Firefox (the exact same version) works OK, without CORS problems.

Where can I view and clear the Firefox CORS cache?

This is with Firefox v121.0 currently, on Linux.

Comment 39

[rhardy]

Please correct severity to S2. Please update bug to show Firefox 121 is affected.
It should not be acceptable to Mozilla that the only work around which exists is to stop using Firefox.
This is a Firefox usage blocking bug which is forcing users off the platform.
This is once again happening for me now with a very popular 3rd party VPN. End result is users cannot login using their Firefox extension.
This should be an S2 severity. Please take this seriously.

Comment 40

[Valentin Gosu [:valentin] (he/him)]

There is a simple workaround to clear the CORS preflight cache without restarting the browser.

Open the browser console and type the following.
https://firefox-source-docs.mozilla.org/devtools-user/browser_console/index.html#opening-the-browser-console

Services.clearData.deleteData(Ci.nsIClearDataService.CLEAR_PREFLIGHT_CACHE, console.log)

Comment 41

[pasik]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #40)

There is a simple workaround to clear the CORS preflight cache without restarting the browser.

Open the browser console and type the following.
https://firefox-source-docs.mozilla.org/devtools-user/browser_console/index.html#opening-the-browser-console

Services.clearData.deleteData(Ci.nsIClearDataService.CLEAR_PREFLIGHT_CACHE, console.log)

THANK YOU! That did help, and now my CORS problem is gone!

I was pulling the hair out of my head for weeks as nothing seemed to help, but now clearing the cors Preflight Cache from the browser console seems to have finally fixed the cors problem.