Closed Bug 1014979 Opened 10 years ago Closed 8 years ago

Error sec_error_ocsp_server_error is cached for too long, which causes inability to access some HTTPS websites when using "hard-fail" OCSP

Categories

(Core :: Security: PSM, defect)

Product:
Core
Component:
Security: PSM
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: a4206901, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0 (Beta/Release)
Build ID: 20140506152807

Steps to reproduce:

I'm using Firefox v29.0.1 on Windows 7 x64.
Build ID: 20140506152807.
User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0.

1. Turn Firefox OCSP "hard-fail" on. See https://wiki.mozilla.org/CA:OCSP-HardFail for more information on OCSP "hard-fail."
2. Trigger error sec_error_ocsp_server_error ("The OCSP server experienced an internal error") when browsing a HTTPS website with Firefox. I used https://news.google.com/, but I believe that the specific HTTPS website used doesn't matter. You probably won't be able to reproduce the exact steps that I used to trigger error sec_error_ocsp_server_error because in my case it's due to a router-specific issue. However, according to https://bugzilla.mozilla.org/show_bug.cgi?id=495380 and https://bugzilla.mozilla.org/show_bug.cgi?id=645389, various issues can cause this error, including some client-side issues. I'll list the exact steps that I used near the end of this report.
3. Use a different instance of Firefox (also configured to use OCSP "hard-fail") to browse the same HTTPS site that you browsed in step 2. Verify that you can browse the website successfully.
4. Using the same instance of Firefox as in step 2 (without first exiting Firefox), browse the same HTTPS website again.


Actual results:

You get error sec_error_ocsp_server_error ("The OCSP server experienced an internal error"), which makes the HTTPS website inaccessible.


Expected results:

You ought to be able to browse the HTTPS website.

Hypothesis of cause: caching of error sec_error_ocsp_server_error. Similar to bug https://bugzilla.mozilla.org/show_bug.cgi?id=933109. According to several sources, such as http://superuser.com/questions/755755/sec-error-ocsp-server-error-when-trying-to-open-a-https-page, this caching lasts 30 minutes, or until Firefox is restarted.

Here are the specific steps that I used to trigger error sec_error_ocsp_server_error in step 2:
2a) Use router WGR614v5.
2b) Set router WGR614v5 setting "Disable SPI Firewall" to unchecked. 
2c) Set Firefox option Options->Tabs->"Don't load tabs until selected" to checked.
2d) Have around 450 open tabs. 
2e) Exit Firefox.
2f) Restart Firefox. At some point before Firefox becomes responsive to user input, the router blocks DNS resolution for a few minutes. I verified this with operating system command Nslookup.
2g) When Firefox first becomes responsive to user input, browse https://news.google.com/. You'll get error "Server Not Found."
2h) Wait until DNS resolution is functioning normally (which takes a few minutes).
2i) Browse https://news.google.com/. You should get error sec_error_ocsp_server_error.

Workaround #1: Restart Firefox. This flushes the OCSP cache.

Workaround #2: Flush OCSP cache by turning off "hard fail" OCSP (making sure to exit Firefox Options), and then turning "hard fail" OCSP back on.
The same issue is present in Firefox Nightly v32.0a1 2004-05-22.
In the last comment, I meant Firefox Nightly v32.0a1 2014-05-22.
Two notes about the specific steps that I used in step 2:
1. Router firmware is v1.0.9_1.0.6.
2. Firefox was set to restore at startup the tabs that were present at last shutdown.
Step 2g should have read:
2g) When Firefox first becomes responsive to user input, browse https://news.google.com/. You'll eventually get error sec_error_ocsp_server_error.
Version: trunk → 3.16
If you're getting this error for the same reason I was, another workaround is to disable Stateful Packet Inspection (SPI) in your router, if possible. For the WGR614v5 router, the setting is called "Disable SPI Firewall."

More information about DNS resolution problems with WGR614v5 router:
http://forums.gentoo.org/viewtopic-p-5385956.html?sid=c083f532b013399e6a409c9300c23dcf
http://ionipti.blogspot.com/2010/03/googles-dns-server-better-than-comcasts.html

Hypothesis about why the router behaves this way:
http://www.benjohnstonphotography.com/theotherside/myblog/files/37a8e75bfb29230efbd8afba2223bbca-365.php

Information about Stateful Packet Inspection:
http://www.wilderssecurity.com/threads/stateful-packet-inspection-dos-attacks.286440/
http://www.dslreports.com/forum/remark,19391059
For some reason, I usually get OCSP errors when browser is restarted after a crash.
I also sometimes get such OCSP errors after restarting Firefox, probably because I have many tabs and they are all reloaded at the same time (this seems to be a Tab Mix Plus limitation/bug). But the main problem is that reloading the tabs with an OCSP failure "The OCSP server experienced an internal error. (Error code: sec_error_ocsp_server_error)" doesn't have any effect.

I don't think that the problem is that the error is cached for too long, because I still get the error with Ctrl-Shift-R, which should by-pass the cache:

https://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly says for Ctrl-Shift-R: "Reload (override cache)".
@Vincent Lefevre: Did you try Workaround #2 in the first post?
(In reply to Brian from comment #9)
> @Vincent Lefevre: Did you try Workaround #2 in the first post?

Since the problem occurred again, I could try this workaround, and it worked.
Assignee: nobody → nobody
Component: Libraries → Security: PSM
Product: NSS → Core
Version: 3.16 → Trunk
(In reply to Brian Smith (:briansmith, was :bsmith; NEEDINFO? for response) from comment #11)
> Maybe bug 1040889?

That would certainly make it worse. Also, the timeout is currently 5 minutes: http://hg.mozilla.org/mozilla-central/file/717cd9d89a80/security/certverifier/NSSCertDBTrustDomain.h#l87
Maybe that's longer than necessary.
I'm also seeing this issue all the time when I restart firefox and my windows PC, could be that windows caching is in effect since most of the time I have to do several restarts of Firefox until gmail is finally loaded.
Today I was still getting this issue even after three restarts of firefox, so I googled and found the magic ctrl+shift+R which resolved the issue. I should note that google.com and some other https sites were working just fine, only gmail.com was not working.
This issue still exists in Firefox 33.0. The caching time empirically is no longer than about 5 minutes, which is in agreement with Comment 12.
Setting OCSP to hard-fail isn't a supported configuration of the browser. As a workaround, the browser can be restarted, which clears the OCSP cache (note that the OCSP cache is not the same as the HTTP cache).
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
(In reply to David Keeler [:keeler] (use needinfo?) from comment #15)
> Setting OCSP to hard-fail isn't a supported configuration of the browser.

Then how does Firefox check for certificate revocation *reliably*?

> As a workaround, the browser can be restarted, which clears the OCSP cache
> (note that the OCSP cache is not the same as the HTTP cache).

This is not a good workaround at this will lose data in forms.
(In reply to Vincent Lefevre from comment #17)
> Then how does Firefox check for certificate revocation *reliably*?

We're using a system called OneCRL: https://blog.mozilla.org/security/2015/03/03/revoking-intermediate-certificates-introducing-onecrl/

> This is not a good workaround at this will lose data in forms.

Those should be cached by the browser unless the site is deliberately clearing them. You might file a bug in the general Firefox component if you're having an issue with this.
(In reply to David Keeler [:keeler] (use needinfo?) from comment #18)
> (In reply to Vincent Lefevre from comment #17)
> > Then how does Firefox check for certificate revocation *reliably*?
> 
> We're using a system called OneCRL:
> https://blog.mozilla.org/security/2015/03/03/revoking-intermediate-
> certificates-introducing-onecrl/

But according to what is said, this does not cover all certificates.

> > This is not a good workaround at this will lose data in forms.
> 
> Those should be cached by the browser unless the site is deliberately
> clearing them. You might file a bug in the general Firefox component if
> you're having an issue with this.

Done: bug 1262035 for Facebook, bug 1262037 for Bugzilla.
You need to log in before you can comment on or make changes to this bug.