788365 - ApplicationCache fails to update in FF16+ via HTTPS when manifest has Network:*

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect)

Description

[Matthew Riddle]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1

Steps to reproduce:

Visited our offline web app in Firefox beta 16 via a HTTPS URL


Actual results:

The application cache checked for updates and the files listed in the application manifest were then downloaded successfully.
The "updateready" event was then fired. Upon refresh the application cache checked for updates again and then fired an error,
triggering the applicationCache "onerror" handler with no useful information. The user is then unable to work offline at this point.
Any subsequent refreshes of the page will continue to error during the updates check.


Expected results:

Upon refresh after the application cache was updated, it should have checked for updates and fired the "noupdate" event, allowing the user to work offline.

Notes:
 * By accessing our offline web app via http instead of https everything works fine.
 * By accessing our offline web app via https in Chrome or previous versions of Firefox (<= 15) it works fine
 * By removing the network section in our application manifest we are able to update and use our offline app via https
   but are no longer able to make any GET requests to the server

I can provide more in depth information if required. I am not sure of what other information is useful and what isn't.
I can also provide the HTTP and HTTPS URLs so you can see the issue for yourself, I would prefer not to list them publicly.

Comment 1

[Loic]

Thanks for the report. I would be good to have an access to a minimal testcase (maybe not your corporate web app, but a simple web app which does nothing, only to expose the issue). Is it possible?

Comment 2

[Matthew Riddle]

I was unable to create a simple test case. I have set aside an environment in which you can replicate the problem, its a duplicate of our prod environment. We're fine with you guys accessing it as the data is locked away. Would you like me to email you the links to the server?

Comment 3

[Jason Duell]

> Would you like me to email you the links to the server?

Yes, please email me a link--that would be great.  You can email jduell at mozilla dot com if you don't trust who is behind random gmail addresses (which you probably shouldn't :)

Comment 4

[Maxime RETY]

I have the same bug where the applicationCache fails to update over HTTPS.
It seems to work fine over HTTP however.

I can reproduce this bug consistently on the latest beta of Firefox 16 (on Aurora and Nightly too), but not on Firefox <= 15.

Jason, I emailed you a link to a simple test case which demonstrates this issue.

Don't hesitate to tell me if I can do anything more to help.

Comment 5

[Alex]

Hi there !

I got the same bug (applicationCache doesn't update in HTTPS),
will this be fixed for Firefox 16 ???

Thank you

Comment 6

[Loic]

(In reply to Alex from comment #5)
> I got the same bug (applicationCache doesn't update in HTTPS),
> will this be fixed for Firefox 16 ???
If it's not a blocker, surely not. The delay is too short to add the patch and do minimal beta-testing.

Comment 7

[Maxime RETY]

Hi Loic,

This ticket needs some clarifications.


First, the bug described aboved has nothing to do with "Network: *", the manifest will not be updated over HTTPS (firing an "error" event just after the "checking" event) whether there is a NETWORK section or not.


Then, the situation is much worse than I thought earlier:

Not only the manifest will not be updated over HTTPS, but the ressources listed in the manifest will not be available anymore when accessed offline.

Here are some example pages gathered from the MDN and from Opera's dev blog :
https://developer.mozilla.org/media/uploads/demos/p/a/paulrouget/8bfba7f0b6c62d877a2b82dd5e10931e/hacksmozillaorg-achi_1334270447_demo_package/todo/
https://people.opera.com/shwetankd/demos/2/index.htm

Go to these pages in Firefox 15.0.1, then go offline: they work as expected.

The same test on Firefox 16 beta shows that the ressources listed in the manifest are not available when requested while being offline.

I emailed you a "private" link to a test page I built, which exposes clearly both issues :
1. The applicationCache fails to update over HTTPS.
2. The ressources listed in a manifest of a page served over HTTPS are not available offline (although they were correctly downloaded once)

Should I file a separate bug on this second issue ?


Those are serious regressions that will defeat every webpage using the applicationCache in combination with HTTPS.

Comment 8

[Jason Duell]

Honza, I've emailed you the link mentioned in comment 2.

Comment 9

[Loic]

Maxime emailed me testcase today. There is a regression in FF16+ indeed.

With FF16, when Firefox works offline, the HTTPS web app fails to work ("Firefox is currently in offline mode and can't browse the Web.") but the HTTP version works fine. 

m-c
good=2012-05-31
bad=2012-06-01
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=3aa566994890&tochange=73783bf75c4c

m-i
good=2012-06-01
bad=2012-06-02
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=50c9995aa7d0&tochange=9abc60f44fd5

There are 2 suspected bugs: bug 722034 and bug 746018.
I think the 2nd one is the culprit (some patches about offline cache).

Both of these bugs have been added into Nightly Firefox 15 in June then backout from Aurora Firefox 15 in July (see https://bugzilla.mozilla.org/show_bug.cgi?id=722034#c79 and https://bugzilla.mozilla.org/show_bug.cgi?id=746018#c45).
But they are still in FF16+, so maybe a backout from FF16 needs to be done before releasing.

Comment 10

[Honza Bambas (:mayhemer)]

(In reply to Matthew Riddle from comment #0)
> Steps to reproduce:
> 
> Visited our offline web app in Firefox beta 16 via a HTTPS URL

Where is the URL please?

Comment 11

[Honza Bambas (:mayhemer)]

(In reply to Honza Bambas (:mayhemer) from comment #10)
> (In reply to Matthew Riddle from comment #0)
> > Steps to reproduce:
> > 
> > Visited our offline web app in Firefox beta 16 via a HTTPS URL
> 
> Where is the URL please?

In the private mail!  Sorry for spam.

Comment 12

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

I am pretty sure this is caused by the check I added to ensure that every HTTPS cache entry has a non-null securityInfo. IIRC, the offline cache never serializes the securityInfo into the cache, so when you read the entry from the cache, it will have a null securityInfo. IIRC, previous versions of Firefox displayed "broken" no "no SSL" indicators for HTTPS pages from the offline cache, but still allowed them to work. If this is the case, for now we can simply remove the check I added, and then separately resolve the issue where the offline cache does not serialize the securityInfo.
.
I am verifying this theory now

Comment 14

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Attachment: Workaround: Do not require securityInfo for offline cache entries for now

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Bug 746018 (Part 6)
User impact if declined: Offline cache entries will not update correctly. In debug builds, an assertion dialog box will pop up.
Testing completed (on m-c, etc.): Tested manually, will create an automated test.
Risk to taking this patch (and alternatives if risky): Very low. This is just partially reverting back to pre-FF16 behavior.
String or UUID changes made by this patch: none.

Comment 15

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Attachment: Workaround: Do not require securityInfo for offline cache entries for now [v2]

Fixed comment.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Bug 746018 (Part 6)
User impact if declined: Offline cache entries will not update correctly. In debug builds, an assertion dialog box will pop up.
Testing completed (on m-c, etc.): Tested manually, will create an automated test.
Risk to taking this patch (and alternatives if risky): Very low. This is just partially reverting back to pre-FF16 behavior.
String or UUID changes made by this patch: none.

Comment 16

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

I have not added an automated test yet, but I'm running this through try to make absolutely sure nothing is regressing:
https://tbpl.mozilla.org/?tree=Try&rev=c94c1c5696be

Comment 17

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e2da1a9124b6

Comment 18

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Try runs for -aurora and -beta are running now:
https://tbpl.mozilla.org/?tree=Try&rev=5c787fc0db0d
https://tbpl.mozilla.org/?tree=Try&rev=be55a62862e4

Comment 19

[Alex Keybl [:akeybl]]

(In reply to Brian Smith (:bsmith) from comment #15)
> Created attachment 665013 [details] [diff] [review]
> Workaround: Do not require securityInfo for offline cache entries for now
> [v2]
> 
> Fixed comment.
> 
> [Approval Request Comment]

We're waiting to hear back about https://bugzilla.mozilla.org/show_bug.cgi?id=770243#c35 before making a final decision here.

Comment 20

[Alex Keybl [:akeybl]]

Comment on attachment 665013 [details] [diff] [review]
Workaround: Do not require securityInfo for offline cache entries for now [v2]

[Triage Comment]
Approving for Aurora in preparation for Beta 16 consideration.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/e2da1a9124b6

Comment 22

[Matthew Riddle]

Thank you so much!

Comment 23

[Alex Keybl [:akeybl]]

Comment on attachment 665013 [details] [diff] [review]
Workaround: Do not require securityInfo for offline cache entries for now [v2]

[Triage Comment]
We don't want to regress offline web app support with our new release, so let's  get this landed on Aurora 17 and Beta 16.

Comment 24

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/b1feef78e8c4
https://hg.mozilla.org/releases/mozilla-beta/rev/fa41f9b6b0f9

Comment 25

[Maxime RETY]

I've just tested on the latest beta and everything works as usual.

Thanks for the fix!

Comment 26

[u279076]

Thanks Maxime. Would you mind testing with the Firefox 16.0.1 release and latest Firefox 18.0a2 Aurora?

Comment 27

[Maxime RETY]

It's ok with Firefox 16.0.1 and Aurora 18.0a2 (2012-10-16).

Comment 28

[u279076]

Thank you Maxime.

Comment 29

[Tanvi Vyas[:tanvi]]

From https://bugzilla.mozilla.org/show_bug.cgi?id=794507#c1, it sounds like this patch should be reverted now since bug 794507 was fixed.  Honza, can you take a look at this?

Comment 30

[Honza Bambas (:mayhemer)]

It still has some meaning: https://bugzilla.mozilla.org/show_bug.cgi?id=1009531#c22