Closed Bug 1591374 Opened 6 years ago Closed 6 years ago

Numerous sites fail to load after upgrade to Firefox 70.0

Categories

(Core :: Storage: localStorage & sessionStorage, defect, P2)

Product:
Core
Component:
Storage: localStorage & sessionStorage
Version:
70 Branch
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla70
Project Flags:
Webcompat Priority ?
Tracking Flags:
Tracking Status
relnote-firefox --- 70+
firefox70 blocking fixed
firefox71 blocking fixed
firefox72 + fixed

People

(Reporter: keith, Unassigned)

References

Details

(Whiteboard: [rca - Design Error])

Attachments

(4 files)

about support text
26.12 KB, text/plain
Details
Facebook Feed, which is blank
93.70 KB, image/png
Details
Asda grocery order description page
184.18 KB, image/png
Details
Results of using the glitch URL
95.14 KB, image/png
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0

Steps to reproduce:

Firefox updated to version 70.0
Trying to use Facebook, Asda.com (grocery order status), yougov.co.uk (survey website), www.e-rewards.co.uk (survey website). Each of these websites encounters an issue where content doesn't load

Actual results:

Website banners are displayed but content fails to load correctly and just shows a blank portion of the page where the detail should be. For example, Facebook shows the top bar but doesn't not display any feed content. The Asda site does not display order details. Both survey sites fail to display any content on the screen.

Expected results:

All of these websites should show data as they did before the Firefox upgrade.

I have since had to create a new profile and this has restored functionality. I was very close to ditching Firefox altogether. I am reporting this as I've seen issues reported similar to mine on other websites and would expect Mozilla to try to recitify this issue rather than expect folk to have to create new profiles. Your average user is not going to do this and, as I almost did, go to use anothe browser.

Can QA give a shot at replicating this?

status-firefox70: --- → affected
tracking-firefox70: --- → +
Flags: needinfo?(tmaity)

(In reply to Liz Henry (:lizzard) from comment #1)

Can QA give a shot at replicating this?

Liz, Argentina team is currently looking into it. @vbalducci will let us know if we can replicate.

Flags: needinfo?(tmaity)
Flags: needinfo?(virginia.balducci)

Some information from the problematic profiles would be useful, e.g. the contents of about:support and the browser console after visiting these sites.

Hi Liz,

We were unable to reproduce the issue on Windows 10 Pro version 1809 and Lenovo Yoga c630 with Windows 10 Home using the following:

Release builds updated to 70.0 Build ID 20191016161957:
Release 68.0 Build ID 20190705220548
Release 69.0 Build ID 20190827005903
Release 69.0.3 Build ID 20191009172106

Beta 69.0b16 Build ID 20190822210915 updated to 71.0b4 Build ID 20191024095932

I agree with Panos, it would be a good idea to check if any of the old profile's extensions or configuration may have caused the issue?

Regards,

Flags: needinfo?(virginia.balducci)
Attached file about support text —
Hi all, Thanks for having a look at this. In reply to the comment regarding the about:support, I just restarted Firefox with the old profile. I then visited Facebook and Asda.com as the two examples that are easier to reproduce. Excuse my terminology, but I believe that the static content loads OK for both of these sites, but dynamic content, however they render it, remains blank. Whilst the creation of a new profile resolved the issue for me, I just wanted to bring this to the attention of support. Thanks, Keith The content of about:support

Hi all,

We've tried downloading EN-GB locale, setting the preference security.enterprise_roots.enabled to true, and browsing a different kind of multimedia content, like playing games, watching online videos, watching non-live videos or scrolling images on Asda. We can't get to reproduce the bug.

@KD, could you share with us the specific URL where you've seen the error and a screenshot, please?

Flags: needinfo?(keith)

They have Kapersky installed, could this be intercepting/modifying requests via JavaScript?

Normally, using this URL would show the Facebook newsfeed; it does so with my new profile

Flags: needinfo?(keith)

The area above 'Back to Top' should show details below the Asda banner. It does so with a new profile or using another browser.

Hi! If you can go to https://firefox-storage-test.glitch.me/ (note that the https is important; clicking on the link here in Bugzilla will work) with your broken profile, that's our self-diagnosis site for storage problems.

If things are fine with the profile, then we'd expect "Specific Subsystem Statuses" on the page to look like:

LocalStorage
    Good: Totally Working. (fullyOperational)
QuotaManager
    Good: Totally Working. (fullyOperational)
IndexedDB
    Good: Totally Working. (fullyOperational)
Cache API
    Good: Totally Working. (fullyOperational)

If, however, there's a problem with the profile's storage, we'd expect some failures in that list. I unfortunately expect that to be the case. If you could copy and paste those lines in, that would be helpful. (There's more detailed data below in "Debug Info", but since you will have never visited the site before today, there won't really be anything too interesting in there.)

Hi Andrew. I've used the glitch link and attached the resulting screenshot, but all it says is "Investigating...
If this doesn't go away, things are unexpectedly broken. "
I'm going to go back to the new profile now, as the ITV player website, for streaming the rugby world cup, doesn't work with the old profile either (https://www.itv.com/hub/itv)
Thanks,
Keith

Using the link https://firefox-storage-test.glitch.me/ produces the following screen

Pasted the following URL into my browser, which worked just fine before the upgrade to 70.0:

https://9gag.com/gag/aAggRn2 - no image displays, just a spinning wheel!

Clicked:

https://firefox-storage-test.glitch.me/

It just displays the message: "Investigating... If this doesn't go away, things are unexpectedly broken." - IT'S NOT GOING AWAY!

Ran the same test on "Comodo IceDragon", which hasn't been updated, and it said "Storage is working. This is your first visit or all storage was automatically cleared."

So, my Firefox browser looks damaged!

Latest problems with Firefox - including login/next buttons not working on YouTube and eBay, meaning I can't log in! - has made the browser unfit for purpose. I've now switched to Chrome. Why can't I download previous versions of Firefox?

Emma, "They have Kaspersky installed". I also have Kaspersky, but "Comodo IceDragon", which is based on Firefox, works just fine! Ages ago, I disabled Kaspersky's injection of javascript - or something - into pages, so it can't be that either! But there are other problems I'm encountering - buttons on YouTube and eBay login pages not working!

Emma, are you part of the Firefox team or just a normal user?

(In reply to tomthumb2001 from comment #16)

Emma, are you part of the Firefox team or just a normal user?

Part of the team, but not a developer, there are developers on this thread working the problem.

Guess, I've created a duplicate: https://bugzilla.mozilla.org/show_bug.cgi?id=1591332
Works fine in private windows but not in safe mode....

I'm investigating the issue and I'll send more information soon. https://firefox-storage-test.glitch.me reports "Investigating... If this doesn't go away, things are unexpectedly broken." because even LocalStorage database for that site seems to be corrupted and script.js doesn't handle that currently.
For now, you can can fallback to the old LocalStorage implementation by changing "dom.storage.next_gen" preference to "false".

It works in private windows because data is isolated in and not persisted to disk in private browsing mode, so there can't be a corrupted database on disk.

This looks like a duplicate of 1591081 that I reported.

A clean profile install and re-install of addons and my prefs tweaks via user.js circumvented the problem. But as I stated in 1591081, this was quite inconvenient especially on the back of having to go through the same procedure for FF52*

*deprecating NPAPI plugins caused other untriaged profile incompatibility such that it was more expedient to wear the pain and start fresh to get rid of any legacy cruft.

(In reply to Emma Humphries Comment 17)

Part of the team, but not a developer, there are developers on this thread working the problem.

Thanks. And thanks for letting me know this bug had already been reported.

(in reply to Colin Templeman Comment 21)

This looks like a duplicate of 1591081 that I reported.

Yes, seems same problem I'm having with sign-in buttons. I also created a separate report for that bug, but no replies yet. Thanks for the information! Have you tried "Comodo IceDragon" browser? It's Firefox under the hood, and it's still working fine for me - as they haven't updated it yet!

https://icedragon.comodo.com/

Status: UNCONFIRMED → NEW
status-firefox71: --- → affected
status-firefox72: --- → affected
Component: Untriaged → DOM: Web Storage
Ever confirmed: true
Product: Firefox → Core
Target Milestone: --- → mozilla70

For now, I'll add this to the Firefox 70 release notes as a known issue.
Joni is this something you could write up in SUMO? The workarounds are likely, either doing a refresh, or, setting dom.storage.next_gen" preference to "false".

Flags: needinfo?(jsavage)

(In reply to tomthumb2001 from comment #23)

Have you tried "Comodo IceDragon" browser? It's Firefox under the hood, and it's still working fine for me - as they haven't updated it yet!

No, it's a Windows only binary (I'm a Linux user). I can use prior versions if needed but I'd prefer to use my work-around on FF70 and use my old profile to test any fixes :)

https://firefox-storage-test.glitch.me has been fixed, you can try it again and report back.

We're discussing disabling LSNG in 70.1 and adding some extra telemetry in 71. Time for 70.1 isn't determined yet but likely this week.

Adding a release note to 70.0 under known issues, "Some websites or page elements using dynamic JavaScript fail to load"

tracking-firefox70: + → blocking
relnote-firefox: --- → 70+
Severity: normal → major

If this is a 70 blocker, then this is also blocking 71 which is well into beta now.

tracking-firefox71: --- → blocking
tracking-firefox72: --- → +

Hi Jan V,
I've just re-tried the ...glitch.me link and got the following results on the original, 'broken', profile

Regards,
Keith

Overview:
Storage is broken. This is your first visit or all storage was automatically cleared.
Specific Subsystem Statuses:

LocalStorage
Bad: Our test logic is broken, please copy and paste the contents of 'Debug Info' below and anything in the devtools console and send to :asuth. (unexpectedBreakage)
QuotaManager
Bad: Totally Broken. (fullyBroken)
IndexedDB
Bad: Totally Broken. (fullyBroken)
Cache API
Bad: Totally Broken. (fullyBroken)

Debug Info:

storage.estimate() threw: The expression cannot be converted to return the specified type.
Failed to create "persistent" IDB.
Failed to create "transient" IDB.

{
"v": 1,
"curVersion": 70,
"prevVersion": 0,
"ls": {},
"qm": {
"lastWorkedIn": 0
},
"idb": {
"persistentCreatedIn": 0,
"persistentLastOpenedIn": 0,
"clearDetectedIn": 0
},
"cache": {
"firstCacheCreatedIn": 0,
"unpaddedOpaqueCreatedIn": 0,
"paddedOpaqueCreatedIn": 0
}
}

Yes, this makes sense. It's not directly fault of new LocalStorage implementation (LSNG), but there's a problem with your storage/ folder in Firefox profile. It's likely that the problem existed before you updated Firefox to version 70.0. You just didn't notice because LocalStorage didn't depend on QuotaManager. We are going to disable new LocalStorage implementation until these problems are solved.
Thanks for posting the report.

As an aside, I just used the dom.storage.next_gen (false) workaround in my original profile, and Facebook now loads. I assume that the other sites affected will do so as well but haven't had chance to try them.
You probably didn't need confirmation of this but I thought I'd try it out :)

Clearing need-info - article with possible workaround already published (see Comment 27)

Flags: needinfo?(jsavage)

Question, has anyone tried using this: https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings# too see if the refresh clears the error? the page indicates that your standard user data that you won't want to be rid of is preserved and restored; what you loose are customizations, addons, and accessibility changes.

Hi Stephenopolos,
I changed dom.storage.next_gen back to true on my old profile and Facebook stopped working.
I then used the refresh link you posted above, https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings#
Using just Facebook as an example; apart from having to sign in again, the feed content did display correctly.
This method would look to be less hassle than creating a new profile. However, the dom.storage.next_gen parameter change has less of an impact in terms of sign-ins I suspect. Depends if it's expected that folk need to risk the 'here be dragons' :)
Thanks,
Keith

I think bug 1592279 I reported is related, it has rendered FF unusable on our systems. Resetting the profile does not resolve it, dom.storage.next_gen is false.

https://bugzilla.mozilla.org/show_bug.cgi?id=1592279

This ticket sounds like the exact same issue that I had when updating to 70 - see #1590989.
As the official workaround was not available at that date, I've found a different solution by using the "about:support" page (in case anybody is interested).

We do get more functionality after resetting and setting dom.storage.next_gen to false. However the storage test does return "fully broken" for the last three items still. We are having self reporting users work around the issue this way.

We are waiting for a definitive resolution before deploying anything to all our users right now.

If refreshing/resetting profile doesn't work, then there are two options:

  1. Run a debug build and send us debug output from the console
  2. Send us problematic profile directory privately and we will investigated it
Priority: -- → P2

I don't see anyone mentioning that this issue is breaking reddit, but it prevented all JavaScript-based functionality (voting, replying, collapsing comment threads, etc.) from working until I discovered this post with other users' complaints, as well as the dom.storage.next_gen workaround.

With respect to #1591081 and here, I've now tried setting "dom.storage.next_gen" preference to "false" in my old profile and https://firefox-storage-test.glitch.me/ yields:

Overview:
Storage is broken. This is your first visit or all storage was automatically cleared.
Specific Subsystem Statuses:

LocalStorage
Good: Totally Working. (fullyOperational)
QuotaManager
Bad: Totally Broken. (fullyBroken)
IndexedDB
Bad: Totally Broken. (fullyBroken)
Cache API
Bad: Totally Broken. (fullyBroken)

Debug Info:

storage.estimate() threw: The expression cannot be converted to return the specified type.
Failed to create "persistent" IDB.
Failed to create "transient" IDB.

{
"v": 1,
"curVersion": 70,
"prevVersion": 0,
"ls": {},
"qm": {
"lastWorkedIn": 0
},
"idb": {
"persistentCreatedIn": 0,
"persistentLastOpenedIn": 0,
"clearDetectedIn": 0
},
"cache": {
"firstCacheCreatedIn": 0,
"unpaddedOpaqueCreatedIn": 0,
"paddedOpaqueCreatedIn": 0
}
}

Going back to "new" refreshed profile. Sorry but I can't conduct further tests until Monday 4th now sorry

See Also: 1591170 →

(in reply to Colin Templeman Comment #41)

Setting "dom.storage.next_gen" to false worked for me - got the "sign in" buttons working and sites back to displaying images/pages - but I'm using Windows 7.0. Maybe Linux is making things harder. I'd love to use Linux, but always too many problems with it.

Was also given this link, but didn't need to do this:

https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings

This is a Mozilla testing team problem, not GNU/Linux problem. From my point of view, Windows has many many problems, since you have to wait several seconds to open a single window, have viruses, need hours to download updates without consult you, etc.. I invite you to test both systems in the same hardware. Anyway is a personal election.

That last link you post is not necessary here. Only turn off dom.storage.next_gen as described above to workaround it.

But I insist, this is not a GNU/Linux problem.

(In reply to tomthumb2001 from comment #43)

Maybe Linux is making things harder. I'd love to use Linux, but always too many problems with it.

Harder how? Pretty broad brushstrokes you're using there. I've used Linux since 1999 and full-time since 2004. I can assure you there aren't "too many problems" with it, quite the reverse in fact. As Lupa says, it runs far more efficiently than Windows of any given hardware and you don't have to wonder what personal info is being snagged by telemetry you have no control over.

Since you can share the same Firefox profile between Firefox on Windows and Firefox on Linux why not boot a version of Linux via DVD or USB and have a play with it. Linux Mint is often cited as a friendly distro for dyed-in-the-wool Windows users, but there are many others.

lupa, Windows isn't great, but it works. I install Linux; it works at first, but then I get a black screen on reboot; I install another version; it initially works, but then I can't update the software; I install yet another distro; it works, but then there's another problem - it's just endless! Linux is bad because it isn't well supported. That's just fact!

I'm NOT cheerleading Windows - it's a third-rate OS as far as professional operating systems go. And that's because Bill Gates has NOT written a single OS in his life - he simply bought a clone of CP/M, the Windows of its day, stole the market from Gary Kildall - who would have given us a far superior OS - and became a billionaire for doing next to nothing. Gates has NEVER taken any pride in Windows, and has probably never even used it.

SIDE NOTE: The only reason IBM allowed Gates to keep the licensing rights to DOS (the CP/M clone) was because they feared being sued. Kildall could have sued, but said "let the market decide", but IBM priced CP/M for the PC at $240 vs $40 for DOS to kill it.

The full story is in the *paperback" edition of "They Made America" by Harold Evans - there's a full chapter on Gary Kildall. Gates is a real nasty bastard! His philanthropy isn't real either - it's effectively, keep the billionaires in charge & throw the rest of us a few crumbs - or not, as the case may be!

https://www.youtube.com/watch?v=Nrg9VVkZQb4 - move slider to 14:23 and watch Kildall (who co-presented that program for six years) introduce GEM. GEM is what we would have got had Gates not stabbed a friend in the back.

What Microsoft gave us: https://www.youtube.com/watch?v=UToIwIe6sSM

That last link you post is not necessary here.

That last link was given to me by Firefox support. I'm NOT an expert on debugging Firefox!

Colin said:

Harder how? Pretty broad brushstrokes you're using there.

It's NOT broad brushstrokes. It's the experiences I've had, and most people have had with Linux BECAUSE IT'S NOT PROPERLY SUPPORTED. Read my comment above!

I can assure you there aren't "too many problems" with it

I can assure you there are!

I'm not a dyed-in-the-wool Windows user

I'm really tired of the way people identify with a product or with a political group and take it personally. I despise Microsoft and have heavily criticised Windows for being badly written and poorly designed. It's NOT my fault Bill Gates stabbed Gary Kildall in the back, who would have given us a quality OS - see my previous comment!

But I need something that works - I've got too much stress in my life to spend hours solving OS problems - so that means Windows for me.

colin & lupa, part of the story about Gary Kildall here: https://www.youtube.com/watch?v=Tdj8gh9GPc4 - rest is in that book I mentioned.

(In reply to tomthumb2001 from comment #48)

It's NOT broad brushstrokes. It's the experiences I've had, and most people have had with Linux BECAUSE IT'S NOT PROPERLY SUPPORTED. Read my comment above!

I'm sorry you feel that way and suggest that if you just looked you would find plenty of support. However I think it would be better for everyone if we keep on topic rather than filling up the bug-tracking system with personal opinions.

Hi folks,
I have just realized that the errors I get with "https://firefox-storage-test.glitch.me/" are directly related to using private browsing mode.
When in private mode, I reproducibly get the following two errors across a number of different Firefox installations.
Without private mode, no errors are reported.
(Sorry if this is already known. None of the discussions or proposed workarounds I found so far seems to mention this relationship.)

LocalStorage
Good: Totally Working. (fullyOperational)
QuotaManager
Good: Totally Working. (fullyOperational)
IndexedDB
Bad: Our test logic is broken, please copy and paste the contents of 'Debug Info' below and anything in the devtools console and send to :asuth. (unexpectedBreakage)
Cache API
Bad: Totally Broken. (fullyBroken)

This should now be fixed in 70.0.1, by the patch in bug 1592136.

Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox70: affected → fixed
status-firefox71: affected → fixed
status-firefox72: affected → fixed
Resolution: --- → FIXED

(In reply to Liz Henry (:lizzard) from comment #52)

This should now be fixed in 70.0.1, by the patch in bug 1592136.

It is not. My observations still apply to 70.0.1 (see comment #51).

(In reply to Spampot from comment #51)

IndexedDB
Bad: Our test logic is broken, please copy and paste the contents of 'Debug Info' below and anything in the devtools console and send to :asuth. (unexpectedBreakage)
Cache API
Bad: Totally Broken. (fullyBroken)

IndexedDB and Cache API don't work in private browsing currently and are completely separate issues. If firefox-storage-test.glitch.me reports no broken stuff in normal windows then your Firefox installation and profile should be fine (at least, from the point of storage APIs).

Hi, not sure how to get help so trying here, it seems I'm still having this issue.
Some pages acted weird so I actually made a new profile, but this issue seemed to get worse.

https://firefox-storage-test.glitch.me/ was originally broken, but I checked for dom.storage.next_gen that was actually set to false by default?
Turning it to on made the tests work but the sites are still broken.

Also TLS handshake on Github (my orginal reason for making a new profile) is still very slow.

Debug Info:

{
  "v": 1,
  "curVersion": 70,
  "prevVersion": 70,
  "ls": {},
  "qm": {
    "lastWorkedIn": 70
  },
  "idb": {
    "persistentCreatedIn": 70,
    "persistentLastOpenedIn": 70,
    "clearDetectedIn": 0
  },
  "cache": {
    "firstCacheCreatedIn": 70,
    "unpaddedOpaqueCreatedIn": 0,
    "paddedOpaqueCreatedIn": 70
  }
}

I'm on FF70.0.1, macOS.

My Developer Edition profile currently on FF71 has never showed any issue, neither has my Windows machine.

My excuses, the weirdness that made me post here was actually due to connectivity issues on my network. Seems to work fine now.

This bug has been identified as part of a pilot on determining root causes of blocking and dot release drivers.

It needs a root-cause set for it. Please see the list at https://docs.google.com/document/d/1FFEGsmoU8T0N8R9kk-MXWptOPtXXXRRIe4vQo3_HgMw/.

Add the root cause as a whiteboard tag in the form [rca - <cause> ] and remove the rca-needed keyword.

If you have questions, please contact :tmaity.

Keywords: rca-needed

Same as https://bugzilla.mozilla.org/show_bug.cgi?id=1570644#c18.

Keywords: rca-needed
Whiteboard: [rca - Design Error]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: