Closed Bug 1345456 Opened 7 years ago Closed 7 years ago

Favicon fails to display for majority of tabs upon restart.

Categories

(Firefox :: Tabbed Browser, defect)

Product:
Firefox
Component:
Tabbed Browser
Version:
52 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox-esr45 --- unaffected
firefox52 + verified
firefox-esr52 52+ fixed
firefox53 --- unaffected
firefox54 --- unaffected
firefox55 --- unaffected

People

(Reporter: marcdpda, Unassigned)

References

Details

(Keywords: regression, regressionwindow-wanted, reproducible)

Attachments

(1 file)

sessionstore.js
4.61 KB, application/javascript
Details 
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20170302120751

Steps to reproduce:

- Start FF 52



Actual results:

- Favicon seems to appear for some recently used tabs (Pinned and non-Pinned). 
- All Tab titles appear Ok.


Expected results:

- All Favicons should appear, as in previous FF 51.0.1.
Component: Untriaged → Tabbed Browser
I'm seeing this as well with version 52, and I have a hunch it's dependent on the number of tabs shown. On one instance of Firefox where I have over 200 tabs, most of them don't show the favicon (about 25 do). On another system where there are 34 tabs, all of them show a favicon.
Status: UNCONFIRMED → NEW
Has Regression Range: --- → no
Has STR: --- → yes
status-firefox52: --- → affected
status-firefox53: --- → ?
status-firefox54: --- → ?
status-firefox55: --- → ?
status-firefox-esr45: --- → unaffected
status-firefox-esr52: --- → affected
tracking-firefox52: --- → ?
tracking-firefox-esr52: --- → ?
Ever confirmed: true
Keywords: regression, regressionwindow-wanted, reproducible
OS: Unspecified → Windows
Hardware: Unspecified → All
Not Windows specific. Present in Linux: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Yes, I can see this on Linux, started immediately on update to FF52. I have about 400 tabs open.
I'm not seeing it on Aurora on the same machine, which profile has much less tabs (~30).
Having the same problem on FF52 Win8.1. I am using auto unload tab extension, when tab unloads - favicon disappear. It was working fine in previous versions of FF.
Same here and it's really annoying so I will install 51.01 until it's fixed.

I read a post of someone saying on reddit that he was on nightly 55 and this was also happening!
If a fix is found shortly, is this the type of bug that would be resolved in a point release or are would we be looking at the next major version? I'm just trying to decide if I should go back to 51.
+1, I can reproduce it on 52 ESR on WinXP with just 20-30 tabs... hope the bug get's fixed on ESR also... will downgrade to 51 in the mean time as it's very annoying and very hard to identify small tabs without icons.
I'm seeing it with about 40 tabs, even with all add-ons disabled. If I create a clean profile it's harder to reproduce, probably why it got missed.
I can say that not all favicons are lost. SOME of the icons do appear at startup. They are the icons on tabs used in the previous session, but not all of those a restored.
Well I installed 51.0.1 and to my surprise the problem persist! When I restart firefox some tabs icons are missing!

Anyone else? What's going on? Did 52 installed/changed something that isn't restored when installing 51.0.1 over it?

Most annoying update ever. :-(
Priority should be raised.
This bug seems to happen when upgrading to 52 from a profile with a lot of tabs open. Something goes wrong during the upgrade process where most of the favicons get lost or corrupted.
Creating a new profile in FF 52 and reinstalling all add-ons and tabs seems to be the only work-around. I would rather wait for a bug fix than reinstall/reconfigure all my add-ons.
Attached file sessionstore.js — 
Here is a sessionstore.js file showing the problem.

1) Create a new profile and open it.
2) In general preferences choose "When Firefox starts: Show my windows and tabs from last time", 
3) Close profile.
4) Swap sessionstore.js with the provided one.
5) Open profile

The 2 first tabs are missing favicons.
Reloading broken tabs doesn't fix the problem.
New tabs work correctly.
Thank you Christophe, for assisting in providing a workaround that we can utilize! :)

- For any tab affected by the problem, load that tab's URL into a new tab.
* Duplicating the tab won't work - it has to be loaded into a brand new tab.
- Restart the browser
- The new tabs keep/load their FavIcons, even though the tabs aren't loaded. Hurray!

I'm in the process of doing this for my ~200 tabs. So far, testing looks great!
I hope Mozilla fixes the base issue, but at least we now have a workaround.
Like I said even after downgrading to 50.0.1 tabs icons sometimes go...
I refresh to get icon, quit, reload firefox and some are missing again.
oops I mean 51.0.1
As for attachment 8846447 [details]
Reproducible on 52.0, but unreproducible on nightly 55.0a1 or beta 53.0b1.
It seems that bug 1338009 already resolved this case, for 53.0 ...

(this message appears in Browser Console
>NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsISerializationHelper.deserializeObject] Utils.jsm:94
in this case)
Apologies for stupid question, but what is the mercurial (hg) syntax for downloading firefox 53.0 source so that I can compile a version that fixes subject bug? 

(I ask because version 53.0 source has not been released yet on ftp site, so I assume I have to use mercurial. Yes, I have googled but found nothing).
Apologies again, but after hg clone of 5GB and 2 hours of reading about mozilla and mercurial, I still have not been able to deduce how to find the revision number or changeset number that corresponds to version 53.0 or even 53.0b2. Could someone please help, it may be enough to post a oneliner command to get me started. Thx.
Can confirm on Win and Mac.
Anyone else reverted to 51.0.1 only to find out the problem is still present?
I installed 51.0.1 on top of 52.0 because I wanted to keep all my settings.
As far as I understand, it's all about
"principalToInherit_base64", "triggeringPrincipal_b64" and "triggeringPrincipal_base64" fields in sessionstore.js
New tabs contain these fields, old tabs doesn't.
Probably fixed in bug 1301666?
Christoph, can you please weigh in here? If there's a fix on 53 already, might be worth considering for a dot release too?
Flags: needinfo?(ckerschb)
(In reply to Ryan VanderMeulen [:RyanVM] from comment #25)
> Christoph, can you please weigh in here? If there's a fix on 53 already,
> might be worth considering for a dot release too?

Ok, so this is complicated because there are so many bugs involved. But I can try to summarize. The problem described here apparently was introduced within FF52, so we would need a regression range to be sure what introduced the problem, but it's totally possible (and also likely) that Bug 1297338 introduced it, which landed in FF52.

I suppose you ni? me because of comment 24. Please note that Bug 1301666 is just some clean up from Bug 1297338 to remove transition variables which were needed to allow session restoration between version updates of Firefox. Uplifting Bug 1301666 is definitely *not* desired.

As described in Comment 19, the problem is not reprodruceable anymore on nightly 55.0a1 or beta 53.0b1. The comment further indicates that Bug 1338009 resolved the problem. It's very likely that that is the case but to make things more complicated I don't think that uplifting Bug 1338009 by itself fixed the problem, but probably a combination of uplifting Bug 1341754, Bug 1337622, in addition to Bug 1338009 resolved the problem. Please note that all three bugs were already uplifted to 53.

I don't think there is anything we can do FF52 at the moment. But we should make sure once again that everything works on FF53 again.
Flags: needinfo?(ckerschb)
Kamil, Matt, can you do me a favor and help verify that this bug is not reproducable from 53 onwards anymore?
Flags: needinfo?(mwobensmith)
Flags: needinfo?(kjozwiak)
Christoph, any idea why the bug persist even after downgrading to 51.0.1?
For me on MacOS 53.0.b2 it's back to normal...
> I don't think there is anything we can do FF52 at the moment.

What about the people using 52ESR?  Are they stuck with the bug?
Christoph and I have been discussing ESR52 offline. I wouldn't say that possibility is dead yet :)
> I don't think that uplifting Bug 1338009 by itself fixed the problem

Why not?  It sounds like we're failing to deserialize the principal, throw an exception, and then never get to deserializing the favicon information.

Bug 1338009 made it so at least deserializing the principal fails in a way that doesn't block other things from happening, no?  Have we tried just checking whether adding that patch makes an upgrade from 51 to 52 work?  Or does it need to be an update from earlier than 51 to trigger this issue?

Given that this permanently corrupts the sessionstore, afaict, it might be worth pushing out a point release with an update for people who haven't updated to 52 yet (or would it fix things even for people who did update?  I guess that depends on whether we're saving the corrupted "no favicon" state back or not).  And yes, that includes the ESR thing.
Flags: needinfo?(ckerschb)
I'm having a hard time reproducing in Fx52. This is working for me. So, I'm afraid that unless I spend more time on this, I can't confirm anything has been fixed. Maybe Kamil will have better luck?
Flags: needinfo?(mwobensmith)
If the sessionstore is corrupted I take it that means I'll never get back the favicons that are missing even when the fix arrives in Fx53 without individually reloading each tab, correct? I'm just trying to weigh my options, given the impact this bug has on usability, with my over 250 tabs.
Whether it's actually corrupted on disk is one of the questions that needs answering here...
I'm upgraded to 53.0.b2 and favicons restored without reloading each tab (~600tabs).
Ok then I guess the fix for now is to switch the beta branch if I don't want to wait.
Nobody ever replied why going back to 51.0.1 didn't fix the problem.
(In reply to Boris Zbarsky [:bz] (still a bit busy) (if a patch has no decent message, automatic r-) from comment #32)
> > I don't think that uplifting Bug 1338009 by itself fixed the problem
> 
> Why not?  It sounds like we're failing to deserialize the principal, throw
> an exception, and then never get to deserializing the favicon information.

I guess you are right. The reason I was skeptical is the fact that quite so many dependencies landed in the meantime that rebasing the serialization bug (Bug 1338009) might be a hard undertaking, but definitely worth a try.
Flags: needinfo?(ckerschb)
Going 51->52, opening some new tabs, then going back to -> 51. Old tabs now shows with favicon, but only few without(probably new, opened in 52).
(In reply to ksb from comment #39)
> Going 51->52, opening some new tabs, then going back to -> 51. Old tabs now
> shows with favicon, but only few without(probably new, opened in 52).

Not exactly. I have always the same 80-90 tabs open and tabs icon disappear with time... I already reloaded all tabs (right click a tab and select reload or refresh, I don't know exactly as my version is in french) and you close FF and open it again and some are missing.

Anyway I plan to install 53 beta 2 when I get home to get rid of this. And get on stable release once fixed.
Installed 35 beta 2 and finally all tabs icon are back without even having to reload any.

*** BUT! ***

Unfortunatly there seems to be a problem with Gmail and authentification. Is 53 the version that makes X-Notifier obsolete? Author wrote: "Current version is based on XUL/XPCOM. It will be deprecated near future."

I kept the old version because the new ones called Neo is not as useful, scripts to check other email systems does not work.

Because of this I tried to revert back to 51.0.1 but it's worse, all tabs have lost their icon. If I reload a tab to get the icon, close and open firefox again, the icon is not there.

Please tell me how to revert back to 51.0.1 like before when everything was perfect. I think once I get it working like before I will stay there for a while because I need the X-Notifier functionality of the old verion.

Wow I think I will repeat what I said in my second post above:

Most annoying update ever. :-(

And this is getting frustrating.
Well back to 51.0.1 with tabs icon working just like before. PHEW!
I had to open a new tab for each and paste the link of previous tab.
Glad I had less than 100.

Now I need to find out if I will be able to update to 53 with X-Notifier still working or if I have to stay at 51.01 forever. XUL/XPCOM anyone?
Still no quick solution or workaround for this one?

Maybe this info will help to get to the bottom of this, or help come up with some kind of workaround... So, I have 700+ tabs, and after F52 install only tabs I've used (loaded) in the last session (while in F51) had favicons shown. All other favicons, in the tabs I've not loaded during the the previous session before upgrade, were not displayed. 

But, if you go and "Bookmark All Tabs...", in F52, all the favicons are displayed in the bookmark folder.

Is there a way we can use those icons and load them in tabs?
Sam, the only fix is to open a new tab for each and paste the link of previous tab so you get a fresh new tabs that will behave correctly. I understand 700+ tabs is a huge amount of tabs and that it will take you a full day but that's the only solution I know. I don't understand how you can have that much tabs open? Nevermind.

Funny thing is that they released 52.0.1 and didn't bother to include a fix for this.
Couldn't a GreaseMonkey script automate the tab "recreation"?
Probably yes but I'm not familiar with GM scripts. I had to ask a friend once to do a little scripts once, that took him 10 minuyes while I would have needed a week with a lot of reading and testing to get it right.
(In reply to Christophe from comment #15)
> Created attachment 8846447 [details]
> sessionstore.js
> 
> Here is a sessionstore.js file showing the problem.

Would you mind telling us what is the symptom or the signature characteristic of a sessionstore.js that has these corrupted favicon properties in it?

I ask because I have multiple old (and very large) sessionstore.js (or recovery.js or sessionmanager equivalents) available from backups, and I would like to be able to tell which ones of them will produce good results if I go back to 51.0.1 (or skip forward to 53.0b4, say).

I did run json_pp prettyprint on your session file but I have a hard time telling what the buggy poperties are.
(In reply to rhialto from comment #44)
> Sam, the only fix is to open a new tab for each and paste the link of
> previous tab so you get a fresh new tabs that will behave correctly. I
> understand 700+ tabs is a huge amount of tabs and that it will take you a
> full day but that's the only solution I know. I don't understand how you can
> have that much tabs open? Nevermind.
> 
> Funny thing is that they released 52.0.1 and didn't bother to include a fix
> for this.

Thanks for the reply! Yeah, 700+ don't ask... :D As you said, nevermind that. :)
So, reloading every tab again is not an viable option, yet. ;)

Yeah, too bad 52.0.1 still have this issue, though someone here said 53 doesn't have it, but some other things are not working in 53 regarding Gmail... and I don't want to mess that up, as I'm logged in 7 Gmail accounts (5 use actively). Yeah, don't ask; nevermind that. :D

Anyway, I was hoping we could use the fact that when bookmarking those tabs without favicons in 52, it would bookmark them correctly (*with* favicons, even though tabs don't show them).
So favicons are still there, just not loaded in tabs.
Is there a way we could use the same source of the favicons, and refresh tabs with that info on the next Firefox52 restart?
> Funny thing is that they released 52.0.1 and didn't bother to include a fix for this.

52.0.1 was a security-fix-only release to fix the problem reported to us in the course of the pwn2own competition.

And we don't _have_ a fix for this bug yet for the 52 branch, unless I'm missing something.
(In reply to reikred from comment #47)
> (In reply to Christophe from comment #15)
> > Created attachment 8846447 [details]
> > sessionstore.js
> > 
> > Here is a sessionstore.js file showing the problem.
> 
> Would you mind telling us what is the symptom or the signature
> characteristic of a sessionstore.js that has these corrupted favicon
> properties in it?
> 
> I ask because I have multiple old (and very large) sessionstore.js (or
> recovery.js or sessionmanager equivalents) available from backups, and I
> would like to be able to tell which ones of them will produce good results
> if I go back to 51.0.1 (or skip forward to 53.0b4, say).
> 
> I did run json_pp prettyprint on your session file but I have a hard time
> telling what the buggy poperties are.

I don't know exactly what's wrong but all the needed data are still here.
For now I think the easiest workaround is to switch to version 53 on the beta channel ( https://www.mozilla.org/firefox/channel/desktop/ )

In fact I just tried opening a profile back with version 52.0.1 and I have now all my favicon back.
So that may be another workaround: side install version 53, open your profile and then just go back to version 52.

For people who wants to got back to v51 a good thing to know is that firefox seems to do a backup of the sessionstore file on each update in the directory sessionstore-backups in files called upgrade.js-xxx
Yep, installing 53 (Firefox Setup Stub 53.0b4) does fixes the favicons problem. (Gmail accs seems to be working fine too)

Are there any drawbacks using 53, like any known major issues? Should I go back to 52 just to be safe, or I'm good with 53?
(In reply to Christophe from comment #50)

Right, but even 53.0b2 cannot FIX sessionstore.js (or equivalent file) that has been damaged by 52.0. I think you said so yourself, and my experience is the same.

The tip about upgrade.js-YYYYMMDDHHMMSSxxx is a good one, but it may not work if a user (like mysef) went back and forth between versions more than the default 3 times to save such files. I'm digging up some older backups, thoug, from Session Manager.
For those of you experiencing this issue, I've pushed a test build of bug 1338009 backported to Fx52. If you'd be willing to Try it out, that would be great confirmation of what fixed it in Fx53 and make it possible to have a real conversation about potentially fixing this in an Fx52 bug fix release (or at least on ESR52 going forward). The builds will be branded as Nightly, but you can verify by looking at the version number that they're based off 52.

win32: https://archive.mozilla.org/pub/firefox/try-builds/ryanvm@gmail.com-c4095426dd201d44a4db9b4cda918977d25de70c/try-win32/firefox-52.0.2.en-US.win32.zip
win64: https://archive.mozilla.org/pub/firefox/try-builds/ryanvm@gmail.com-c4095426dd201d44a4db9b4cda918977d25de70c/try-win64/firefox-52.0.2.en-US.win64.zip

Thanks!
(In reply to reikred from comment #53)
> Right, but even 53.0b2 cannot FIX sessionstore.js (or equivalent file) that
> has been damaged by 52.0. I think you said so yourself, and my experience is
> the same.

In my case I don't think there is any damage.
Maybe it's just that the format change is not backward compatible with version 51.
I've switched to version 53 and got all my favicons back.
Yesterday I've tried version 52.0.1, and all my favicon works OK, so in my case version 53 "fixed" my sessionsstore.
So I've switched back to version 52, and all my profiles works correctly.
(In reply to Ryan VanderMeulen [:RyanVM] from comment #54)
> For those of you experiencing this issue, I've pushed a test build of bug
> 1338009 backported to Fx52. If you'd be willing to Try it out, that would be
> great confirmation of what fixed it in Fx53 and make it possible to have a
> real conversation about potentially fixing this in an Fx52 bug fix release
> (or at least on ESR52 going forward). The builds will be branded as Nightly,
> but you can verify by looking at the version number that they're based off
> 52.

The test build 52.0.2 seems to fix the problem. 

Will it update automatically to proper final v52 when it comes out?
Installed 52.0.2 and it works like a charm - All the tabs show up with their Favicons. Will post if anything weird comes up. Many Thanks!
> Will it update automatically to proper final v52 when it comes out?

No, it will not.  It's most likely to update to nightly, I think, if it updates to anything at all.  Please don't actually use that build for anything other than just checking whether it fixes the problem, because you run a real risk of getting stuck on an outdated build forever.  I know it's tempting to use it for normal browsing, but please don't.
Depends on: 1338009
I'm pretty sure it won't update at all. As Boris said, please don't use the test build from comment 54 for anything beyond checking to see if it fixes the issue or not. It's a one-off build that's a dead-end from a support standpoint. Thanks again for testing it, though! It's good to know that it worked :)
Can you share what was the problem?
And once again, a request to consider ESR52.
The patch from bug 1338009 was just uplifted for the Firefox 52.0.2 bugfix release planned for later this week. Thanks to those who were able to try out the test build and confirm that it worked for solving this problem. I'm going to resolve this bug now that there's no more work planned for it, but feel free to reopen it if you find that there's still a problem once the 52.0.2 release ships.
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox52: affected → fixed
status-firefox53: ? → unaffected
status-firefox54: ? → unaffected
status-firefox55: ? → unaffected
status-firefox-esr52: affected → fixed
Flags: needinfo?(kjozwiak)
Resolution: --- → FIXED
And to be clear, the patch was uplifted to ESR52 as well.
Nice, I will then update. Thanks for that!

Will have to revalid if 53 is the version that makes X-Notifier obsolete because the author wrote: "Current version is based on XUL/XPCOM. It will be deprecated near future." so anyone know if it's 53 that drop XUL/XPCOM?
(In reply to Ryan VanderMeulen [:RyanVM] from comment #54)

(In reply to Maxim from comment #57)
> Installed 52.0.2 and it works like a charm - All the tabs show up with their
> Favicons. Will post if anything weird comes up. Many Thanks!

Crucial question for me: Can version 52.0.2 (when it comes out) "REPAIR" bad session files written by 52.0? I ask because I have several huge session files that got wrecked by 52.0 and has caused me to spend all my free time on debugging firefox the last several weeks. I had hoped that 53.0b4 would "repair" bad session files, but for me it does not. Does 52.0.2 have better/newer code than 53.0b4 in this regard? WOuld be great if [:RyanVM] could confoirm this.

(I know, I COULD in theory test with the posted mswindows version but I do not possess an mswindows machine with enough memory and cpu to handle these sessions in any reasonable amount of time, so I have to wait for linux release.)
Sorry, I can't say with any certainty what the answer is to that question. I would suspect that if things didn't work with 53.0b4, the results will probably be the same with 52.0.2 since all that was done in this situation was to backport a patch that had already landed for Fx53. There wasn't any real "new" work done. Sorry for the hassle if that ends up being the case. I know how much of a pain it can be when things go crazy :(
(In reply to Ryan VanderMeulen [:RyanVM] from comment #67)
> Sorry, I can't say with any certainty what the answer is to that question.

(In reply to ksb from comment #24)
> As far as I understand, it's all about
> "principalToInherit_base64", "triggeringPrincipal_b64" and
> "triggeringPrincipal_base64" fields in sessionstore.js
> New tabs contain these fields, old tabs doesn't.
> Probably fixed in bug 1301666?

How about the following: If I hacked up a python script that removed all such items from a session file, is it likely that this would make the file work properly again? While I'm at it, may I ask if anyone would care to explain what the meaning of "Principal" is in this context? It probably doesn't hurt to understand at least somewhat what the intent was when the buggy code was introduced in FF 52.0.
(In reply to reikred from comment #68)
> (In reply to ksb from comment #24)
> > As far as I understand, it's all about
> > "principalToInherit_base64", "triggeringPrincipal_b64" and
> > "triggeringPrincipal_base64" fields in sessionstore.js
> > New tabs contain these fields, old tabs doesn't.
> > Probably fixed in bug 1301666?
> 
> How about the following: If I hacked up a python script that removed all
> such items from a session file, is it likely that this would make the file
> work properly again? While I'm at it, may I ask if anyone would care to
> explain what the meaning of "Principal" is in this context? It probably
> doesn't hurt to understand at least somewhat what the intent was when the
> buggy code was introduced in FF 52.0.

I'm not sure that my assumptions are one hundred percent precise.
For sessionstore.js you can do by yourself(fix it if it does not work):
cat sesssionstore.js | python -m json.tool | grep -Ev 'principalToInherit_base64|triggeringPrincipal_b64|triggeringPrincipal_base64' > sessionstore-new.js
> may I ask if anyone would care to explain what the meaning of "Principal" is in this context?

It's an object that captures a security origin.

It needs to be stored in session history because just a URL is not enough to restore a tab with the right security origin, in cases when the URL is data: or javascript: or a few other things like that.  That's why the "principalToInherit" is needed.  The "triggeringPrincipal" is needed to make loads play correctly with adblockers and such: it's part of the load information and is needed to properly "replay" the load.
(In reply to ksb from comment #69)
(In reply to Boris Zbarsky [:bz] (still a bit busy) (if a patch has no decent message, automatic r-) from comment #70)

Hi ksb, it turns out that using the grep method is a bit risky because a datum which is at the end of a list will leave a dangling COMMA at the end of the previous line. While some browser may accept that, jsonlint and some json pretty printers definitely will complain. So to be safe, I read up on the "jq" command line tool and came up with the following:


# recipe for a Session Mananger (SM) file that has 4 lines of non-JSON header (can put the header back in afterwards if desired).
tail -n +5 my.sessionmanager.session | jq 'del(.windows[].tabs[].entries[].principalToInherit_base64)|del(.windows[].tabs[].entries[].triggeringPrincipal_b64)|del(.windows[].tabs[].entries[].triggeringPrincipal_base64)' > my.sessionstore.js

I tried the recipe on one of my botched .session files. But then I got a surprise when I selected the file for loading inside SM/firefox-51.0.1: SM popped a window with the error message

"There was an error decrypting your session/window data.
Most likely the session/window data is corrupted".

Hey Boris, got any opnion on that message? Here is the diff -w between the two files:


diff -w 2017-03-09.11_55.session.win=14.tabs=1022.session.pp 2017-03-09.11_55.session.win=14.tabs=1022.session.pp.stripPrincipal | m

------------------------------------------------------------------------------------------------------
2c2
< name=[ 2017-03-09.11_55.session.win=14.tabs=1022.session ]
---
> name=[ 2017-03-09.11_55.session.win=14.tabs=1022.session stripPrincipal ]
17624d17623
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17633d17631
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17649d17646
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17667d17663
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17685d17680
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17702d17696
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17710d17703
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17729d17721
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17737d17728
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17749d17739
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17757d17746
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17769d17757
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17807,17808c17795
<                      "url" : "about:support",
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY="
---
>               "url": "about:support"
17843d17829
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
17861d17846
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
76680d76664
<                      "triggeringPrincipal_b64" : "ZT4OTT7kRfqycpfCC8AeuAAAAAAAAAAAwAAAAAAAAEYB3pRy0IA0EdOTmQAQS6D9QJIHOl
RteE8wkTq4cYEyCMYAAAAD//////////8BAAAAIGZpbGU6Ly8vaG9tZS9sYXJzL3dnZXQvdXYtaW5kZXgvAAAAAAAAAAQAAAAHAAAAAAAAAAD/////AAAAAP//
//8AAAAA/////wAAAAcAAAAZAAAABwAAABkAAAAHAAAAGQAAACAAAAAAAAAAIP////8AAAAA/////wAAAAf/////AAAAB/////8BAAAAAAABAAAAAQAAAAAAAA
==",
136795d136778
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY=",
136824,136825c136807
<                      "persist" : true,
<                      "triggeringPrincipal_b64" : "SmIS26zLEdO3ZQBgsLbOywAAAAAAAAAAwAAAAAAAAEY="
---
>               "persist": true

------------------------------------------------------------------------------------------------------
> Hey Boris, got any opnion on that message?

Sorry, I have no idea what the sessionstore file format is.
(In reply to Boris Zbarsky [:bz] (still a bit busy) (if a patch has no decent message, automatic r-) from comment #72)
> > Hey Boris, got any opinion on that message?
> 
> Sorry, I have no idea what the sessionstore file format is.

But my question is not about formats. It's not a matter of the sessionstore format. I just posted the diffs so that people could see that I had not done anything wrong, nothing else than removing the triggeringPrincipal_b64 properties (et al, but in reality only one kind in this example). The question is, for anyone that can answer, why does removing all of triggeringPrincipal_b64 cause FF 51.0.1 to say 

"There was an error decrypting your session/window data.
Most likely the session/window data is corrupted".

In other words, who is it that needs triggeringPrincipal_b64? And mosty importantly: What other properties do I have to remove to repair my sessions files? THAT is what I really want to know.

I have searched with google to try and find out what triggeringPrincipal_b64 really does, but I get only 9 matches and they are all either bugzilla threads or directly in the firefox code. I had expected a reference to a plan, a whitepaper or something similar that explains what the developers are trying to do with these new properties that are getting sprinkled into sessionstore.js files. Well, I did I made a little progress in understanding all this newfangled and unfortunately BROKEN stuff. Apparently it has to do with preventing clickjacking. Here are some references for everyone:

# search: firefox security origin
https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy
https://developer.mozilla.org/en-US/docs/Mozilla/Gecko/Script_security
https://blog.mozilla.org/security/2016/11/10/enforcing-content-security-by-default-within-firefox/
http://stackoverflow.com/questions/17088609/disable-firefox-same-origin-policy
https://en.wikipedia.org/wiki/Clickjacking


For the record: the properties being discussed are principalToInherit_base64, triggeringPrincipal_b64, triggeringPrincipal_base64.
Hey mikedeboer - willing to weigh in here (comment 71, comment 73)
Flags: needinfo?(mdeboer)
Hi reikred, I think manipulating the json file directly probably will not work; it'd be more useful to figure out what the source of the problem is together.
Since your story does not imply that this bug is the source of your problems, I'd like to ask/ propose if you can contact me directly (via email, for example) to sort things out most efficiently. What do you think?
Flags: needinfo?(mdeboer) → needinfo?(reikred)
I was running on the buggy 52.0 since it was released and didn't try to fix the session file in any way (no editing it, no trying other versions). Today's update to 52.0.2 fixed the problem completely for me. All tabs have icons again, even those I never loaded in the time using 52.0.{0,1}. I have about 400 tabs in multiple tab groups (from Tab Groups extension).
It even fixed a glitch with Tree Style Tabs extension, where it didn't show tab hierarchies correctly in some group. Without trying to fix those hierarchies, they are back restored properly with today's update.

Thanks, good work guys!

May I set the status to Verified, or do you have dedicated people that are allowed to do so in Firefox?
I'll do it now to avoid some bugspam, but in the future, feel free to do so :). Thanks!
status-firefox52: fixed → verified
I can report that although version 52.0.2 initially fixed the problem I know have seen again that large amounts of favicons are missing  when I had to restart my firefox about 2 days ago.

Whatever the fix was, it turned out not to last and something tripped things up to such an extent that I am again missing 100s of favicons.
Flags: needinfo?(reikred)
I'm still on Trusty

washuu@washuu:~/development$ uname --all
Linux washuu.goip.de 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:40 UTC 2017 i686 athlon i686 GNU/Linux

It solved the problen for me.

You never mentioned your version here or did I miss it?
Flags: needinfo?(reikred)
(In reply to AHJ from comment #79)

Fedora release 23 (Twenty Three)
Linux version 4.8.13-100.fc23.x86_64 (mockbuild@bkernel02.phx2.fedoraproject.org) (gcc version 5.3.1 20160406 (Red Hat 5.3.1-6) (GCC) ) #1 SMP Fri Dec 9 14:51:40 UTC 2016
Flags: needinfo?(reikred)
I have now also tested with FF 53.0. Still missing 100s of favicons from my session. THE SUBJECT BUG HAS NOT BEEN FIXED, OR THERE ARE ADDITIONAL BUGS THAT CAUSE THE SAME PROBLEM.
Flags: needinfo?(ckerschb)
(In reply to reikred from comment #81)
> I have now also tested with FF 53.0. Still missing 100s of favicons from my
> session. THE SUBJECT BUG HAS NOT BEEN FIXED, OR THERE ARE ADDITIONAL BUGS
> THAT CAUSE THE SAME PROBLEM.

:reikred, sorry to hear you are still experiencing that problem. I sent you a private email asking if you are willing to share your sessionstore files so we can assist you better.
Flags: needinfo?(ckerschb)
(In reply to Christoph Kerschbaumer [:ckerschb] from comment #82)
> (In reply to reikred from comment #81)
> > I have now also tested with FF 53.0. Still missing 100s of favicons from my
> > session. THE SUBJECT BUG HAS NOT BEEN FIXED, OR THERE ARE ADDITIONAL BUGS
> > THAT CAUSE THE SAME PROBLEM.
> 
> :reikred, sorry to hear you are still experiencing that problem. I sent you
> a private email asking if you are willing to share your sessionstore files
> so we can assist you better.

Hi Christoph, there is a new wrinkle: After two restarts, about maybe 10 days ago, the missing favicons persisted, and I finally reported here about 4 days ago. Then yesterday I did another restart, still with FF 53.0, and the favicons were back. While that is an improvement, I wonder what happened? Does the triggeringPrincipal_b64 (et al) stuff have an expiration date that causes it to be regenerated or discarded after a certain time? Is that why the favicons came back again? 

The overall conclusion I'm drawing is that in ff 53.0 the session file is still brittle and unreliable. I wish there was a flag to disable/clean/ignore it. I'd rather miss out on clickjack protection (security origin) than have this headache.

PS: My session file is about 60MB, of which about 40MB appears to be clickjack protection. I looked into removing personal information from the session file so that I can post it here, but I don't feel very confident about succeeding with the removal.
(In reply to rhialto from comment #13)
> Well I installed 51.0.1 and to my surprise the problem persist! When I
> restart firefox some tabs icons are missing!
> 
> Anyone else? What's going on? Did 52 installed/changed something that isn't
> restored when installing 51.0.1 over it?
> 
> Most annoying update ever. :-(
> Priority should be raised.

Same for me with reverting to 51.   I went all the way back to 48 and all the favicons now load properly.
Ugh, in firefox 54.0, if the -contentproc subprocess dies (for example, because out of memory, killed by cgroup oom), I get the same problem that tons of favicons get replaced by the grayed-out globe junk favicon. So now AGAIN I cannot easily find some tabs I'm using. It seems that many tab titles also get changed to "New Tab".

What should I do, should I file a separate bug report?
Flags: needinfo?(ckerschb)
(In reply to reikred from comment #85)
> What should I do, should I file a separate bug report?

I believe that was fixed in Firefox 55 with bug 1227630.
(In reply to reikred from comment #85)
> What should I do, should I file a separate bug report?

Is it possible that bug 1227630 already fixed that issue?
Flags: needinfo?(ckerschb)
(In reply to Mike Conley (:mconley) from comment #86)
> (In reply to reikred from comment #85)
> > What should I do, should I file a separate bug report?
> 
> I believe that was fixed in Firefox 55 with bug 1227630.

55.0b3 does indeed behave better so far with respect to favicons, although for some reason I have not had any tab-crashes yet, nor a full -contentproc crash. I'll keep an eye on it and report back if anything interesting happens.

I should mention that I had several other more or less solvable problems migrating a 54.0 session to 55.0b3, including a seemingly infinite loop and a missing -contentproc process. These two problems only corrected themselves after I did a "skip loading session" restart and then another skip loading session restart followed by a full session load from a saved file. Good thing I found a usable backup session file somewhere, because Session Manager strangely saved a file that could not be loaded (syntax error, some missing parenthesis or some such). It seems that when I update firefox these types of things often happen since around 51.0 version or so.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: