1578539 - Upgrade from TB 60.8.0 to TB 68.0 breaks UI / blank UI. Workaround delete global-messages-db.sqlite

Status: NEW

(Thunderbird :: Mail Window Front End, defect, P3)

Description

[Martin]

Attachment: Screenshot 2019-09-03 at 22.05.52.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

I've updated TB from 60.8.0 64bit on MacOS Mojave 10.14.6 to TB 68.0 by downloading the .dmg from thunderbird.net. After replacing the thunderbird.app file, i started it (it verified it) and afterwards i only see a blank TB window where i cannot even open Preferences or Account Settings. Only showing the version for example works.

Starting TB in safe-mode without add-ons also doesn't help.

Only a downgrade helped.

Actual results:

See attached screenshots

Expected results:

It should show my 3 mail accounts and folders/emails

Comment 1

[Benjamin Flanagin]

Just for the sake of testing, did you try it without the darktheme applied?

Comment 2

[Martin]

Attachment: Screenshot 2019-09-04 at 14.39.55.png

I've upgraded from the default theme as you can see, so this doesn't help

Comment 3

[Jorg K (CEST = GMT+2)]

Something in your profile seems broken. Can you try on a new profile? How does that work on Mac? On Windows/Linux you start Thunderbird with -p.

Comment 4

[Martin]

When i start TB's ProfileManager and create a new profile and start with that, everything seems to work, so your guess @Jorg K seems to be right. Is there a way to find out what went wrong with my profile?
I don't really wanna reconfigure every little detail :/ Last TB updates went through fine as well, so something in TB68 have changed here.

Comment 5

[Wayne Mery (:wsmwk)]

Personally I'd go back to v60 before recofiguring a profile.

What do you see in tools > error console just after startup in safe mode?
Perhaps this is Mac-only?

Comment 6

[Martin]

Hi,
i can't even open the Error Console. I started TB68, tried the error console there (didn't open). Clicked "Help" and "Restart with Add-ons disbaled". It asked to make changes permanent but i just continued to use "safe mode" but even here, the Error console could not be opened. I clicked it open in the menu but nothing happens. Just the TB windows is a bit "smaller" in size in the safe made than normal, but no content at all - like in my first screenshot i attached here.
Opening a new profile works and the "new mailserver connection" dialog opens just fine.
Is there anything i could try to get from other error logs that may help?

Comment 7

[Pete Boyd]

I'm seeing this identically to everything reported so far by the original submitter (including it beong OK when downgrade back to 60), apart from this is when going from 60.9 to 68.0 on Windows 7 Pro.

Comment 8

[Martin]

Today i updated TB 60.8.0 to 60.9.0 without any issues. After that i tried to update to 68.1.0 but that failed, so i now have to stick with 60.9.0. Does anyone has a clue or further things i can try out to help you investigate this issue?

Comment 9

[Pete Boyd]

I'm not sure which of these things fixed it but I deleted global-messages-db.sqlite (175 MB) and deleted mail from Local Folders and then the upgrade from 60.9 to 68.1 worked fine, where previously the upgrade from 60.9 to 68.0 was broken.

Comment 10

[Martin]

Perfect tip, @Pete. I made a copy of my Profiles folder and then only(!) deleted the global-messages-db.sqlite (237 MB). After that TB 68.1 started just wonderful!

I don't know if we want to further dig into this issue but this one fixed it for me.

Comment 11

[Pete Boyd]

I have seen this with yet a different person's Thunderbird on upgrade from 60 to 68, and again fixed it by removing global-messages-db.sqlite.

Comment 12

[Pete Boyd]

...and another person's; fixed the same way.

Don't worry I shan't report any more instances, just thought you might be interested in how common place this seems to be.

Comment 13

[Wayne Mery (:wsmwk)]

This was also reported in a support forum?

Comment 14

[WaltS48 [:walts48]]

(In reply to Wayne Mery (:wsmwk) from comment #13)

This was also reported in a support forum?

I haven't noticed any recent reports for Windows or Linux.

Comment 15

[Magnus Melin [:mkmelin]]

To make sure the error console opens, you can start with thunderbird.exe -jsconsole

Comment 16

[Matt]

I have a user in support that this workaround did it for.
https://support.mozilla.org/en-US/questions/1269159?page=3#answer-1265781

I have Queried him as to the size of the file. Would it be of any use?

Comment 17

[Marco Gaiarin]

I've hit this bug.

I'm upgrading TB from 60 to 68 in my organization, and for some users (4 out of 100) i've hit this bug. I can confirm that the fix proposed (remove global-messages-db.sqlite file) works.

Comment 18

[Kai Engert [:KaiE:]]

Just a drive by comment from someone not affected, and having not worked on the related code:

Are the contents of global-messages-db.sqlite just a cache, that can be deleted without harm?

Should TB 68.x automatically delete it on first start, after the most recently used version was 60.x ?

It sounds like global-messages-db.sqlite gets into a state that causes harm with 68.x, but doesn't disturb 60.x. It might be interesting to know the cause, but it also might be difficult to detect it at runtime, and repair only if necessary. Deleting on migration to a major new version is probably easier.

Comment 19

[Magnus Melin [:mkmelin]]

A cache yes, but it's a very expensive thing to rebuild, so we don't want to do it unless really necessary.

Comment 20

[Matt]

Basically this file is fragile, fragile enough that there is a support article on how to rebuild it.
https://support.mozilla.org/en-US/kb/rebuilding-global-database

I am not entirely sure if anyone actually knows what it in there and what consumes the data it contains.

Comment 21

[Magnus Melin [:mkmelin]]

It contains the data that gloda (the global index for message search) will use for searches.

Comment 22

[Bill Dimm]

I've encountered somewhat similar (but maybe different) problems when upgrading from 60.x to 68.3.1 (64-bit) on Linux. My problem was NOT fixed by deleting global-messages-db.sqlite, but it was fixed by invoking thunderbird with the "--ProfileManager" option and selecting the default profile. It continued to work without the "--ProfileManager" option afterward.

Quick overview: Two computers running two different versions of OpenSUSE Linux (kernel 3.7.10-1.45-desktop and 4.11.8-2-default). Using an exact copy of the .thunderbird directory (via rsync), one worked and one didn't until the --ProfileManager option was used.

Specifics: Upgraded TB on the old computer (as root, through Help > About). Launched new TB as a regular user. Everything working as normal. Turned on the dark theme (never used it before). Seems fine. Exited TB. Upgraded TB on the new computer. Copied .thunderbird directory over to the new computer. Launched TB as a regular user and it tried to create a new profile instead of using the existing one. Turned off dark theme on the old computer and re-copied the .thunderbird directory and launched with "--profile" option on the new computer to force it to use my existing profile. Everything came up blank (no mail folders) until I did View > Folders > All (normally I use Unread instead of All) and it then showed the top-level folder for each email account, but no sub-folders (no Inbox, Sent, etc.). Relaunched TB using --ProfileManager, selected the default (only) profile, and everything is fine since (also works if I use dark theme on old computer before copying .thunderbird directory over). I hope that helps someone.

Comment 23

[Bill Dimm]

Attachment: Output from --jsconsole while using --profile to force it to use the old profile

Comment 24

[Alexander]

Attachment: Bildschirmfoto 2020-01-14 um 17.10.07.png

Thunderbird seems stuck when trying to rebuild the global-messages-db.sqlite

Comment 25

[Alexander]

I'm sorry to report that deleting the global-messages-db.sqlite did not work for me :-(
Deleted it, then deleted the Thunderbird app in 'Programs', then copied the new TB 68.4.1 into 'Programs', restarted, and it was stuck in the status above. I'm running Mac OS 10.14.6.
Reinstalling 60.0 (the last one I have at hand) and everything starts fine, global-messages-db.sqlite is being rebuilt as stated above.
I noticed the file store.json.mozlz4 in the 'crashes' folder; if it helps I can upload it as well...
Would be great if this is fixed, obviously I'm missing out on all the recet security fixes...

Comment 26

[winkelement]

(In reply to Alexander from comment #25)

I'm sorry to report that deleting the global-messages-db.sqlite did not work for me :-(
Deleted it, then deleted the Thunderbird app in 'Programs', then copied the new TB 68.4.1 into 'Programs', restarted, and it was stuck in the status above. I'm running Mac OS 10.14.6.
Reinstalling 60.0 (the last one I have at hand) and everything starts fine, global-messages-db.sqlite is being rebuilt as stated above.
I noticed the file store.json.mozlz4 in the 'crashes' folder; if it helps I can upload it as well...
Would be great if this is fixed, obviously I'm missing out on all the recet security fixes...

Can you open the console.app (Konsole), filter for thunderbird and try to start TB 68.4.1 again and see if there are errors? Noticing all *.sqlite files seem 0 KB in size. Looks similar to the issue i reported bug 1609050.
Somehow TB 68.x can't open/write sqlite db files?

Comment 27

[Alexander]

(In reply to winkelement from comment #26)

(In reply to Alexander from comment #25)

I'm sorry to report that deleting the global-messages-db.sqlite did not work for me :-(
Deleted it, then deleted the Thunderbird app in 'Programs', then copied the new TB 68.4.1 into 'Programs', restarted, and it was stuck in the status above. I'm running Mac OS 10.14.6.
Reinstalling 60.0 (the last one I have at hand) and everything starts fine, global-messages-db.sqlite is being rebuilt as stated above.
I noticed the file store.json.mozlz4 in the 'crashes' folder; if it helps I can upload it as well...
Would be great if this is fixed, obviously I'm missing out on all the recet security fixes...

Can you open the console.app (Konsole), filter for thunderbird and try to start TB 68.4.1 again and see if there are errors? Noticing all *.sqlite files seem 0 KB in size. Looks similar to the issue i reported bug 1609050.
Somehow TB 68.x can't open/write sqlite db files?

The console.app reports 'directory services are not configured', I attach another screenshot.

I can report that the Windows version (through Parallels Desktop) 68.4.1. has no issues opening the profile, although connecting to the mail servers is terribly slow (a few minutes) which may be due to the virtual machine (or Windows or whatever). However, after I did access my profile through Windows, even the TB 60.0 Mac does not start anymore (Windows still doing fine). I consider copying back my "old" profile (which worked with 60.0 Mac). Sad, after many years of few (if any) issues with TB...

Comment 28

[Alexander]

Attachment: screenshot to comment #27

to comment #27

Comment 29

[winkelement]

(In reply to Alexander from comment #28)

Created attachment 9121039 [details]
screenshot to comment #27

to comment #27

Remove the filters in the console log, relevant output might not be flagged as "problem" or "error".

Comment 30

[winkelement]

Attachment: Bildschirmfoto 2020-01-15 um 16.09.26.png

Comment 31

[Alexander]

Attachment: console output.rtf

... as requested :-)
I hope; I'm not really a programming expert...

Comment 32

[Alex Ihrig [:Thunderbird_Mail_DE]]

Many similar bug reports in german support forums in the last few days.

Comment 33

[Anje]

This problem is a frequent issue in Support Forum. Deleting the 'global-messages-db.sqlite' fixes.
eg:
https://support.mozilla.org/en-US/questions/1277737
https://support.mozilla.org/en-US/questions/1277164

Comment 34

[Alexander]

(In reply to Anje from comment #33)

This problem is a frequent issue in Support Forum. Deleting the 'global-messages-db.sqlite' fixes.
eg:
https://support.mozilla.org/en-US/questions/1277737
https://support.mozilla.org/en-US/questions/1277164

I'm sorry Anje, it did nor help for me and as it seems for some others, see my explanations above.
As much as I am happy for those who have it runnoing now, I can assure the issue is not fixed.

Comment 36

[Alexander]

Attachment: ... still blank screen ...

tried the 68.5.0 (also with deleting the 'global-messages-db.sqlite') - still no success (screenshot)
TB 60.0 still working (luckily...)

Comment 37

[Anje]

Alexander:
Have you tried resetting the default profile?
Help > Troubleshooting Information
click on 'about:profiles' which opens in a new tab.

It is possible the current default profile is a new profile.
If your original profile is listed below the default one, you will see a 'set as default' button below the profile you want to use.
Click on that 'set as default' button.
Click on 'Launch profile in a new browser' - it does not open in a new browser, the button label is due to be corrected.
Close the first instance of Thunderbird.

Comment 38

[Alexander]

(In reply to Anje from comment #37)

Alexander:
Have you tried resetting the default profile?
Help > Troubleshooting Information
click on 'about:profiles' which opens in a new tab.

It is possible the current default profile is a new profile.
If your original profile is listed below the default one, you will see a 'set as default' button below the profile you want to use.
Click on that 'set as default' button.
Click on 'Launch profile in a new browser' - it does not open in a new browser, the button label is due to be corrected.
Close the first instance of Thunderbird.

Good idea, unfortunately my correct profile is listed as default, see screenshot attached below.
But I really appreciate your efforts and ideas, many thanks!!!

Comment 39

[Alexander]

Attachment: Profile page

Comment 42

[Piotr Biesiada]

Deleting global-messages-db.sqlite file does not help in 68.6.0 x64. Still the icons in message list are not visible.
I suspect problem may be in *.msf files or while upgrading TB from version to version.
Question is what has changed about message list view or global-messages-db.sqlite-journal file (or even *.msf files) treatment between last few versions.
After deleting global-messages-db.sqlite-journal and waiting several minutes (~30) to rebuild the file, then restart TB - didn't helped.

Comment 44

[Piotr Biesiada]

Can someone please check if the problem with not visible icons in message list is related to turned on addon "Remove Duplicate Messages (Alternate)"?

I've turned off this addon and the message list has all icons. Restarting TB gives icons, so seems like this addon is spoiling the message list icons - at least for me.

Comment 45

[Eliot Moss]

I have that same addon ("Remove Duplicate Messages (Alternate)" and deleting the global-messages-dq.sqlite file fixed the problem. Just disabling addons did not help. Note: it is .sqlite that I deleted, not .sqlite-journal. And yes, it is a pain to rebuild (mine is 2.5Gb - I have a lot of mail!).

Comment 46

[Piotr Biesiada]

I am not sure if it is related to this bug but the icons in folder pane are missing. Installing TB once again (overwrite) does bring back the icons for few TB restarts.

Comment 47

[Piotr Biesiada]

Attachment: Missing icons in folder pane

Comment 48

[Wayne Mery (:wsmwk)]

There have been enough reports to confirm this as a probable bug. But the exact cause is still unknown. And another report is
https://support.mozilla.org/en-US/questions/1280487

Regarding "directory services are not configured" does comment 31 help or can we determine whether that warning is relevant? (perhaps opening of global-messages-db.sqlite failed? Not much in https://www.google.com/search?client=firefox-b-1-d&ei=-5yqXoimNqupytMP_dqzEA&q=%22directory+services+are+not+configured%22+-%22active+directory%22&oq=%22directory+services+are+not+configured%22+-%22active+directory%22&gs_lcp=CgZwc3ktYWIQAzoECAAQRzoGCAAQFhAeUOORAljF9gJgufoCaAJwAngAgAFxiAGXDpIBBDIzLjGYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab&ved=0ahUKEwjIj4rP7Y_pAhWrlHIEHX3tDAIQ4dUDCAs&uact=5

Comment 49

[Wayne Mery (:wsmwk)]

found quite a few more reports - listed in user story

Comment 50

[Xuân Baldauf]

This problem (in case renaming or deleting the file "global-messages-db.sqlite" does not help) is the following:

Analysis

In https://hg.mozilla.org/mozilla-central/file/e7bb5cb77c6caea59d8c6dc958edf7fe8d14aea3/browser/components/customizableui/CustomizableUI.jsm#l417, the call

  Services.prefs.getComplexValue("browser.menu.showCharacterEncoding", Ci.nsIPrefLocalizedString)

is attempted but fails, because the preference browser.menu.showCharacterEncoding is unset (both in the default preferences and in the user profile's preferences).

Workaround

A workaround is:

1. Open the preferences editor in Thunderbird (see https://support.mozilla.org/en-US/kb/config-editor),
2. Add a new preference of type "String" with the name "browser.menu.showCharacterEncoding" and the value "true". (Value "false" should also work.)
3. Restart Thunderbird.

Conclusion

With that workaround working, it will not be difficult for Mozilla software developers to pinpoint why it did work before, why it does not work now (for certain users) and what kind of testcase needs to be written in order to prevent any future regressions.

Comment 51

[Alexander]

Thank you for looking into this, but the workaround listed above did not work for me.
In TB 60 I opened the preferences editor to add the string above (copy/paste from above with value "true"), restarted TB 60, still working.
Then installed TB 78.0.1, started it, blank page :-(
Did I miss something? Should I have deleted the "global-messages-db.sqlite", or some other (hidden?) files before?
It's always quite time-consuming to back-up and restore the profile and all (hidden) settings, so I'm willing to help if there is some valid idea but I'm not too much into trial-and error...

Comment 52

[Matt]

I suggest you try 78.1, I got it to load a working user interface and it did not work in 78.0.1

Comment 53

[Alexander]

Attachment: screenshot with TB 78.1.1

Today I tried 78.1.1, still no success. After first attempt I also renamed the "global-messages-db.sqlite", lots of HD activity (probably trying to recreate it but without success = file size 0 after 10+ minutes when HD activity had ended for some minutes already), but again no change, see pic attached (incl. added prefs line).

Comment 54

[Piotr Biesiada]

(In reply to Alexander from comment #53)

Today I tried 78.1.1, still no success. After first attempt I also renamed the "global-messages-db.sqlite", lots of HD activity (probably trying to recreate it but without success = file size 0 after 10+ minutes when HD activity had ended for some minutes already), but again no change, see pic attached (incl. added prefs line).

What happens if you also try to remove .msf files beside global file?

And rebuild takes very long time - you can monitor it.

Comment 55

[Alexander]

(In reply to Piotr Biesiada from comment #54)

What happens if you also try to remove .msf files beside global file?

Nothing = no noticeable change in behaviour

And rebuild takes very long time - you can monitor it.

Understood, however when I tried it once on TB60 it took about 50 minutes and the sqlite file kept growing in size during that time.
Here I notice just some seconds harddisk activity and the new sqlite file stays at file size 0, even after 20 minutes (also the activity monitor just shows bored processor cores, no HD activity, etc.)

Comment 56

[Magnus Melin [:mkmelin]]

(In reply to Xuân Baldauf from comment #50)
Thanks! I think this could be a good clue.

Comment 57

[Magnus Melin [:mkmelin]]

Filed bug 1659946 about that. This bug, about global-messages-db.sqlite seems different.

Comment 58

[Alexander]

After installing 78.1.1 and starting it with Profile Manager and creating a new profile, I ran into the same issue again. Therefore I believe what I'm running into here is actually defect bug #1612414; the description is much overlapping. Might be the same cause for all where deleting the "global-messages-db.sqlite" did not help?

Comment 59

[Wayne Mery (:wsmwk)]

Has anyone seen this in the past year? ... if so, state the version.
Or have a record of what was seen in the Error Console

From bug 1600222
JavaScript error: resource:///modules/gloda/datastore.js, line 1510: TypeError: this.asyncConnection is null

Comment 62

[Matt]

(In reply to Wayne Mery (:wsmwk) from comment #59)

Has anyone seen this in the past year?

It appears to have been having a resurgence in support. Or at least the description that the page is blank is having a resurgence. I have not looked into details.

Comment 63

[Wayne Mery (:wsmwk)]

Thanks. To clarify, we need to determine whether the specific cause of this bug is still a factor to be resolved. I don't see this as correlating strongly to version 115 issues.

(My intention in setting this as blocking bug 1860253 was not to imply that the cause of this bug correlates directly to the new bugs of version 115)

Comment 64

[Alexander]

After receiving the change notifications, I tested once again if my original issue was solved (opening TB profiles stored on a (Synology-)NAS), and SURPRISE: it worked again!!! Since my Mac is old, I can only confirm for Mac OS 12.7.2 with TB 115.6.0, so it would be helpful if others can test and confirm as well for modern Macs!

Comment 65

[Wayne Mery (:wsmwk)]

Vlad, can you test whether this occurs, by making a small text file named global-messages-db.sqlite? And is there anything in error console?

Comment 66

[Ramona Gavrilescu]

Hello Wayne!

I have tested using the step you specified above on Mac Sonoma 14.4 for TB 115.11.0 and everything works, no errors in the console.