Closed Bug 893786 Opened 11 years ago Closed 9 years ago

summary folder not found, and/or "Unable to load address book file abook-13.mab. It may be read-only, or locked by another application"

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Version:
24 Branch
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: mitra_lists, Unassigned)

References

Details

(Whiteboard: [closeme 2015-08-20]) 

Earlybird is complaining at startup. about the summary folder not being found, (either a disk problem or path is too long). 

It then loads up with no folders shown in either of my iMap accounts, and proceeds to download all the messages in the folder (a problem since I'm tethered on a cellphone much of the time). It then appears to work ok, except if I quit and restart the same thing happens.

I'm thinking to restore a backup but wondered if someone wanted to help track it down first I'll delay as Thunderbird shouldn't really be unable to rebuild a good folder in a case like this.
Can you attach the exact error message?
I am not sure what is the summary folder.
Hi - sorry I couldn't wait on this one, it was making it impossible to work so I unsubscribed and re-subscribed to the account in question.
Actually - got it, one of the other accounts was also showing the error. 

Alert: Unable to open the summary file for Inbox, Perhaps there was an error on disk, of the full path is too long. 

More info - it looks like the disk filled up at some point, something or other used tonnes of swap space, which of course was released when whatever it was exited, but I think TB might have been running on a full disk at one point. 

Whatever caused it, inability to recover from a corrupt file - especailly with iMap where the messages are actually all still there on the server - should probably be a problem (it was one of the big weaknesses of the early versions of Outlook).
I'm also seeing: 

ALert Unable to load address book file abook-13.mab. It may be read-only, or locked by another application. Please try again later.
Is the disk free now?

Can you see the Inbox.msf file for that account with error and see in what state it is? Does it exist? Is the size 0 or more?
The INBOX.msf (all caps first art of name) file is there, and is being written to as it rebuilds the file, I'm not sure what you mean by "what state it is" how would you like me to look at it ? Its not zero-bytes currently though its just been rebuilt, next time I see the problem I'll check the size before I continue.

I notice that this is occcuring even on the two accounts I removed and recreated, i.e. something is causing the problem to recur. 

I don't have a disk space problem any more, so my suggestion in #Comment 3 was probably wrong.
Ok - the INBOX.msf file is missing when TB opens and sees the bug. The Inbox is there and about the right size (though once I say OK to the dialog it will download all the contents again) rather than just rebuild the .msf file.
And do you have proper permissions to write into that filesystem directory? I think you said the account was IMAP so the Inbox "being the right size" means you have the Inbox set for offline use (synchronize) and the whole folder is downloaded?
Sorry- I keep missing your responses because with this bug it doesn't alert me if there are new messages in the folder these get filtered to. 

Yes - IMAP
Yes - the whole inbox is downloaded 
Yes - I have permissions - (which is confirmed since it rebuilds the MSF file each time I click through the dialoge (and re-downloads the entire folder))

This is still happening on a regular basis - I've run disk utilities etc to check I don't have disk corruption problems or anything like that.
(In reply to Mitra Ardron from comment #9)
> This is still happening on a regular basis - I've run disk utilities
> etc to check I don't have disk corruption problems or anything like that.

If your "summary folder not being found" is "write open failur of abc.msf file for mail folder named abc", it can easily be observed by;
(1) By other software, open(read exclusive and/or write exclusive) file named abc.msf, while Tb doesn't open mail folder named "abc".
(2) At Tb, try to open mail folder named "abc".
    => Tb fails to write open of file named abc.msf

No interfare by your disk utilities use?
I'm sorry WADA but I dont understand your english so I'm guessing. 

As I said in Comment #7, in response to aceman, the .msf file is missing when this bug occurs.  It doesn't happen all the time - the file is there and being updated at the moment. But when - at starting TB about 50% of the time - I see the dialogue about missing summary folder, then the file is missing.

My file system is fairly full, but not completely full - is it possible that some test for a file system is causing an error and not writing the file on exit?
(In reply to Mitra Ardron from comment #11)
> I'm sorry WADA but I dont understand your english so I'm guessing. 

(Off-Topic)

I'm sorry Mitra Ardron but I usually can't understand your many *BUG* report in pretty accurate English at BUGZILLA.MOZZILA.ORG where it should be place to report actual "bug of Tb === flaw in program code of a software called Thunderbird" to developers of softwara called Thunderbird.

Please note that here is not support forum or Help Center to report funny phenomenon what you saw during your Thunderbird use to someone else.
What kind of "problem determination work" did you do before open *BUG* at bugzilla.mozilla.org where should be for developers, in order to solve Tb's actual bug?
Why following can happen if you are actually trying to report Tb's bug to developers? 
> Sorry- I keep missing your responses
"Interfere by disk utility etc." can produce "delete of .msf and/or offline-store file(if IMAP)/mail data ile(if local mail folder)".
Bug 674742, dup of that bug, and bugs in dependency tree, are examples such problem.
If auto Compact fails due to disk full, auto Compact perhaps silently fails, and auto Compact will be tried multiple times. This kind of issue may produce contention of .msf file. See bugs listed in dependency tree for meta Bug 498274 for such problems.

If Tb deletes .msf, it can be tracked by system utility for file access(for example, Process Monitor on MS Win).
If "This is still happening on a regular basis", why such trace data is still not get and checked by you?
Auto compact is unlikely to be the problem because it prompts before compacting, and there is no correlation between compacting and this problem.

The disk is fairly full but not to the point where its generating errors now. (e.g. today I have 8Gb free, and the file is only 120Mb. 

I don't know how to setup any tool that would watch the deletion of this file. I'm on a Mac and familiar with using terminal, but haven't heard of something that could watch a file getting deleted.
opensnoop is tool on Mac OS X which is found by Google Search for "mac os x tool file access".
http://osxdaily.com/2011/04/22/monitor-mac-os-x-filesystem-usage-access-with-opensnoop/
If your problem occurs on Inbox frequently, "watching .../Inbox.msf only" is sufficient for first trouble shooting step.

By the way, do you see "suffixed .msf file" such as "Abc-1.msf for Mbox named Abc"?(see Bug 520437). Does your server support "case sensitive Mox name"?(Abc.msf for ABC, abc-1.msf for abc, ABC-2.msf for ABC, can occur. Yahoo! IMAP is an example) Do you see "hexa decimal string in .msf file name? (for example, d35d4f8e.msf for Mbox named #)
In any case, "Modified-UTF-7 of Mbox name != .msf file name used by Tb" can occur. In such case, association between "Mbox name" and ".msf file name" may be lost. This may produce ".msf file not found" condition.
Thanks - I'm running opensnoop now, I very much doubt it is anything other than TB that is accessing it. 

I do not see any suffixed files, or hexadecimal file and am not using any non-ascii characters in mbox names 

Yes - the server allows case sensitive mailboxes, the case on my Mac matches that on the server for the files, though there aren't any .msf folders on the server, and of course "INBOX" doesn't have a corresponding file on the server. 

I think that answers all your questions?
(In reply to Mitra Ardron from comment #16)
> I do not see any suffixed files, or hexadecimal file
> and am not using any non-ascii characters in mbox names 

It's pretty good.

> Yes - the server allows case sensitive mailboxes, (snip)

In IMAP, Mbox named "INBOX" is mandatory Mbox and is always case insensitive at any place(spec of IMAP).
"Possible problem with case sensitive mailboxes" may depend on case sensitiveness of file system where Tb is used, and it perhaps can occur only when "Mboxes of different case" is used.
                               File name used by Tb
  Mbox name at server   Tb on MS Win         Tb on Linux & Mac OS X
  (case sensitive)      (case insensitive)   (case sensitive)
  Abc                   Abc.msf              Abc.msf
  abc                   abc-1.msf            abc.msf or abc-1.msf
  ABC                   ABC-2.msf            ABC.msf or ABC-2.msf
  note-1: In Win, file system itself is case insensitive,
          but keeps case of file name.
  note-2: Associated file depends on "order of creation" on Win.
          If panacea.dat is deleted and Tb is restarted, following may
          occur on Win, because Mboxes are returned in Alphabetic order
          usually.
                           In Win, ABC==Abc==abc.
          ABC  Abc.msf     In Win, ABC  ==Abc,   then Abc.msf is used.
          Abc  abc-1.msf   In Win, Abc-1==abc-1, then abc-1.msf is used.
          abc  ABC-2.msf   In Win, abc-2==ABC-2, then ABC-2.msf is used.

Do you use "Mboxes of different case" like "Abc", "abc", and "ABC"?
The server is linux, and client is mac.

I don't have any places where I have same mbox in different cases. (e.g. ABC and abc)
Another question about your environment.
- namespace is used by your IMAP server?
  what is "folder hierarchy delimiter" string?
- do you use "IMAP server directory:" setting?
Check Server Settings/Advanced, while Tb opens an IMAP folder.
Namespace - Personal ""  (that's a pair of double quote) Private and Other are both blank. It is set to allow Server Override. 

There is no "folder hierarchy delimiter" string in the interface, but its a Linux server (Dovecot) and the folders are in normal hierarchy, i.e. in the file system (I can login to the server and see anything there).

I don't use IMAP server directory setting 

The settings are the same on both Imap accounts which run on the same IMAP server, but with different account names.
Any more ideas - this is happening all the time,
(In reply to Mitra Ardron from comment #21)
> Any more ideas - this is happening all the time,

You say;
(i) When error occurred, file named Inbox existed, but file named
    Inbox.msf didn't exist.
(ii) once I say OK to the dialog it will download all the contents again

According to (ii), Inbox.msf was properly created by Tb and OS by (ii).
If so, when (i) occurred again, "deletion of Inbox.msf by someone" should be hooked by opensnoop.
No log by opensnoop for "deletion of Inbox.msf by someone"?

IIUC, Mac OS X is far stabilize than System 7, where system crash/freeze/hung/data corruption was pretty ordinal phenomenon... :-)
So I can't imagine reason why Inbox.msf disappears even though no one erased Inbox.msf on Mac OS X.
Is "drive where the Inbox.msf is held" or "directory which holds meta data of Inbox.msf" volatile storage such as main memory?
Or it's USB memory and you pulled off the USB memory before OS/Tb normally writes file meta data/file data to USB memory?

Have you fully checked all items in following error message?
> (either a disk problem or path is too long).

In Mac OS X, there are two kinds of "file"(or "directory"/"folder").
(a) File in Finder
(b) File in file system which is similar to *nix.
    (IIRC, OS X is BSD based)
Actually "no path length problem" in both (a) and (b)?
(for Inbox.msf, 4 bytes longer path length is needed than "Inbox".)
Please note that following may occur.
- "creation and open of file-name.file-extension under .../directory"
  was possible, because full path of the directory is within length
  limit, and file writing via file handle was possible,
  because lowest directory can hold file meta data for file named
  file-name.file-extension.
- Next access to the file using full path is impossible, due to path
  length limitation by OS.

Another concern - number of file handles.
How many Mboxes are defined at your IMAP server?
Other questions.

When did your problem start to occur, with the Tb profile, with the problematic IMAP account defined?
No problem if Tb 17 is used?

Doesn your problem occur with newly created Tb profile, with problematic IMAP account only defned?
(after account definition, kill "Global Searrch ad Indexer", kill auto-sync by
Woops. Too early enter...

Does your problem occur with newly created Tb profile, with problematic IMAP account only defined?
If possible, create Tb profile at different place, for example, /Test/Tb-prof/prof-1, which has pretty short full path length.
Just after account definition, before first server access, kill "Global Search ad Indexer", kill auto-sync by mail.server.default.autosync_offline_stores=false, and restart Tb, for ease of test.
Re your questions in Comment #22
I ran opensnoop for a while but it did not catch it being deleted, which means it was probably deleted by TB 
The drive is the main drive on a MacBook Air, i.e. its solid-state disk

I don't understand the point about .../directory, 
The path to inbox is /Users/mitra/Library/Thunderbird/profiles/bhqffpqt.default/mail/mail.path.net
which is about as short as it can get - I have not moved the profile since it was created, i.e. it used to work at this location. 
This account has about 26 mailboxes; There are another 17 in two other IMAP accounts, maybe 27 in a POP account and a lot in the Local Folders directory (because all the Archives are here).

Re Comment #23
The problem started with no change that I can remember., I believe I reported it the first day I saw the problem (15th July) and I think that day may have been the day I ran out of disk (some other program used up a massive swap partition) , i.e. its possible that something was corrupted. 

I have since then deleted both the IMAP profiles I use and recreated them, and still see the bug. 

I'm using TB 25.0a2 and am on nightly updates (trying to help debug things !) 

Re Comment #24
I'm nor sure how to do what you are asking in Comment #24 without losing all the mail in Pop and Local Folders. Can you give me a series of steps (Unix or Finder) to do what you are suggesting - but also to save out my LocalFolder and Pop mail first, and then restore after testing. 
I also don't know how to kill "Global Search and Indexer" 

Thanks
(In reply to Mitra Ardron from comment #25)
> Re your questions in Comment #22
> I ran opensnoop for a while but it did not catch it being deleted, which
> means it was probably deleted by TB

Because Inbox.msf is already deleted by someone when your problem occurs, "who deleted Inbox.msf when" is most important thing for Trouble Shooting, isn't it?

> The path to inbox is
> /Users/mitra/Library/Thunderbird/profiles/bhqffpqt.default/mail/mail.path.net

On Win, file for Mbox of POP3/Local Folders is created under ...\Mail and file for Mbox of IMAP is created under  ...\ImapMail if default is used in account definition.
Is file for Mbox of IMAP account is created under /mail on Mac OS X?
Or it means that you changed "Local Directory:" of Server Settings?
What is set in mail.server.serverX.directory-rel and mail.server.serverX.directory for IMAP account where your problem occurs?

> Re Comment #24
> I'm nor sure how to do what you are asking in Comment #24 without losing all the mail in Pop and Local Folders. 

I asked you "new profile with the IMAP account only is defined".
Because IMAP, Mbox on IMAP server can be accessed by multiple mail clients, and multiple Tbs can run at concurrently on one PC if -no-remote option is used, although it should be used carefully.

> Can you give me a series of steps (Unix or Finder) to do what you are suggesting

Almost all steps to do is done on Tb, so it's common among OSes.
(1) Create a new Tb profile. Call profXXX, path is / ... /profXXX.
(2) Copy created Tb's profile directory(which holds prefs.js in it)
    to directory(in file system, Folder in Finder),
    for example, /Test/Tb-Profs/profX.
    Because setting in prefs.js is "relative from [ProfD]",
    copied profile directory is usable as "Tb's profile directory"
(3) Create new Tb profile using -ProfileManager parameter,
    call named profX, using "Choose Folder" button which is shown by 
    "Next", and select /Test/Tb-Profs/profX.
    /Test/Tb-Profs/profX is pointed from profX entry in profiles.ini.
    (You can delete profXXX and associated profile directory now.)
(4) Start Tb with the newly created profX,
    and define an IMAP account only.
(5) Before first access to IMAP server, do following.
(5-1) via Tb's Options setting(preference setting),
(5-1-a) Advanced/General, Uncheck "Enable Global Indexer and Search"
(5-2) via Account Settings of the IMAP account
(5-2-a) Server Settings,
        Uncheck all automatic new mail check, purge aetings,
        Choose "just mark it as deleted" fr ease of test,
(5-2-b) Synchronization & Storage,
        Uncheck "Keep messages ..." (Offline-Use=Off is set by this)
(5-2-c) Junk Setting,
        disable Junk filter, Junk move
(6) Terminate Tb, with keeping one of following.
    - "Local Folders" is selected at Folder Pane
    - An folder of "Local Folders" is selected at Folder Pane
    - IMAP account is selected at Folder Pane
      (no IMAP folder is selected at Folder pane)
(7) Restart Tb with profX.
(8) Click Inbox of IMAP account at folder pane.
    => login to server, Mbox list is obtaned from server,
       and all folders are shown at folder pane,
       and associate .msf file to IMAP Mbox is created by Tb.
    => Message header of all messages of Inbox is fetched.
       But mail data is not doenloaded by auto-sync,
       because Offline-Use=Off
    => Nothing is fetched at other Mbox until folder is clecked
(9) Terminate Tb
Because minimum configuration with minimum path length, checking all directories and files by "ls" command, getting IMAP log, checking prefs.js content and so on, is far easier than daily use profile.
And, this minimum configuration can be used as "Reference" to know "what is different of your daily use profile with which your problem often occurs".

If you need to run two Tbs at same time, one Tb with daily use profile and one Tb with profX, "start Tb from terminal with -p prefX -no-remote" is possible.
(In reply to WADA from comment #26)
> > The path to inbox is
> > /Users/mitra/Library/Thunderbird/profiles/bhqffpqt.default/mail/mail.path.net
> 
> On Win, file for Mbox of POP3/Local Folders is created under ...\Mail and
> file for Mbox of IMAP is created under  ...\ImapMail if default is used in
> account definition.
> Is file for Mbox of IMAP account is created under /mail on Mac OS X?
> Or it means that you changed "Local Directory:" of Server Settings?
> What is set in mail.server.serverX.directory-rel and
> mail.server.serverX.directory for IMAP account where your problem occurs?

Good catch WADA.
Yes, if you want to store the mail under the profile folder (/Users/mitra/Library/Thunderbird/profiles/bhqffpqt.default) only ever use the safe subfolders, like Mail and ImapMail. in the recent versions (18+) TB even enforces this and forbids you to set anything else in the Local directory (you can of course still put the mail elsewhere outside profile folder).
Even though I wrote the checking code, it is in JS and I am not yet sure if it is case-sensitive and if "mail" and "Mail" are considered different. I have now tested on Windows and it allows "mail" too (as on Windows they are equivalent).
Thanks for the detailed reply. 

Re Comment #26 
I am waiting for the bug to occur again, my suspicion is that TB itself is deleting it, but I'm running opensnoop to look for it.

My mistake the inbox is at 
/Users/mitra/Library/Thunderbird/profiles/bhqffpqt.default/ImapMail/mail.path.net/INBOX
There was an old mailbox (from before I converted to IMAP in "mail"  ) in that other location.

mail.server.server5.directory-rel;[ProfD]ImapMail/mitra.biz 
mail.server.server6.directory-rel;[ProfD]ImapMail/imap.lumeter-2.net

I haven't tried creating that new profile yet, there are several steps in that process I can't follow and am too concerned about losing my stored email as a result of getting it wrong.
Setting dependency to bug 495911 for ease of problem analysis, tracking, search.
Summary: summary folder not found → summary folder not found, and/or "Unable to load address book file abook-13.mab. It may be read-only, or locked by another application"
Mitra, are you still seeing this?
Flags: needinfo?(mitra_lists)
See Also: → 495911
Whiteboard: [closeme 2015-08-20]
See Also: → 855751
Not since august 2013.  I've set it to "Resolved/WorksForMe", I'm not sure if that is the right status for a bug that just went away since maybe related to other similar problems seen in the two bugs you referenced.
Status: UNCONFIRMED → RESOLVED
Closed: 9 years ago
Flags: needinfo?(mitra_lists)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.