Closed Bug 474717 Opened 16 years ago Closed 15 years ago

Crash when big folders nested

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: mitra_lists, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: crash, stackwanted) 

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20090121 Shredder/3.0b2pre

I had just gone through and rebuilt the MSF files on some old mailboxes, now that Shredder has had the bug with indexing supposedly fixed (Bug 472446).  Shredder ran fine after I'd gone to each box and allowed it to rebuild. BUT when I restarted my machine Shredder failed to start - and the Crash Reporter crashed while reporting hte bug.

I pulled all the old folders out of the Profile and restarted (no problem) and then put them back in one-by-one. 

It seems that the problem is when they are nested - I have a nested hierarchy with a lot of files and about 5 levels deep. If I place this at the top (undereath the folder with hte name of the server) it works fine, if I put it one more level down Shredder crashes when starting. 

I'm moving this in Finder, with Shredder NOT running, and its repeatable. I can't get the crash report since Crash Reporter crashes ! 

Reproducible: Always

Steps to Reproduce:
1. Exit Shredder
2. Drag hierarchy into a .sbd directory one level further down
3. Restart Shredder
Safe mode?
Severity: normal → critical
Keywords: crash
Version: unspecified → Trunk
(In reply to comment #0)
> It seems that the problem is when they are nested - I have a nested hierarchy
> with a lot of files and about 5 levels deep. If I place this at the top
> (undereath the folder with hte name of the server) it works fine, if I put it
> one more level down Shredder crashes when starting. 

Sounds to be problem relates to path length. Check path of all files under mail directory(ls -a is sufficient?)
> http://linux.about.com/od/commands/l/blcmdl1_ls.htm) 
Non-ascii character is used in folder file name/folder name?
Following is example of problem on MS Win.
  - Tb 1.5 : Considered max path length = 254 BYTES
  - Tb 2   : Considered max path length = 254 CHARACTERS(in unicode)
  So Tb 1.5 couldn't process more than 127 non-ascii file path.
  Similar issue may occur on Mac OS X.
AFAIR, file name in file system is Normalization Form D (NFD, Canonical Decomposition) on Mac OS X. So real path length can become larger than application usually expects.
Saved Search Folder(Virtual Folder) is involved? If yes, special character such as "/" is used in virtual folder name?
Sorry, I'm not quite understanding you, but will try and answer the questions I think you are asking.

I don't believe total file length is a problem the maximum length is about 90 characters, which even if it was doubled (into Unicode) would be 180. All names are ascii, no funny characters.
> Gecko/20090121 Shredder/3.0b2pre
> Shredder ran fine after I'd gone to each box and allowed it to rebuild.

What is your exact operation of above?
 - Old ".msf" file is manually deleted before restart?
 - If not, "go to each box" mean "left click the mail folder(open via UI)"?
If old ".msf" exists, and if no-left-click and manual rebuild index from context menu, some problems still remain even after land of patch for bug 471130 on Jan 19. See Bug 471682. Bug 471682 comment #37 and Bug 471682 comment #42 is my test result. If restarted before "clean .msf rebuilding", "corrupted .msf status(or lack of ".msf")" keeps even after restart of Tb.
Procedure was ...
Manually delete all .msf (from terminal (unix) window) 
Go to each mail folder (click on folder in folders pane)
Allow it to rebuild.
Quit Shredder
Start shredder
(In reply to comment #5)
> Procedure was ...

And when Crash occurred?
What is Crash Reporter ID for the crash? ( see http://kb.mozillazine.org/Breakpad )
You terminated Shredder after "Rebuild" of all mail folders completed?

What is your "auto compact" related setting?
> mail.prompt_purge_threshhold = true / false
> mail.purge_threshhold        = NNN (KB)
> mail.purge.ask = false (If set to "true" or reset, this entry doesn't appear)
Crash occurred when I started Shredder (after Comment #5) 

No Crash Reporter ID was sent - as I have reported here, and in other bugs, Crash Reporter crashes filing the report.

Yes - I terminated Shredder after Rebuild had completed, or at least I believe I had, I was using down-arrow to go through folders rapidly (since there is no way to reindex all folders). 

mail.prompt_purge_threshhold = false
mail.purge_threshhold        = 100
mail.purge.ask = true
(In reply to comment #7)
> Crash occurred when I started Shredder (after Comment #5) 
> 
> Yes - I terminated Shredder after Rebuild had completed,
> or at least I believe I had, I was using down-arrow to go through folders rapidly (since there is no way to reindex all folders). 

I guess that not all ".msf" was rebuilt completely when you terminated Tb.
When terminated;
  (a) Some ".msf" files are rebuilt successufully
  (b) Some ".msf" files are during rebuild-index process
      (rebuild-index of child folders are initiated)
      (while parent's rebuild-index is in progress?)
  (c) Some ".msf" files still don't exist
When restarted;
  (d) Child folders of (b) possibly produce internal errors.
Because you disable "auto compact", I suspect issue in "rebuild index" or issue in "out of date" .msf handling. For example;
  .msf of parent folder is "out of date" -> but rebuild index of child folder
  is initiated, because parent's .msf exists -> parent's rebuild index will be
  invoked, because parent's .msf is "out of date"
Watch progress of Bug 471682, please.

I'll try quick test with nested folders of no ".msf", by left-clicking(folder open) many folders quickly.
Thanks Wada,

I think these bugs need to be broken down sometimes into 
a) what causes the corruption
b) why can't TB recover from it. 

It *does* sound possible that a quit during a rebuild caused the problem, but that wouldn't explain.
1: Why Shredder crashes rather than recovering from corrupt index
2: Why I can repeat the problem again by moving the folder (with now good indexes) from one position to another.

- Mitra
(In reply to comment #0)
> Crash Reporter crashes

Mitra, if you crashed AFTER clicking reporter's submit or restart you might be seeing bug 461702. I find no other bugs about reporter dying, so if it's not that  please file a critical bug under product toolkit, component breakpad integration and cc: me.
Hi Wayne

Crash Report crashes WITHOUT offering any dialogue, I know it was called because it's icon appears in the Task Bar, and the APPLE report says it crashed. 

I reported this - and posted an Apple trace (there is no Crash Report produced, i.e. its not just failing in sending it) in Bug 474091 and posted the Apple trace in Comment #3.  I reported again in Bug 474144 ,
(In reply to comment #9)
> but that wouldn't explain.
> 1: Why Shredder crashes rather than recovering from corrupt index
> 2: Why I can repeat the problem again by moving the folder (with now good indexes) from one position to another.

You are right. My comment #8 is merely a my guess with no foundation. It's simply based on your next description.
> if I put it one more level down Shredder crashes when starting.

Healthy check of mail folder files of "one more level down" should be executed.
1. Say "one more level down" is files/directories under xxx.sbd
   (sub folders under parent folder named "xxx")
2. Create dummy POP3 account(no Global Inbox use). Say accountX.
3. Create mail folder of "xxx" under accountX => "xxx" and "xxx.msf" is created
4. Terminate Tb.
5. Copy back up of "one more level down"(back up of xxx.sbd)
   to "xxx.sbd" under mail directory of accountX.
   => Set of "xxx", "xxx.msf", and "xxx.sbd" is used.
6. Restart Tb, and click(open) each folder/subfolder, step by step.
Mitra, with beta 4, does Mac crash reporter still die?
I haven't seen a situation where it crashed recently, if I do I'll report it here again.

Typically it crashed when the database got corrupt, which I haven't seen recently.

- Mitra
In that case, let's close this WFM. If you see again, please reopen
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago
Keywords: stackwanted
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.