Closed Bug 670566 Opened 13 years ago Closed 12 years ago

Spotlight on Mac not finding ANY messages

Categories

(Thunderbird :: Search, defect)

Product:
Thunderbird
Component:
Search
Platform:
All
Other
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 16.0

People

(Reporter: mitra_lists, Assigned: estesp)

References

Details

Attachments

(1 file, 1 obsolete file)

proposed fix for perms + reindexing
3.36 KB, patch
standard8
: review+
Details | Diff | Splinter Review 
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

I've used spotlight to search for word that I know occurs in multiple messages (e.g. Facebook). I've used this before, but not for a while.  Now zero messages from TB are showing up, though files, and other components (e.g. Address book entries) are showing up fine.   Searching in TB's search field finds the messages fine, so it doesn't appear to be an indexing problem on TB.   Preferences/General/Allow Spotlight is enabled, as i Preferences/Enable Gloda.
This is not mission critical to me, so I'm not going to try and fix this for a while, in case someone wants to work with me to figure out what is broken and maybe what caused it.   BUT if someone knows how to fix it,, then maybe they could post a suggestion.
Do you have a : 
Thunderbird.mdimporter in /Applications/Thunderbird.app/Contents/Library/Spotlight
 ?

If you unset set the  preference - do you get mdimporter process running on your machine ? (if not anything on console.app showing an error )
Only message on error console is:
Error: this._syncInfoPerFolder[folder.URI] is undefined
Source File: resource:///modules/activity/autosync.js
Line: 337
No errors show up when I initiate a spotlight search (but I don't think I'd expect one).
Its /Applications/Shredder.appand the .mdimporter file is there.

I don't understand the last part - which proference (Allow Spotlight, or Enable Gloda) do you want me to unset, and I presume you mean that you want me to look at "ps -eax" for mdimporter but why would this show up when I UNSET either of these preferences.  (I've tried all 4 combinations now, and no mdimporter process appears in ps).
(In reply to comment #3)
> Only message on error console is:
> Error: this._syncInfoPerFolder[folder.URI] is undefined
> Source File: resource:///modules/activity/autosync.js
> Line: 337

Can you file another bug for that one ?

> Its /Applications/Shredder.appand the .mdimporter file is there.

ok good.
 
> I don't understand the last part - which proference (Allow Spotlight, or
> Enable Gloda) do you want me to unset, and I presume you mean that you want

Allow spotlight

> me to look at "ps -eax" for mdimporter but why would this show up when I
> UNSET either of these preferences.  (I've tried all 4 combinations now, and
> no mdimporter process appears in ps).

Yes that's what I meant.

Is spotlight still configured to search into emails ( System Preferences -> Spotlight ) ?

Does rebuilding the spotlight database changes anything ? (warning lengthy and long process)
That error  with autosync is Bug#534602 (you reported it in Dec 2009 , so I guess its not the direct cause of this problem). 

I'm unclear why you'd want me to turn OFF allowing spotlight to search messages in TB ? What am I missing?


>Is spotlight still configured to search into emails ( 
> System Preferences -> Spotlight ) ?

Interesting - there seems no option in Spotlight to search TB messages, I've got #5 that searches Mail Messages (and it appears to do that) but nothing for searching Thunderbird 

I'll hold on rebuilding Spotlight till I hear whether you want me to do anything else to allow it to search, 

otherwise if you know how to rebuild a hint would save me searching for it.
To see if it triggers something. Mail or emails should be the same.

see http://www.wwco.com/~wls/blog/2007/08/29/rebuilding-spotlights-index-on-os-x-manually/ for instructions.
Thanks - I've reindexed and its still not finding TB messages (it finds Apple Mail messages just fine).
Si any more idea on what to look at and look for ?
s/Si/Sid0
Hm, that's strange.
- What is mail.spotlight.enable set to?
- Does ~/Library/Caches/Metadata/Thunderbird exist, and do you see .mozeml emails in them?
- Is that folder or any of its parents excluded from Spotlight search by any chance?
- Try opening up a few of the files in a text editor. Do the files have text in them?
- Try searching for some of the text you saw in the .mozeml files.

This could be a 32/64 bit problem... wonder whether mdimporters now need to be fat binaries.
Also:

- Could you generate the list of mdimporters on your computer with:

mdimport -L > ~/mdimporter-list.txt

and attach mdimporter-list.txt?

- Could you try indexing one of the subfolders of the folder I mentioned with

mdimport -d 4 directory > ~/mdimport-output.txt

and attach mdimport-output.txt?

If you think some of the information in those files could be private, I'm fine if you email the contents directly to me instead.
mail.spotlight.enable;true 

~/Library/Caches/Metadata/Thunderbird exist contains two directories ImapMail and Mail and a hierarchy of directories that seems to mirror folders - there are ONLY directories under here, no files (so none to open up in text editor, and no text to search for)

There is nothing set in Spotlight/Privacy 

Sounds like lack of production of these files is the issue ? do you still want the mdimport questions answered?
That is really weird... what happens if you delete ~/Library/Caches/Metadata/Thunderbird and then restart Thunderbird? We should start regenerating those files.

The contents of mdimport -L would be useful, yes.
It didn't start rebuilding the files, or the directory - it didn't even recreate ~/Library/Caches/Metadata/Thunderbird
I'm still seeing this problem even after upgrading to Lion (which does a complete reindex). 

~/Library/Caches/Metadata/Thunderbird has been created at some point, with subdirectories, but no files (at any level).
I've not seen old messages get recreated if the caches folder is cleared, ever. At the end of bug 482724 (https://bugzilla.mozilla.org/show_bug.cgi?id=482724) you indicated that the older messages were regenerating, but I've not seen them do it - even if I open the older message.

I'm currently running 3.1.11, but Thunderbird.mdimporter is the same code as in 3.1.15 and 7.0.1.

chris
Mitra, ludo, sid, I've tagged relevant topics in getsatisfaction http://getsatisfaction.com/mozilla_messaging/tags/spotlightfail where people could use some assistance

mtepper, ever seen this?
and there is a recent topic in mozilla.support.thunderbird "TB 8 and Spotlight search"
What help to give Wayne - "the bug's been here 5 months, and still marked as Unconfirmed so it probably won't get fixed in the near future :-( 

I've changed status to New since its certainly Confirmed.
Status: UNCONFIRMED → NEW
Ever confirmed: true
 
> mtepper, ever seen this?

I can confirm this behavior (failure to create files ~/Library/Caches/Metadata/Thunderbird) 

I hadn't been using Thunderbird in OS X so I did a fresh install (on Lion 10.7.2), added a Gmail account, and waited for some messages to load. I then performed a spotlight search for messages that I had loaded -- no results. Finally I searched the ~/Library/Caches/Metadata/Thunderbird directory, but found no files. There were .mozmsgs directories each mailbox, it appears the actual metadata files for the messages are not being created.

Here's what's in under ~/Library/Caches/Metadata/Thunderbird/ImapMail/imap.googlemail.com 

$ ls -la
total 0
drwxr-xr-x@ 7 mtepper  staff  238 Dec 10 20:43 .
drwxr-xr-x@ 5 mtepper  staff  170 Dec 10 20:30 ..
drw-r--r--@ 2 mtepper  staff   68 Dec 10 20:30 INBOX.mozmsgs
drwxr-xr-x@ 5 mtepper  staff  170 Dec 10 20:42 [Gmail].sbd

$ sudo ls -la INBOX.mozmsgs/
total 0
drw-r--r--@ 2 mtepper  staff   68 Dec 10 20:30 .
drwxr-xr-x@ 7 mtepper  staff  238 Dec 10 20:43 ..
I recently hit this issue when I took a Mac that had been using Thunderbird and created a new user and cleared out the metadata caches from my copied over user.  When the spotlight indexer ran the Thunderbird plugin it created the '.mozmsgs' directories (as you see above) but it created them without execute bit!  The plugin can't create subfolders or files within the directory because of the missing "x" perms.  If you manually fix each '.mozmsgs' dir as it is created (chmod +x), then over time it will start to fill with the actual metadata content.

I believe if someone looks up the code that is creating .mozmsgs dirs the problem will be found.  I'm currently running Thunderbird 10b2, but initially set this system up with Tbird 9beta{something}.
Although I can't see where/when this was changed (unless the underlying create code used to catch "bad" perms for a directory and do fixups), there are 4 places in searchCommon where directories are created with explicit 0644 perms (no execute bit set).  Seems this is the root cause of not getting "mozeml" files on a fresh installation that didn't already have Library/Caches/Metadata set up.

http://mxr.mozilla.org/comm-central/source/mail/components/search/content/searchCommon.js#374
(also lines 391, 682, and 882)
Phil how would you feel to propose a patch to fix this ?
Whatever patch is released needs to fix the permissions on the bad directories, and somehow trigger Spotlight to reindex them.  

I changed the permissions manually, but spotlight hasn't reindexed.
Here is a patch that addresses the directory permissions.  It seems like a separate issue (tool/script?) to handle fixups to wrongly created metadata directories on currently installed Macs.

One quick fix is to run `find . -type d -exec chmod +x \;' in the Library/Caches/Metadata/Thunderbird directory.  In my case, the indexer did begin to create .mozeml files after that, but I'm not sure if any notes are marked as "indexed" during the initial run that will never be tried again?
Comment on attachment 588157 [details] [diff] [review]
proposed fix for directory permissions

Sid's probably the best person to look at this.
Attachment #588157 - Flags: review?(sagarwal)
Two issues here.

Sure ... developers know how to run that find command (I'd already tried it) but regular users won't. 

I see .mozeml files being created, but not for already existing messages.

I tried the normal trick of adding to Privacy and removing and that did NOT cause spotlight to reindex.
(In reply to Mitra Ardron from comment #27)
> Two issues here.
> 
> Sure ... developers know how to run that find command (I'd already tried it)
> but regular users won't. 
> 
> I see .mozeml files being created, but not for already existing messages.
> 
I'm surprised that it won't index messages that don't have files in the .mozmsgs dirs based on the code in searchCommon.js.  There are only 2 places it looks like it sets the reindex time in the msgHdr, and both of those are within predicated "if file(exists)"--making the simplest interpretation that "if .mozeml file not created, it will have to be indexed at some point!".  I'm not sure how folder reindex time fits into that, but any time something changes in a folder, it should cause the reindex code to walk that folder, and it uses a search messages filter on reindex time > last index time.  For email that never got indexed, they should come back in those search results.  Looks like some debug needs to happen to validate that.  I do have a question about whether the mapping of message ID -> "support file" is working right.  When I was below 100 .mozeml files (after fixing perms), I was deleting notes and then watching what happened in the .mozeml space.  The number would go down by 1, but it would not be the note that I just deleted--so I'm wondering about the integrity of the mapping. (I was able to verify this by having a Spotlight window up with Kind = "Mail Message" and see that the note subject I just deleted didn't go away in the Spotlight search output--but another mail did disappear!)

Here is the code that sets index time on a specific message in searchCommon.js:

1) This first path is after searching a folder for messages to index.. if the .mozeml file exists, it isn't going to try and reindex.  If it doesn't exist, it will go off and return it as a message to be indexed.

  457         if (this._getSupportFile(msgHdr).exists())
  458         {
  459           this._log.debug("Message time not set but file exists; setting " +
  460                           "time to " + reindexTime);
  461           msgHdr.setUint32Property(this._hdrIndexedProperty, reindexTime);
  462         }
  463         else
  464         {
  465           return [msgHdr, reindexTime];
  466         }

2) this second spot is an "observer" which seems to trigger on activity (like message display)..again, if the file doesn't exist, it should add it to the queue for indexing.

   578     else if (aTopic == "MsgMsgDisplayed")
   579     {
   580       this._log.debug("topic = " + aTopic + " uri = " + aData);
   581       let msgHdr = this._messenger.msgHdrFromURI(aData);
   582       let reindexTime = this._getLastReindexTime(msgHdr.folder);
   583       this._log.debug("Reindex time for this folder is " + reindexTime);
   584       if (msgHdr.getUint32Property(this._hdrIndexedProperty) < reindexTime)
   585       {
   586         // Check if the file exists. If it does, then assume indexing to be
   587         // complete for this file
   588         if (this._getSupportFile(msgHdr).exists())
   589         {
   590           this._log.debug("Message time not set but file exists; setting " +
   591                           " time to " + reindexTime);
   592           msgHdr.setUint32Property(this._hdrIndexedProperty, reindexTime);
   593         }
   594         else
   595         {
   596           this._queueMessage(msgHdr, reindexTime);
   597         }


> I tried the normal trick of adding to Privacy and removing and that did NOT
> cause spotlight to reindex.
Comment on attachment 588157 [details] [diff] [review]
proposed fix for directory permissions

Hi Phil,

Thanks for the patch! It looks like a great start.

As Mitra's pointed out, we still need to go back and fix up any incorrectly created directories. So what we need to do is (a) check if a directory didn't have the executable bit set, and (b) if it didn't then set it and force a reindex.

(b) is simple. There's already some code to do that at [1]  -- all you need to do is bump the folder's reindex time as shown with setStringProperty.

(a) is a bit harder, unfortunately.
- On Mac, luckily ancestors would have already had their execute permissions set properly per the code in nsLocalFileUnix.cpp [2], so all we need to worry about are the permissions of the folder itself. So a simple check will work:

  if (folder.permissions != 0755) {
    folder.permissions = 0755;
    ... // force a reindex
  }

- On Windows, as it turns out, the permissions accessor would never have its execute bit set (since directories aren't really "executable" on Windows), so the above check wouldn't make sense. i.e. we need to make it OS-specific. Fortunately we already have a way of doing so -- by including code in WinSearchIntegration.js and SpotlightIntegration.js. So here's what I propose:

1. Add a method to both files called _pathNeedsReindexing(aPath). In SpotlightIntegration.js, do the following:

  _pathNeedsReindexing: function spotlight_folderNeedsReindexing(aPath) {
    // We used to set permissions incorrectly (see bug 670566).
    if (aPath.permissions != 0755) {
      aPath.permissions = 0755;
      return true;
    }
    return false;
  }

In WinSearchIntegration.js, always return false.

2. At [2], add a check. Here's one way it could be done.

...
  if (!searchPath.exists())
  {
    this._log.debug("using folder " + folder.URI + " because " +
                    "corresponding search folder does not exist");
    // Create the folder, so that next time we're checking we don't hit
    // this
    searchPath.create(Ci.nsIFile.DIRECTORY_TYPE, 0644);
    folder.setStringProperty(this._hdrIndexedProperty,
                             "" + (Date.now() / 1000));
    yield folder;
  }
  // Otherwise if the folder needs to be reindexed for some other reason,
  // do so.
  else if (this._pathNeedsReindexing(searchPath)) {
    folder.setStringProperty(this._hdrIndexedProperty,
                             "" + (Date.now() / 1000));
    yield folder;
  }
...

Would you be willing to make these changes? If not, I think I should be able to handle them. In either case, thanks again for the patch.

[1] https://mxr.mozilla.org/comm-central/source/mail/components/search/content/searchCommon.js#382
[2] http://mxr.mozilla.org/mozilla-central/source/xpcom/io/nsLocalFileUnix.cpp#484
Attachment #588157 - Flags: review?(sagarwal) → feedback+
(In reply to Siddharth Agarwal [:sid0] from comment #29)
>   _pathNeedsReindexing: function spotlight_folderNeedsReindexing(aPath) {

Sorry, I meant
    _pathNeedsReindexing: function spotlight_pathNeedsReindexing(aPath) {
This patch adds the path reindexing check suggested in the feedback.  This should appropriately reindex folders after fixing the permissions.
Attachment #588157 - Attachment is obsolete: true
Attachment #589759 - Flags: review?(sagarwal)
Thanks for the patch! Looks great. I've fired off a Mac try build so that people can test it out (Mitra?) http://hg.mozilla.org/try-comm-central/pushloghtml?changeset=40f94954c864
(In reply to Siddharth Agarwal [:sid0] from comment #32)
> Thanks for the patch! Looks great. I've fired off a Mac try build so that
> people can test it out (Mitra?)

I'd be happy to give it a try..have another Mac in the house where I haven't fixed the dir permissions.
Phil or Mitra: have you tried the try build out?
Sorry - I'm in India currently, minimal net access - will have to try late next week when I get back . 

- Mitra
I had downloaded the test build and it was on my todo list all weekend, but didn't have a chance.  I've got it running now and looks like we may still have some issues--or at least there may be earlier spots to integrate the "fixup/reindex" check.

First of all, the observer has yet to get an "idle" message, so the sweep code has never been run..Tbird has been running on the system for nearly 2 hours.  There is not that much activity on the system and nothing appears to be actively performing work in Tbird, but it's not clear to me how "idle" gets defined or triggered in the Tbird framework.

Given the lack of "idleness", all the observer activity has been "msgAdded" or "msgDisplayed", which all continue to fail as the sweep code which would do file perm fixup and reindexing has yet to run.   It seems like another place to possibly integrate the fixup is in the _getSupportFile code (which every note passes through to get the directory + filename), or in the startStreaming code, which is one of the other locations where ".mozmsgs" directories are created [side note: it's actually the "!file.exists()" on searchCommon.js:884 which is the source of the errors as that gets ACCESS_DENIED because of dir perms].

Thoughts?
(In reply to Phil Estes from comment #37)

> directories are created [side note: it's actually the "!file.exists()" on
> searchCommon.js:884 which is the source of the errors as that gets
> ACCESS_DENIED because of dir perms].
> 
Maybe this would be a good place to catch the perm error without adding checks to "success path" code (like the getSupportFile method)? ..excuse the pseudo-code and improper indentation (so you can see the patch easier) for my thinking here [we have access both to the folder object and the file here]:

+        try {
           if (!file.exists())
           {
             try {
@@ -888,7 +889,16 @@
               file.create(Ci.nsIFile.DIRECTORY_TYPE, 0755);
             }
             catch(ex) { this._log.error(ex); }
-          }
+          } 
+        }
+        catch(ex) {  
+          if ex == ACCESS_DENIED
+            //check path needs reindex due to perms failure (bug 670566)
+            if (this._pathNeedsReindexing(file)) {
+               folder.setStringProperty(this._hdrIndexedProperty,
+                                     "" + (Date.now() / 1000));
+            }
+        }
(In reply to Phil Estes from comment #37)
> I had downloaded the test build and it was on my todo list all weekend, but
> didn't have a chance.  I've got it running now and looks like we may still
> have some issues--or at least there may be earlier spots to integrate the
> "fixup/reindex" check.
> 
> First of all, the observer has yet to get an "idle" message, so the sweep
> code has never been run..Tbird has been running on the system for nearly 2
> hours.  There is not that much activity on the system and nothing appears to
> be actively performing work in Tbird, but it's not clear to me how "idle"
> gets defined or triggered in the Tbird framework.

"idle" is as defined by Mozilla's idle service [1], which in turn relies on the OS. The threshold is defined as 30 seconds, which means that if you leave your computer idle for 30 seconds the logic should kick in. If you turn logging on (set mail.spotlight.logging.console to true, then run Thunderbird from a terminal) you should see idle/back messages in the terminal.

That's how it's supposed to work in theory, anyway. I'm not sure if there are any particular issues with the idle service on Lion. I found bug 692225, but I don't think it's related.

[1] https://developer.mozilla.org/en/nsIIdleService


> 
> Given the lack of "idleness", all the observer activity has been "msgAdded"
> or "msgDisplayed", which all continue to fail as the sweep code which would
> do file perm fixup and reindexing has yet to run.   It seems like another
> place to possibly integrate the fixup is in the _getSupportFile code (which
> every note passes through to get the directory + filename), or in the
> startStreaming code, which is one of the other locations where ".mozmsgs"
> directories are created [side note: it's actually the "!file.exists()" on
> searchCommon.js:884 which is the source of the errors as that gets
> ACCESS_DENIED because of dir perms].

Yeah, possibly, though it'll be tricky recursively traversing directories I guess. The reason I liked this fix was because it's pretty much guaranteed we'll be idle at some point.
(In reply to Phil Estes from comment #38)
> +            if (this._pathNeedsReindexing(file)) {

Implementing this is going to be a bit trickier. You're going to have to fix perms recursively, by moving up the directory tree, seeing if you can set perms, then if not then moving further up. This needs to be limited to the Tb metadata directory I think -- if we go further up we risk messing up home directory permissions.
(In reply to Siddharth Agarwal [:sid0] from comment #40)
> (In reply to Phil Estes from comment #38)
> > +            if (this._pathNeedsReindexing(file)) {
> 
> Implementing this is going to be a bit trickier. You're going to have to fix
> perms recursively, by moving up the directory tree, seeing if you can set
> perms, then if not then moving further up. This needs to be limited to the
> Tb metadata directory I think -- if we go further up we risk messing up home
> directory permissions.

I thought from comment #29 your detail on file creation made worrying about ancestors moot?
"- On Mac, luckily ancestors would have already had their execute permissions set properly per the code in nsLocalFileUnix.cpp [2], so all we need to worry about "

The "file" in question is a ".mozmsgs" (folder) directory, and looks like it must have been created via the paths already seen in searchCommon.js..take subfolders, even, for example:

Library/Caches/Metadata/Thunderbird/ImapMail/mail.{example}.net/INBOX.sbd/Friends.sbd/Old.mozmsgs

The folder Inbox->Friends->Old ..only the Old.mozmsgs directory has improper permissions.  I assume the ".sbd" directories have correct permissions because they were ancestors of the file.create (Old.mozmsgs, 0644) that happened a long time ago, and were fixed up by the code you mention in comment #29.

But..maybe I'm too focused on the specific issue of Tbird creating wrong perms and you are saying an access denied could happen at any layer in the tree because of external actions that may have created any possible mess of permissions?
Yeah, I think you're correct. Good point. I'd still want to find out why the idle logic isn't triggering, though.

> But..maybe I'm too focused on the specific issue of Tbird creating wrong
> perms and you are saying an access denied could happen at any layer in the
> tree because of external actions that may have created any possible mess of
> permissions?

Well, I'm sure users can shoot themselves in the foot in a number of ways. But let's not worry about that for the moment.
(In reply to Siddharth Agarwal [:sid0] from comment #39)
> 
> "idle" is as defined by Mozilla's idle service [1], which in turn relies on
> the OS. The threshold is defined as 30 seconds, which means that if you
> leave your computer idle for 30 seconds the logic should kick in. If you
> turn logging on (set mail.spotlight.logging.console to true, then run
> Thunderbird from a terminal) you should see idle/back messages in the
> terminal.
> 

Yes--I've been watching the console, and grepping for "idle" (which gets logged from the observer) and it has never happened yet.  So--maybe there is an issue on that Mac.  On my Mac, when I turned on console, I definitely saw some idle messages in the log (when I originally was looking at the problem).

Both these systems are Snow Leopard, latest fixpack.  The one I installed the test build on has been sitting idle (screensaver mode) for probably 30-45 minutes now, and still no idle message...maybe I'll look at activity monitor and see if for some reason another process is making the OS think it's not "idle"
Well, I had hoped I had found the culprit with idle--Skype was sitting at 100% CPU on that system for some reason; oops--but still not getting proper behavior:

Jan 24 13:09:49 imac06 [0x0-0x60060].org.mozilla.daily[0]: 2012-01-24 13:09:49	SearchInt	DEBUG	Idle detected, continuing sweep
Jan 24 13:09:54 imac06 [0x0-0x60060].org.mozilla.daily[0]: 2012-01-24 13:09:54	SearchInt	DEBUG	Non-idle, so suspending sweep

That was over 1.5 hours ago and it has never gotten idle again even though the machine hasn't been touched in that time period..and usage shows load at < 0.20 ; cpu roughly idle..not sure what to try next..the idle period at start was so short (above--shows 5 seconds) that it didn't have time to do anything to get to our code fix
I'm convinced something is wrong with idle in the tinderbox build from comment 34.  I've now tried two different Macs with various states of Metadata and accounts, and no matter how long they run they never get a single idle notification in the log.  Because of that, there is no way to test the "fixup" part of the patch.  I did try a system with 'cleaned out' Metadata (forcing T-bird to recreate) and it worked fine..all directories created properly and lots of messages got queued and created as .mozeml files.

As soon as a switch back to released builds and turn on logging, I see all the idle messages working again (idle/back, with sweeps starting and stopping).  So, before I tear out my hair anymore, someone needs to validate whether idle notifications have a bug in 12a1 level generally.
Comment on attachment 589759 [details] [diff] [review]
proposed fix for perms + reindexing

I'm cancelling this review request whilst we work out the issues around idle, and Sid is busy on other things at the moment.

Irving - could you take a look and see if you can reproduce Phil's issues with idle?
Attachment #589759 - Flags: review?(sagarwal)
My trunk thunderbird doesn't seem to be getting idle signals either. The idle service doesn't have any tracing, so I'm going to add some and try to see what's going on.
Filed https://bugzilla.mozilla.org/show_bug.cgi?id=744527 to resolve one issue I found with the idle service; there have been several other recent patches to the idle service that may also have been the cause (https://bugzilla.mozilla.org/show_bug.cgi?id=720493, maybe).

I've tested with Phil's patch, and also tried some of my own changes based on the ideas in comment 37 and 38. I think it would be a good idea to add that, if we can get the exception handled in the right place.

As far as I can tell, JS was not reporting the line number of the error correctly - the permission denied failure wasn't in the file.exists() test on the directory, it was actually on the file.create() for the metadata file around line 888: http://mxr.mozilla.org/comm-central/source/mail/components/search/content/searchCommon.js#888
See Also: → 744527
Oh, also, I think the idle service is in good enough shape that we should move the rest of this fix forward. Also, assigned to Phil to honour his patch submission ;-)
Assignee: nobody → estesp
Status: NEW → ASSIGNED
Let me know if you need me to re-test anything given the corrected idle service.  And, yes, in hindsight, I remember in further debug, the problem was at 888, not 884 (the actual file.create), but never got around to correcting that comment.
Hah. Line numbers for preprocessed files are always misleading, and will continue to be until (a) Gecko has sourcemap [1] support and (b) we support them in preprocessed files.

[1] https://wiki.mozilla.org/DevTools/Features/SourceMap
Comment on attachment 589759 [details] [diff] [review]
proposed fix for perms + reindexing

Irving has said the idle service issues are solved now, so I'll pick up the review of this soon.
Attachment #589759 - Flags: review?(mbanner)
Comment on attachment 589759 [details] [diff] [review]
proposed fix for perms + reindexing

Review of attachment 589759 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the big delay in getting back to this. I'd tried it a few times and hadn't realised that it needed to go into idle mode to actually fix the permissions, rather than fixing them on message indexing. I think that's fine as it saves checking the permission on each message when we're indexing it.

Hence, r=me.

Thanks for working on the patch, I'm sorry it has taken so long to get it in.
Attachment #589759 - Flags: review?(mbanner) → review+
Checked in: https://hg.mozilla.org/comm-central/rev/c24070161c87
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 16.0
https://hg.mozilla.org/comm-central/rev/c24070161c87#l3.12
Hssst. Raw octals now cause warnings (Octal Numeric Constants are deprecated). Please use parseInt("0755", 8) instead next time. See Bug 546300 for example.
And Bug 572890 - fix "octal literals and octal escape sequences are deprecated" in c-c
(In reply to Philip Chee from comment #55)
> https://hg.mozilla.org/comm-central/rev/c24070161c87#l3.12
> Hssst. Raw octals now cause warnings (Octal Numeric Constants are
> deprecated). Please use parseInt("0755", 8) instead next time. See Bug
> 546300 for example.

Good catch. I was going to suggest using FileUtils.jsm (http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/mozapps/shared/FileUtils.jsm) but it also uses octal literals; I'm not sure why we're not getting warnings from that.

There are several other places where octal has crept back into C-C JavaScript. 
I suggest we clone Bug 572890 and fix them again.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: