mdworker is unable to load thunderbird.mdimporter, because of Library not loaded: @executable_path/libmozglue.dylib

NEW
Assigned to

Status

defect
5 years ago
2 years ago

People

(Reporter: Nomis101, Assigned: Nomis101)

Tracking

(Blocks 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [regression:TB31])

Attachments

(2 attachments)

(Assignee)

Description

5 years ago
While testing TB 31, I had a look into Console.app. I've found there a lot of entries like:

16.05.14 12:29:45,214 mdworker[27874]: Error loading /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:  dlopen(/Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird, 262): Library not loaded: @executable_path/libmozglue.dylib
  Referenced from: /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird
  Reason: image not found
16.05.14 12:29:45,215 mdworker[27874]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7f8f98c13840 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, not loaded)
16.05.14 12:29:45,216 mdworker[27874]: (Error) Import: Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
16.05.14 12:29:45,216 mdworker[27874]: (Error) Import: BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/


I don't know if this means, that the mdimporter is not working currently.
(Assignee)

Updated

5 years ago
Blocks: TB31found
I'm not seeing this myself but I'm sure I've disabled search integration.

Nomis can you try to figure out if it's working or not ?
(Assignee)

Comment 2

5 years ago
Sorry, the search integration is not working. I've tried with the newest TB 31 build. If I do 
$ /usr/bin/mdimport -L
Than the plugin shows up properly in the list of Spotlight plugins. But
$ /usr/bin/mdimport -d2
fails with:

$ /usr/bin/mdimport -d2 /Users/Library/Caches/Metadata/Thunderbird/Mail/pop3.web-3.de/Sent.mozmsgs/536F9366.8030409\%40web.de.mozeml
2014-06-04 19:44:57.397 mdimport[9261:507] Error loading /Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:  dlopen(/Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird, 262): Library not loaded: @executable_path/libmozglue.dylib
  Referenced from: /Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird
  Reason: image not found
2014-06-04 19:44:57.398 mdimport[9261:507] Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7ff4e0603ba0 </Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, not loaded)
(Error) Import: Could not create instance for plugIn 'file:///Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
(Error) Import: BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter/
2014-06-04 19:44:57.399 mdimport[9261:507] Imported '/Users/Library/Caches/Metadata/Thunderbird/Mail/pop3.web-3.de/Sent.mozmsgs/536F9366.8030409%40web.de.mozeml' of type 'com.mozilla.thunderbird.mozeml' with plugIn /Applications/Earlybird.app/Contents/Library/Spotlight/thunderbird.mdimporter.
2014-06-04 19:44:57.400 mdimport[9261:507] Attributes: {
    ":MD:kMDItemPath" = "/Users/Library/Caches/Metadata/Thunderbird/Mail/pop3.web-3.de/Sent.mozmsgs/536F9366.8030409%40web.de.mozeml";
    "_kMDItemContentChangeDate" = "2014-05-11 15:11:35 +0000";
    "_kMDItemCreationDate" = "2014-05-11 15:11:35 +0000";
    "_kMDItemCreatorCode" = 0;
    "_kMDItemFileName" = "536F9366.8030409%40web.de.mozeml";
    "_kMDItemFinderFlags" = 0;
    "_kMDItemFinderLabel" = 0;
    "_kMDItemIsExtensionHidden" = 0;
    "_kMDItemOwnerGroupID" = 20;
    "_kMDItemOwnerUserID" = 501;
    "_kMDItemStaticInterestScore" = "1.111111";
    "_kMDItemTypeCode" = 0;
    "com_apple_metadata_modtime" = 421513895;
    kMDItemContentCreationDate = "2014-05-11 15:11:35 +0000";
    kMDItemContentModificationDate = "2014-05-11 15:11:35 +0000";
    kMDItemContentType = "com.mozilla.thunderbird.mozeml";
    kMDItemContentTypeTree =     (
        "com.mozilla.thunderbird.mozeml",
        "public.data",
        "public.item",
        "public.content",
        "public.email-message",
        "public.message"
    );
    kMDItemDateAdded = "2014-05-11 15:11:35 +0000";
    kMDItemDisplayName =     {
        "" = "536F9366.8030409%40web.de.mozeml";
    };
    kMDItemKind =     {
        "" = "Thunderbird Mail Message";
    };
    kMDItemLogicalSize = 597;
    kMDItemPhysicalSize = 4096;
}



And it also finds no TB message from me if I search for them via Spotlight.

https://developer.apple.com/library/mac/documentation/Carbon/Conceptual/MDImporters/Concepts/Troubleshooting.html
Mark do you also see this ?
(Assignee)

Comment 4

5 years ago
10.10 is a bit more detailed (from Cosole app):

18.08.14 22:23:45,172 mdworker[5313]: Error loading /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport:  dlopen(/Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport, 262): Library not loaded: @executable_path/libmozglue.dylib
  Referenced from: /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport
  Reason: unsafe use of @executable_path in /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport with restricted binary

Comment 5

5 years ago
I have the same issue. Mac OS X 10.9.4

Comment 6

5 years ago
Same here. Noticed it with TB 31 on OS X 10.9.4. Used to be able to see TB emails in Spotlight search. Get the same messages in console when I look to see why I no longer get TB emails in Spotlight searches. Note that at times (very rarely), TB email files (not messages per se) somehow show up for certain Spotlight searches (I assume because some text matching was going on). I can grab and share more info if it's requested of me.

Comment 7

5 years ago
FWIW: I went ahead and renamed the Thunderbird.mdimporter directory in the Thunderbird.app for 31.0, and copied in the same from a backup of 24.6.0 (as of Jul7 19, 2014).

After doing so, no more errors, and rebuilding the Spotlight index now shows real messages found within the TB 31.0 folders. However, TB 31 is not handling the request to open the specific message correctly - it just opens and displays the default account's inbox. So, I still have to search again within TB. (No errors I can see...)

With the Thunderbird.mdimporter from 24.6.0, messages are again indexed correctly and there are no more errors logged regarding Spotlight and related Thunderbird 31.0 objects and functionality. However, Thunderbird 31.0 is not handling the request from the system/Spotlight to open the specific message selected within the Spotlight search results and no specific errors are written.

Hope this helps.
(Assignee)

Comment 8

5 years ago
Thats good to know. So, maybe it is related to Bug 944952? Thats the only relevant change in mdimporter I could find that happened between TB 24 and 31.

Yesterday I also played a bit with the mdimporter plugin, and I was able to remove this Console log and $ /usr/bin/mdimport -d2 gives now a proper output. But I only can test this on 10.10., I don't have 10.9 anymore. I will attach a patch with the changes I've made later today.
(Assignee)

Comment 9

5 years ago
This is, what I did yesterday to help the console message to go away. I have no idea, why this should help, but it did. Maybe it only triggered something. But, it only works in combination with the fix from Bug 1045026. And it removes this odd com.apple.yourcfbundle which is only meant as an example in the ADC documentation.

Comment 10

4 years ago
After the TB 31.3.0 installed on my Mac I started getting these errors again showing up in the console log:

14·12·17 10:24:55.588 PM mdworker[4112]: Error loading /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:  dlopen(/Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird, 262): Library not loaded: @executable_path/libmozglue.dylib
  Referenced from: /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird
  Reason: unsafe use of @executable_path in /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird with restricted binary
14·12·17 10:24:55.589 PM mdworker[4112]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7f8ac04143a0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, not loaded)
14·12·17 10:24:55.589 PM mdworker[4112]: (Import.Error:711) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
14·12·17 10:24:55.589 PM mdworker[4112]: (Import.Error:867) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

which would repeat every half hour or so, I imagine each time the Spotlight indexer was trying to load TB's mdimporter. It seems like @executable_path is unsafe to use as it is for some reason or perhaps was just aimed at the wrong place to find TB's mdimporter. I used the following to change the reference to the libmozglue.dylib to use @loader_path rather than @executable_path and ran "mdimport" and the errors from the console log are now gone:

  install_name_tool -change @executable_path/libmozglue.dylib @loader_path/../../MacOS/libmozglue.dylib

(which changes:
  @executable_ path/libmozglue.dylib
to
  @loader_path/../../MacOS/libmozglue.dylib )

 I imagine this change can be made when the application is linked and the dylibs are specified or it could be made after linking as I did. (I'm not skilled enough to know how to put it all together, though, but I hope this helps someone to make the change if needed.
(Assignee)

Comment 11

4 years ago
This is interesting to know and reminds me to Bug 578751. Does it also work for you if you use @rpath?

Comment 12

4 years ago
At the moment it's not working again. As far as I know, it had worked but something is different (maybe the relative directory reference since I'm not totally comfortable with where the @loader_path is starting from). I'm looking further though I don't have much time today. Instead of trying to work on the @loader_path method I'll make an attempt to try a fix using @rpath. If it works (or worked) with @loader_path I'd think it should work with @rpath but didn't have the time or focus to continue earlier. I'll get back as soon as I have more idea of what's going on.

Comment 13

4 years ago
Sorry, was researching the various ways that libraries are found and then went to bed with a cold for 2 days. The problem I had is not clear to me yet but appeared as system console errors after an update from an earlier version of Thunderbird (24.?.?) to the 31.3.0. In my incremental learning process I somewhat lost track of the exact changes I had made and finally "reinstalled" the newest release of Thunderbird (31.3.0) and then everything was fine once more. Fortunately I still have the original 24.?.? version I had during the update which I want to put back in place and then do an upgrade to 31.3.0 and see if the problem recurs. If so it may be a a delay OS X's use of the newest Spotlight importer or a cache file that needed to be updated which could be just a temporary glitch until the system irons out everything or it may indicate that something should be done in an upgrade to get everything in sync right away. At this point my thought is that it isn't a permanent problem when it occurs but that's a guess.

As to the @executable_path versus the @loader_path (and possibly @rpath), since I'm not yet sure of what state everything was in when the problem occurred it's hard to say if there is any problem. Now that I know more it seems that, unless there has been a restriction of some sort added as to using @executable_path in some situations (due to jeopardizing one's system), it seems that either @executable_path or @loader_path or @rpath (which would reference one of those anyway) should work since the directory structure of the Thunderbird.app where the Spotlight importer lives is stable and doesn't shift around in a way that @executable_path or @loader_path couldn't deal with.

I'll add more as I find out anything of concern. (Interestingly, while playing around with this issue I've encountered the "thunderbird triggered DYLD shared region unnest for…" issue which both Thunderbird and Firefox seem to be running against which might be a more useful thing to pursue but I think that can be done in parallel with this. Also, what I've learned from this may shine some light on Bug 578751 too and I'll keep that in mind as I go.

Comment 14

4 years ago
I see the same errors on TB 34 on my Yosemite MacPro:
Jan  3 15:06:15 jarmac-2.local mdworker[38527]: Error loading /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport:  dlopen(/Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport, 262): Library not loaded: @executable_path/libmozglue.dylib
	  Referenced from: /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport
	  Reason: unsafe use of @executable_path in /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport with restricted binary
Jan  3 15:06:15 jarmac-2.local mdworker[38527]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7f838850e2d0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, not loaded)
Jan  3 15:06:15 jarmac-2.local mdworker[38527]: (Import.Error:711) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
Jan  3 15:06:15 jarmac-2.local mdworker[38527]: (Import.Error:867) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Comment 15

4 years ago
@James: Did you upgrade the TB to 34 in response to TB letting you know it was there and telling it to upgrade or did you download a new copy and install it yourself? At some point once you've rebooted and after starting TB at least once to kick in the Spotlight Importer it would be good to know if those are still occurring for you. If not that might indicate that there's a conflict between the pre and post upgraded TB importers or libraries that might resolve itself after rebooting. My current TB 31 seems to be working OK now. When I went back and forth installing the old version I had been using and upgrading to the newer one I wasn't able to replicate the problem.

Comment 16

4 years ago
I don't remember how I upgraded. I think it updated. But I stopped and started TB again this morning, and so far none of these messages have appeared.

Comment 17

4 years ago
James> Thanks for checking that out. That seems to fit in with my experience. If I remember correctly about Spotlight Importers that reside in an application bundle (as this one is) they are loaded when the application is started. Ignoring other subtleties I'd assume that if TB updates itself and asks to be restarted for updates to take effect that it restarts in a way that doesn't trigger reloading the importer. Then the TB application bundle (and the dynamic libraries and references it contains) might be out of sync with the currently loaded older importer causing this issue to emerge until TB is restarted in a way that  would reload the importer and bring everything back into sync. That be a reboot or logoff/login or whatever but it seems to be a transitory issue that solves itself quickly (unless, for example, should a reboot be required, one doesn't reboot for a long time). If this issue were to be looked at it should probably be examined in the light of upgrading TB since the current code does seem to work OK.

Comment 18

4 years ago
I am getting werrors again:
Jan 14 09:13:50 jarmac-2.local mdworker[90209]: Error loading /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport:  dlopen(/Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport, 262): Library not loaded: @executable_path/libmozglue.dylib
	  Referenced from: /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport
	  Reason: unsafe use of @executable_path in /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport with restricted binary

Comment 19

4 years ago
I am also getting the "@executable_path" and "BAD IMPORTER ~~~~~~…" errors again myself. I've now done some exhausting (if not exhaustive) testing to try to see why it happens and then may disappear or come back at seemingly random times. At this point I'm almost certain (I'll never be totally!) that is has to do with the executable files with their quarantine attribute set when they are downloaded (which is correct) but not being removed after the first execution of Thunderbird. This seems to agree with problems other developers have had to deal with when securing applications for OS X. 

Here is what I'm finding on my system (and which is probably the same for others (though the behavior might vary depending on what security preferences a user has set):

- If I remove TB from my /Applications folder, download a fresh copy and drag it to /Applications all of the files in the bundle are marked with the quarantine attribute. 

- When I open it I'm asked to OK it (since it has been downloaded) so I click OK and it runs. When I check the bundle's files with xattr I see the quarantine attributes are still set although I am no longer asked to OK it when I start it (same behavior as other developers have described). (Currently I have my security set to allow applications from the AppStore or identified developers.) 

- Even though TB starts up OK the TB MDImporter fails with the errors mentioned earlier. If I use xattr to clear the extended attributes from the downloaded files and then use mdimport to reindex TB mail files (or wait until it is done automatically) then the TB MDImporter functions correctly.

- If a manual download and complete copy of a new TB release is done then the situation is the same as loading a fresh copy since all the files' quarantine attributes are turned on again. If an automatic update occurs then any files that are replaced are marked as quarantined; that means that most or all files are quarantined but (due to the way TB updates itself, I think) most or all of the directories remain non-quarantined. The effect is the same though since the "thunderbird.mdimporter" executable is quarantined.

- The problem is that since the TB MDImporter is started by MDS automatically the only place that one would see a problem is from the error messages that are written to the console log; most user's would never see them and the only indication of a problem is when spotlight can't find TB mail files that it should. (And, IMHO, spotlight has enough problems due to various issues that cause it to miss showing relevant files that even that is hard to know is happening.)

- If the quarantine attribute is cleared from the TB MDImporter sub-bundle then it will again function correctly.

- Finally I found in Apple's code signing guide the following requirement: "You should sign every executable in your product, including applications, tools, hidden helper tools, utilities and so forth. Signing an application bundle covers its resources, but not its subcomponents such as tools and sub-bundles. Each of these must be signed independently." TB has the bundle's main application code signed but there is no code signature (which should be identical to the existing one) applied to the MDImporter sub-bundle. I've not tested it (unfortunately, I don't have time to learn the skills to do so or even time to do it myself) but it seems likely that with the proper code signing that the current problems with the TB MDImporter might be resolved.

- In the meantime for those who want to attempt to fix their TB spotlight indexing it may be necessary to use xattr to clear the quarantine attributes from the entire TB bundle and contained sub-bundles, etc. This can be done within the terminal as:

  xattr -cr /Applications/Thunderbird.app

That command will recursively [r] clear [c] the entire Thunderbird.app directory tree. The next time Thunderbird is started MDS should pick up the TB MDImporter and run it without errors. I'm still getting a spurious error when I use mdimport to reindex the TB files but don't seem to get it if I invoke the TB MDImporter by starting up TB and letting MDS do the auto start up of TB's importer. Hopefully some improvement with the code signing (and perhaps other security issues that may be involved) will get things totally in working order. (I wish I was 100% certain of that but that's all I can deal with just now; I'll post again if things go bad.) 

Zhora

Comment 20

4 years ago
Almost forgot! If you do the xattr clear fix I mentioned you should also make sure that the dates on the files of the TB MDImporter are changed or a new execution of it won't happen. Probably only the MDImporter sub-bundle needs to be changed but I updated the dates on the entire Thunderbird.app tree like this:

find /Applications/Thunderbird.app -exec touch {} \;

That seems to have fixed the remaining errors but as I said, I'll post again if there are problems.

Zhora

Comment 21

4 years ago
After doing the above, I still get these messages:
Feb 10 09:48:29 jarmac.local thunderbird[95220]: Performance: Please update this scripting addition to supply a value for ThreadSafe for each event handler: "/Library/ScriptingAdditions/SIMBL.osax"

Feb 10 09:48:31 jarmac.local thunderbird[95220]: GetDYLDEntryPointWithImage(/System/Library/Frameworks/AppKit.framework/Versions/Current/AppKit,_NSCreateAppKitServicesMenu) failed.

Comment 22

4 years ago
And the mdimporter messages are back:
Feb 12 10:46:37 jarmac.local mdworker[26976]: (Import.Error:867) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Also, I can no longer send mail!

Comment 23

4 years ago
James Rome: You didn't say exactly what you did but if it was any of the above then the only thing that would be changed from doing those things would be your Thunderbird.app bundle. In that case you can revert from any changes that may have caused problems by downloading your current version of Thunderbird and copying it to /Applications (replacing the version you have). That doesn't touch any of your mailboxes or mail or extensions which are stored in your Thunderbird profile, not in the Thunderbird.app application.

SIMBL is a third party "extension" that alters some of the internal opertations (code) in OS X and is often installed by other extensions or applications to allow them to do certain things that can't be done in the allowed (by Apple) ways. Depending on how it is being used it may or may not cause problems in the operation of other software or OS X in general. (Note that I'm not saying that one should or shouldn't use it but should be aware that it may cause other OS X internal or third-party code to misbehave, especially if it isn't kept updated as need be. It's sometimes used for ad blockers or anything that needs to perform some function that for good or bad Apple has made impossible or difficult to do. I used to have it installed and eventually removed it and the applications that depended on it due to various problems that it caused for me.)

From what you've said I'd guess that some extension for Thunderbird that has been installed has installed SIMBL for it's own use. From the message it appears that something referred to as "jarmac.local" is using SIMBL in a way that is failing and causing an error in a call to the AppKit library. The message says to update jarmac to hopefully correct the problem. Whether that suggestion is coming from "jarmac" itself or some other place I can't say and can't say if or if not "jarmac" can be updated or if it will help. As I said earlier, the way to fix it is to either install a fresh copy of Thunderbird.app or (if "jarmac" or SIMBL have altered the Thunderbird.app in some way (which they shouldn't really) reload Thunderbird.app from a recent backup before you made any of the changes that may have caused the problem. (If somehow there have been other changes to files outside of the Thunderbird.app then they should be identified and corrected in the same sort of way or by some other means.

That is the situation with the error messages you show in comment 21. The "BAD IMPORTER" messag in comment 22 is again the original problem which, after finally downloading several of the minor updates to Thunderbird 31 (31.4, 31.3, 31.2 and 31.0), shows that the MDImporer for Thunderbird is broken (there appears to be no generated code in the importer app). I did downloaded Thunderbird 24.? which I was able to find quickly and verified that it did contain the MDImporter code. So at this point it appears that none of the Thunderbird MDImporters that have shipped with Thunderbird 31 have been functional copies. (I'll expand on this soon but don't have the time just now…in any case it doesn't matter until a working MDImporter is able to be produced which means only that currently, Thunderbird 31 is not able to index it's mail files for Spotlight's use. If that is absolutely needed by someone then for now they probably need to downgrade to a working version though I haven't myself determined when the broken MDImporter began to be distributed.)

Comment 24

4 years ago
Noomis, I now see what's wrong though I don't know enough about how the build process works to be able to say why. All of the TB releases starting at some point after 24.8.1 (including all the current 31.x releases have mdimporter executables that appear to contain no compiled object code. I've been running the otool command on the mdimporter executable for various TB releases to decompile the code like so…

   otool -tV /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird 

(shown for TB in /Applications though it can be looked at wherever one copies TB.)

Here is the beginning of the results of the otool disassembly for TB 24.8.1:

    
    /Users/zhora/Desktop/test TB/Thunderbird 24.8.1.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:
------------------------------------------
    (__TEXT,__text) section
    _AllocMetadataImporterPluginType:
    000000000000074c	pushq	%rbp
    000000000000074d	movq	%rsp, %rbp
    0000000000000750	pushq	%r14
    0000000000000752	pushq	%rbx
    0000000000000753	movq	%rdi, %rbx
    0000000000000756	movl	$0x18, %edi
    000000000000075b	callq	0xc7a                   ## symbol stub for: _malloc
    0000000000000760	movq	%rax, %r14
    0000000000000763	movq	$0x0, 0x10(%r14)
    000000000000076b	movq	$0x0, 0x8(%r14)
    0000000000000773	movq	$0x0, (%r14)
    000000000000077a	leaq	0x96f(%rip), %rax
    0000000000000781	movq	%rax, (%r14)
    0000000000000784	movq	%rbx, %rdi
    0000000000000787	callq	0xc56                   ## symbol stub for: _CFRetain
    000000000000078c	movq	%rax, 0x8(%r14)
    0000000000000790	movq	%rbx, %rdi
    0000000000000793	callq	0xc2c                   ## symbol stub for: _CFPlugInAddInstanceForFactory
    0000000000000798	movl	$0x1, 0x10(%r14)
    00000000000007a0	movq	%r14, %rax
    00000000000007a3	popq	%rbx
    00000000000007a4	popq	%r14
    00000000000007a6	popq	%rbp
    00000000000007a7	retq
    _MetadataImporterPluginAddRef:
    00000000000007a8	pushq	%rbp
    00000000000007a9	movq	%rsp, %rbp
    00000000000007ac	movl	0x10(%rdi), %eax
    …and so on, you get the idea.
------------------------------------------

Here is what I'm seeing in the later, including the current, mdimporters for TB…
------------------------------------------
    /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:
    (__TEXT,__text) section
------------------------------------------

That's all…pretty much nothing! So it seems that something is going wrong in the TB build that causes a failure in the compilation/linking/whatever that constructs the mdimporter execuatble (at least). I'm pretty sure that this "null" executable is the cause of the error we're seeing in the console log regarding @executable_path error. Since our TB build is stable in terms of where each of the libraries is located in relation to anything else I believe that it doesn't matter if we use @executable_path, @loader_path or @rpath since we have a known hierarchy of directories to traverse to get to (in this case) the libmozglue.dylib. That can be checked once the source of the build error is determined. It, of course, would also be good to see what needs to be done to test that the build of the importer at least produces code and better yet correct code.)

I attempted to set up my own build of FF/TB to be able to test some of this but I'm afraid that it's too much for me to comprehend at the moment and would be lots easier for someone with the right experience to handle (such as you?). Anyway, that's why I'm aiming this at you so that you can make whatever changes, flagging, comments, etc. are required to bring this bug into it's correct form to be handled. Sorry I don't have the time to do more just now but I unfortunately have basic living issues, i.e. food, money, shelter that need to be dealt with.

I'll note one other thing…I looked at the latest beta release, TB 36.0b1, and in that release the final MacOS/thunderbird branch of the mdimporter appears to be missing completely. I'm not sure if that is intentional or not but being a beta release (I think) I suspect it's a problem.

In anycase I plan to graft the mdimporter from 24.8.1 onto 31.4 and see if that works OK since it doesn't appear that much has changed with the importer and I think the dynamic library's connections to the importer should be…well…dynamic and should work. I'll try to remember to come back and say what I find.

And after all the false leads that seemed to be this or that or whatever along the way, I have to finish by quoting the very first paragraph of Apple's "Troubleshooting Spotlight Importers" section of the "Spotlight Importer Developer's Guide:

"Rare is the project that is flawless from the start. Troubleshooting a Spotlight importer can be difficult given that it is run by the system automatically as required and is run outside the development environment."

Damn right it's difficult! I think it's worth noting that without removing (or obscuring) every other version of an mdimporter that has previously loaded and restarting (and perhaps even more than that) you'll never be sure if the mdimporter you are running is the version you think it is or not. In trying to test an mdimporter for a different version of TB than I had been "running" I eventually found that the system was actually running the mdimporter execuatable that was in the TB that was in the Trash! (And, if I recall correctly, which I won't vouch for, that was occurring AFTER I had restarted the machine with TB in the Trash. Difficult, difficult, difficult!!!
Flags: needinfo?(Nomis101)
(Assignee)

Comment 25

4 years ago
Interestingly, in the newer builds (current beta and trunk) I don't see any folder "MacOS" in /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents
This is only present in older builds (24). Thats an issue! What versions have you tested?
Flags: needinfo?(Nomis101)

Comment 26

4 years ago
Here's the mdimporter tree portion of various TB releases that I've looked at (I've added the version numbers into the "Thunderbird.app" name after downloading to keep things organized:

------------------------------------------------------------------------

Here's the TB 24.8.1 release which is the last non-beta release that seems to contain a valid TB mdimporter:

Thunderbird\ 24.8.1.app/
├── Contents
│   ├── CodeResources
│   ├── Info.plist
│   ├── Library
│   │   └── Spotlight
│   │       └── thunderbird.mdimporter
│   │           └── Contents
│   │               ├── Info.plist
│   │               ├── MacOS
│   │               │   └── thunderbird
│   │               └── Resources
│   │                   ├── English.lproj
│   │                   │   ├── InfoPlist.strings
│   │                   │   └── schema.strings
│   │                   └── schema.xml
│   ├── MacOS
│   │   ├── XUL
│   │   ├── application.ini
│   │   ├── blocklist.xml
│   │   ├── crashreporter.app
(rest of TB.app tree omitted)

otool -tV for the 24.8.1 mdimporter (mdimporter is still named "thunderbird") shows what is probably normal code:

Thunderbird 24.8.1.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:
(__TEXT,__text) section
_AllocMetadataImporterPluginType:
000000000000074c	pushq	%rbp
000000000000074d	movq	%rsp, %rbp
0000000000000750	pushq	%r14
0000000000000752	pushq	%rbx
0000000000000753	movq	%rdi, %rbx
0000000000000756	movl	$0x18, %edi
000000000000075b	callq	0xc7a                   ## symbol stub for: _malloc
0000000000000760	movq	%rax, %r14
0000000000000763	movq	$0x0, 0x10(%r14)
000000000000076b	movq	$0x0, 0x8(%r14)
0000000000000773	movq	$0x0, (%r14)
000000000000077a	leaq	0x96f(%rip), %rax
0000000000000781	movq	%rax, (%r14)
0000000000000784	movq	%rbx, %rdi
0000000000000787	callq	0xc56                   ## symbol stub for: _CFRetain
000000000000078c	movq	%rax, 0x8(%r14)
0000000000000790	movq	%rbx, %rdi
0000000000000793	callq	0xc2c                   ## symbol stub for: _CFPlugInAddInstanceForFactory
0000000000000798	movl	$0x1, 0x10(%r14)
00000000000007a0	movq	%r14, %rax
00000000000007a3	popq	%rbx
00000000000007a4	popq	%r14
00000000000007a6	popq	%rbp
00000000000007a7	retq
_MetadataImporterPluginAddRef:
00000000000007a8	pushq	%rbp
00000000000007a9	movq	%rsp, %rbp
00000000000007ac	movl	0x10(%rdi), %eax
00000000000007af	incl	%eax
00000000000007b1	movl	%eax, 0x10(%rdi)
00000000000007b4	popq	%rbp
00000000000007b5	retq
_MetadataImporterQueryInterface:
00000000000007b6	pushq	%rbp
00000000000007b7	movq	%rsp, %rbp
00000000000007ba	pushq	%r15
00000000000007bc	pushq	%r14
00000000000007be	pushq	%r12
00000000000007c0	pushq	%rbx
00000000000007c1	subq	$0x60, %rsp
00000000000007c5	movq	%rcx, %rbx
00000000000007c8	movq	%rdi, %r14
00000000000007cb	movq	0x83e(%rip), %rax       ## literal pool symbol address: _kCFAllocatorDefault
00000000000007d2	movq	(%rax), %r15
00000000000007d5	movq	%r15, %rdi
00000000000007d8	callq	0xc68                   ## symbol stub for: _CFUUIDCreateFromUUIDBytes
00000000000007dd	movq	%rax, %r12
00000000000007e0	movl	$0xfc, 0x50(%rsp)
00000000000007e8	movl	$0x26, 0x48(%rsp)
00000000000007f0	movl	$0x67, 0x40(%rsp)
00000000000007f8	movl	$0x93, 0x38(%rsp)
0000000000000800	movl	$0x3, 0x30(%rsp)
0000000000000808	movl	$0x0, 0x28(%rsp)
0000000000000810	movl	$0xae, 0x20(%rsp)
0000000000000818	movl	$0x84, 0x18(%rsp)
0000000000000820	movl	$0xd8, 0x10(%rsp)
0000000000000828	movl	$0x11, 0x8(%rsp)
0000000000000830	movl	$0x9c, (%rsp)
0000000000000837	movl	$0x6e, %esi
000000000000083c	movl	$0xbc, %edx
0000000000000841	movl	$0x27, %ecx
0000000000000846	movl	$0xc4, %r8d
000000000000084c	movl	$0x89, %r9d
0000000000000852	movq	%r15, %rdi
0000000000000855	callq	0xc6e                   ## symbol stub for: _CFUUIDGetConstantUUIDWithBytes
000000000000085a	movq	%r12, %rdi
000000000000085d	movq	%rax, %rsi
0000000000000860	callq	0xc26                   ## symbol stub for: _CFEqual
0000000000000865	testb	%al, %al
0000000000000867	je	0x884
0000000000000869	movq	(%r14), %rax
000000000000086c	movq	%r14, %rdi
000000000000086f	callq	*0x10(%rax)
0000000000000872	movq	%r14, (%rbx)
0000000000000875	movq	%r12, %rdi
0000000000000878	callq	0xc50                   ## symbol stub for: _CFRelease
000000000000087d	xorl	%eax, %eax
000000000000087f	jmp	0x91d
0000000000000884	movl	$0x46, 0x50(%rsp)
000000000000088c	movl	$0x0, 0x48(%rsp)
0000000000000894	movl	$0x0, 0x40(%rsp)
000000000000089c	movl	$0x0, 0x38(%rsp)
00000000000008a4	movl	$0x0, 0x30(%rsp)
00000000000008ac	movl	$0x0, 0x28(%rsp)
00000000000008b4	movl	$0x0, 0x20(%rsp)
00000000000008bc	movl	$0xc0, 0x18(%rsp)
00000000000008c4	movl	$0x0, 0x10(%rsp)
00000000000008cc	movl	$0x0, 0x8(%rsp)
00000000000008d4	movl	$0x0, (%rsp)
00000000000008db	movq	0x736(%rip), %rax       ## literal pool symbol address: _kCFAllocatorSystemDefault
00000000000008e2	movq	(%rax), %rdi
00000000000008e5	xorl	%esi, %esi
00000000000008e7	xorl	%edx, %edx
00000000000008e9	xorl	%ecx, %ecx
00000000000008eb	xorl	%r8d, %r8d
00000000000008ee	xorl	%r9d, %r9d
00000000000008f1	callq	0xc6e                   ## symbol stub for: _CFUUIDGetConstantUUIDWithBytes
00000000000008f6	movq	%r12, %rdi
00000000000008f9	movq	%rax, %rsi
00000000000008fc	callq	0xc26                   ## symbol stub for: _CFEqual
0000000000000901	testb	%al, %al
0000000000000903	jne	0x869
0000000000000909	movq	$0x0, (%rbx)
0000000000000910	movq	%r12, %rdi
0000000000000913	callq	0xc50                   ## symbol stub for: _CFRelease
0000000000000918	movl	$0x80000004, %eax       ## imm = 0x80000004
000000000000091d	addq	$0x60, %rsp
0000000000000921	popq	%rbx
0000000000000922	popq	%r12
0000000000000924	popq	%r14
0000000000000926	popq	%r15
0000000000000928	popq	%rbp
0000000000000929	retq
_DeallocMetadataImporterPluginType:
000000000000092a	pushq	%rbp
000000000000092b	movq	%rsp, %rbp
000000000000092e	pushq	%rbx
000000000000092f	subq	$0x8, %rsp
0000000000000933	movq	0x8(%rdi), %rbx
0000000000000937	callq	0xc74                   ## symbol stub for: _free
000000000000093c	testq	%rbx, %rbx
000000000000093f	jne	0x948
0000000000000941	addq	$0x8, %rsp
0000000000000945	popq	%rbx
0000000000000946	popq	%rbp
0000000000000947	retq
0000000000000948	movq	%rbx, %rdi
000000000000094b	callq	0xc32                   ## symbol stub for: _CFPlugInRemoveInstanceForFactory
0000000000000950	movq	%rbx, %rdi
0000000000000953	addq	$0x8, %rsp
0000000000000957	popq	%rbx
0000000000000958	popq	%rbp
0000000000000959	jmp	0xc50                   ## symbol stub for: _CFRelease
_MetadataImporterPluginRelease:
000000000000095e	pushq	%rbp
000000000000095f	movq	%rsp, %rbp
0000000000000962	movl	0x10(%rdi), %eax
0000000000000965	decl	%eax
0000000000000967	movl	%eax, 0x10(%rdi)
000000000000096a	testl	%eax, %eax
000000000000096c	jne	0x975
000000000000096e	callq	_DeallocMetadataImporterPluginType
0000000000000973	xorl	%eax, %eax
0000000000000975	popq	%rbp
0000000000000976	retq
_MetadataImporterPluginFactory:
0000000000000977	pushq	%rbp
0000000000000978	movq	%rsp, %rbp
000000000000097b	pushq	%r14
000000000000097d	pushq	%rbx
000000000000097e	subq	$0x60, %rsp
0000000000000982	movq	%rsi, %rbx
0000000000000985	movl	$0xfc, 0x50(%rsp)
000000000000098d	movl	$0x26, 0x48(%rsp)
0000000000000995	movl	$0x67, 0x40(%rsp)
000000000000099d	movl	$0x93, 0x38(%rsp)
00000000000009a5	movl	$0x3, 0x30(%rsp)
00000000000009ad	movl	$0x0, 0x28(%rsp)
00000000000009b5	movl	$0xf9, 0x20(%rsp)
00000000000009bd	movl	$0xb3, 0x18(%rsp)
00000000000009c5	movl	$0xd8, 0x10(%rsp)
00000000000009cd	movl	$0x11, 0x8(%rsp)
00000000000009d5	movl	$0x5b, (%rsp)
00000000000009dc	movq	0x62d(%rip), %rax       ## literal pool symbol address: _kCFAllocatorDefault
00000000000009e3	movq	(%rax), %rdi
00000000000009e6	movl	$0x8b, %esi
00000000000009eb	movl	$0x8, %edx
00000000000009f0	movl	$0xc4, %ecx
00000000000009f5	movl	$0xbf, %r8d
00000000000009fb	movl	$0x41, %r9d
0000000000000a01	callq	0xc6e                   ## symbol stub for: _CFUUIDGetConstantUUIDWithBytes
0000000000000a06	movq	%rbx, %rdi
0000000000000a09	movq	%rax, %rsi
0000000000000a0c	callq	0xc26                   ## symbol stub for: _CFEqual
0000000000000a11	testb	%al, %al
0000000000000a13	je	0xa46
0000000000000a15	movq	0x5f4(%rip), %rax       ## literal pool symbol address: _kCFAllocatorDefault
0000000000000a1c	movq	(%rax), %rdi
0000000000000a1f	leaq	0x6f2(%rip), %rsi       ## Objc cfstring ref: @"37401ADE-1058-42DB-BBE5-F2AAB9D7C13E"
0000000000000a26	callq	0xc62                   ## symbol stub for: _CFUUIDCreateFromString
0000000000000a2b	movq	%rax, %rbx
0000000000000a2e	movq	%rbx, %rdi
0000000000000a31	callq	_AllocMetadataImporterPluginType
0000000000000a36	movq	%rax, %r14
0000000000000a39	movq	%rbx, %rdi
0000000000000a3c	callq	0xc50                   ## symbol stub for: _CFRelease
0000000000000a41	movq	%r14, %rax
0000000000000a44	jmp	0xa48
0000000000000a46	xorl	%eax, %eax
0000000000000a48	addq	$0x60, %rsp
0000000000000a4c	popq	%rbx
0000000000000a4d	popq	%r14
0000000000000a4f	popq	%rbp
0000000000000a50	retq
_GetMetadataForFile:
0000000000000a51	pushq	%rbp
0000000000000a52	movq	%rsp, %rbp
0000000000000a55	pushq	%r15
0000000000000a57	pushq	%r14
0000000000000a59	pushq	%r13
0000000000000a5b	pushq	%r12
0000000000000a5d	pushq	%rbx
0000000000000a5e	subq	$0x18, %rsp
0000000000000a62	movq	%rsi, %rbx
0000000000000a65	movq	0x5a4(%rip), %rax       ## literal pool symbol address: _kCFAllocatorDefault
0000000000000a6c	movq	(%rax), %r14
0000000000000a6f	movq	%r14, %rdi
0000000000000a72	movq	%rcx, %rsi
0000000000000a75	xorl	%edx, %edx
0000000000000a77	xorl	%ecx, %ecx
0000000000000a79	callq	0xc5c                   ## symbol stub for: _CFURLCreateWithFileSystemPath
0000000000000a7e	movq	%rax, %r15
0000000000000a81	movq	%r14, %rdi
0000000000000a84	movq	%r15, %rsi
0000000000000a87	callq	0xc44                   ## symbol stub for: _CFReadStreamCreateWithFile
0000000000000a8c	movq	%rax, %r12
0000000000000a8f	movq	%r12, %rdi
0000000000000a92	callq	0xc4a                   ## symbol stub for: _CFReadStreamOpen
0000000000000a97	movq	$0x0, -0x38(%rbp)
0000000000000a9f	leaq	-0x30(%rbp), %r8
0000000000000aa3	leaq	-0x38(%rbp), %r9
0000000000000aa7	movq	%r14, %rdi
0000000000000aaa	movq	%r12, %rsi
0000000000000aad	xorl	%edx, %edx
0000000000000aaf	xorl	%ecx, %ecx
0000000000000ab1	callq	0xc38                   ## symbol stub for: _CFPropertyListCreateFromStream
0000000000000ab6	cmpq	$0x0, -0x38(%rbp)
0000000000000abb	je	0xae2
0000000000000abd	leaq	0x2d1(%rip), %rdi       ## literal pool for: "failed creating property list from stream"
0000000000000ac4	callq	0xc86                   ## symbol stub for: _puts
0000000000000ac9	movq	-0x38(%rbp), %rsi
0000000000000acd	leaq	0x2eb(%rip), %rdi       ## literal pool for: "error = %s\n"
0000000000000ad4	xorb	%bl, %bl
0000000000000ad6	xorb	%al, %al
0000000000000ad8	callq	0xc80                   ## symbol stub for: _printf
0000000000000add	jmp	0xbe4
0000000000000ae2	movq	%rax, %r14
0000000000000ae5	movq	0x54c(%rip), %rax       ## literal pool symbol address: _kMDItemTitle
0000000000000aec	movq	(%rax), %rsi
0000000000000aef	movq	%r14, %rdi
0000000000000af2	callq	0xc1a                   ## symbol stub for: _CFDictionaryGetValue
0000000000000af7	testq	%rax, %rax
0000000000000afa	je	0xb11
0000000000000afc	movq	0x535(%rip), %rcx       ## literal pool symbol address: _kMDItemTitle
0000000000000b03	movq	(%rcx), %rsi
0000000000000b06	movq	%rbx, %rdi
0000000000000b09	movq	%rax, %rdx
0000000000000b0c	callq	0xc20                   ## symbol stub for: _CFDictionarySetValue
0000000000000b11	movq	0x518(%rip), %rax       ## literal pool symbol address: _kMDItemTextContent
0000000000000b18	movq	(%rax), %rsi
0000000000000b1b	movq	%r14, %rdi
0000000000000b1e	callq	0xc1a                   ## symbol stub for: _CFDictionaryGetValue
0000000000000b23	testq	%rax, %rax
0000000000000b26	je	0xb3d
0000000000000b28	movq	0x501(%rip), %rcx       ## literal pool symbol address: _kMDItemTextContent
0000000000000b2f	movq	(%rcx), %rsi
0000000000000b32	movq	%rbx, %rdi
0000000000000b35	movq	%rax, %rdx
0000000000000b38	callq	0xc20                   ## symbol stub for: _CFDictionarySetValue
0000000000000b3d	movq	0x4dc(%rip), %rax       ## literal pool symbol address: _kMDItemDisplayName
0000000000000b44	movq	(%rax), %rsi
0000000000000b47	movq	%r14, %rdi
0000000000000b4a	callq	0xc1a                   ## symbol stub for: _CFDictionaryGetValue
0000000000000b4f	testq	%rax, %rax
0000000000000b52	je	0xb69
0000000000000b54	movq	0x4c5(%rip), %rcx       ## literal pool symbol address: _kMDItemDisplayName
0000000000000b5b	movq	(%rcx), %rsi
0000000000000b5e	movq	%rbx, %rdi
0000000000000b61	movq	%rax, %rdx
0000000000000b64	callq	0xc20                   ## symbol stub for: _CFDictionarySetValue
0000000000000b69	xorl	%edi, %edi
0000000000000b6b	movl	$0x3, %edx
0000000000000b70	xorl	%esi, %esi
0000000000000b72	movl	$0x3, %ecx
0000000000000b77	callq	0xc0e                   ## symbol stub for: _CFDateFormatterCreate
0000000000000b7c	movq	%rax, %r13
0000000000000b7f	movq	0x4a2(%rip), %rax       ## literal pool symbol address: _kMDItemLastUsedDate
0000000000000b86	movq	(%rax), %rsi
0000000000000b89	movq	%r14, %rdi
0000000000000b8c	callq	0xc1a                   ## symbol stub for: _CFDictionaryGetValue
0000000000000b91	testq	%rax, %rax
0000000000000b94	je	0xbe2
0000000000000b96	testq	%r13, %r13
0000000000000b99	je	0xbe2
0000000000000b9b	movq	%rax, %r14
0000000000000b9e	leaq	0x226(%rip), %rdi       ## literal pool for: "trying to parse date "
0000000000000ba5	callq	0xc86                   ## symbol stub for: _puts
0000000000000baa	xorl	%edi, %edi
0000000000000bac	movq	%r13, %rsi
0000000000000baf	movq	%r14, %rdx
0000000000000bb2	xorl	%ecx, %ecx
0000000000000bb4	callq	0xc14                   ## symbol stub for: _CFDateFormatterCreateDateFromString
0000000000000bb9	movq	%rax, %r14
0000000000000bbc	leaq	0x21e(%rip), %rdi       ## literal pool for: "got cur date"
0000000000000bc3	callq	0xc86                   ## symbol stub for: _puts
0000000000000bc8	testq	%r14, %r14
0000000000000bcb	je	0xbe2
0000000000000bcd	movq	%r14, %rdx
0000000000000bd0	movq	0x451(%rip), %rax       ## literal pool symbol address: _kMDItemLastUsedDate
0000000000000bd7	movq	(%rax), %rsi
0000000000000bda	movq	%rbx, %rdi
0000000000000bdd	callq	0xc20                   ## symbol stub for: _CFDictionarySetValue
0000000000000be2	movb	$0x1, %bl
0000000000000be4	movq	%r12, %rdi
0000000000000be7	callq	0xc3e                   ## symbol stub for: _CFReadStreamClose
0000000000000bec	movq	%r12, %rdi
0000000000000bef	callq	0xc50                   ## symbol stub for: _CFRelease
0000000000000bf4	movq	%r15, %rdi
0000000000000bf7	callq	0xc50                   ## symbol stub for: _CFRelease
0000000000000bfc	movzbl	%bl, %eax
0000000000000bff	addq	$0x18, %rsp
0000000000000c03	popq	%rbx
0000000000000c04	popq	%r12
0000000000000c06	popq	%r13
0000000000000c08	popq	%r14
0000000000000c0a	popq	%r15
0000000000000c0c	popq	%rbp
0000000000000c0d	retq

------------------------------------------------------------------------


Here is TB 30.0b1 (mdimporter executable is still named "thunderbird" but now appears to not have code):

Thunderbird\ 30.0b1.app/
├── Contents
│   ├── CodeResources
│   ├── Info.plist
│   ├── Library
│   │   └── Spotlight
│   │       └── thunderbird.mdimporter
│   │           └── Contents
│   │               ├── Info.plist
│   │               ├── MacOS
│   │               │   └── thunderbird   <----mdimporter executable
│   │               └── Resources
│   │                   ├── English.lproj
│   │                   │   ├── InfoPlist.strings
│   │                   │   └── schema.strings
│   │                   └── schema.xml
│   ├── MacOS
│   │   ├── XUL
│   │   ├── application.ini
│   │   ├── blocklist.xml
│   │   ├── crashreporter.app
(rest of TB.app tree omitted)

otool -tV output for 30.0b1 (seems like no code in the executable):

Thunderbird 30.0b1.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:
(__TEXT,__text) section

------------------------------------------------------------------------

This is the current release (31.4.0) directly from my Applications folder. At this point the mdimporter executable is named "thunderbird":

Thunderbird.app/
├── Contents
│   ├── CodeResources
│   ├── Info.plist
│   ├── Library
│   │   └── Spotlight
│   │       └── thunderbird.mdimporter
│   │           └── Contents
│   │               ├── Info.plist
│   │               ├── MacOS
│   │               │   └── thunderbird   <----mdimporter executable
│   │               └── Resources
│   │                   ├── English.lproj
│   │                   │   ├── InfoPlist.strings
│   │                   │   └── schema.strings
│   │                   └── schema.xml
│   ├── MacOS
│   │   ├── XUL
│   │   ├── application.ini
│   │   ├── blocklist.xml
│   │   ├── crashreporter.app
(rest of TB.app tree omitted)

otool -tV showing the disassembled mdimporter for the current release, 31.4.0, shows nothing:

Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird:
(__TEXT,__text) section

------------------------------------------------------------------------

FYI, I think I looked at all of the release 31 versions and the mdimporter was lacking code in every one though it was still included and called "thunderbird".

------------------------------------------------------------------------


Here is 34.0b1. The mdimporter name is now named "thunderbird-mdimport" in this release, changed from just "thunderbird" at some point and still lacking object code:

Thunderbird\ 34.0b1.app/
└── Contents
    ├── Info.plist
    ├── Library
    │   └── Spotlight
    │       └── thunderbird.mdimporter
    │           └── Contents
    │               ├── Info.plist
    │               ├── MacOS
    │               │   └── thunderbird-mdimport   <----mdimporter executable
    │               ├── Resources
    │               │   ├── English.lproj
    │               │   │   ├── InfoPlist.strings
    │               │   │   └── schema.strings
    │               │   └── schema.xml
    │               └── _CodeSignature
    │                   └── CodeResources
    ├── MacOS
    │   ├── XUL
    │   ├── crashreporter.app
    │   │   └── Contents
(rest of TB.app tree omitted)

otool -tV:

MacOS/thunderbird-mdimport
Thunderbird 34.0b1.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport:
(__TEXT,__text) section

(again, nothing)

------------------------------------------------------------------------

Here's 36.0b1: now the mdimporter structure is partially there but the executable has disappeared (and I didn't see it anywhere in the rest of the TB tree):

Thunderbird\ 36.0b1.app/
└── Contents
    ├── Info.plist
    ├── Library
    │   └── Spotlight
    │       └── thunderbird.mdimporter
    │           └── Contents        <-(should MacOS/(mdimporter executable) is now completely gone)
    │               ├── Info.plist
    │               └── Resources
    │                   ├── English.lproj
    │                   │   ├── InfoPlist.strings
    │                   │   └── schema.strings
    │                   └── schema.xml
    ├── MacOS
    │   ├── XUL
    │   ├── crashreporter.app
    │   │   └── Contents
    │   │       ├── Info.plist
    │   │       ├── MacOS
    │   │       │   └── crashreporter
(rest of TB.app tree omitted)

(There is no apparent mdimporter executable to run otool -tV on for TB 36.0b1)

------------------------------------------------------------------------

Now that I look at them in order I'm wondering if at some point the TB mdimporter name was changed to "thunderbird-mdimport" but when the build is constructed or updated perhaps there is a place where the name wasn't changed, remaining "thunderbird" (possibly containing no executable code) and so the final mdimporter (whatever it's supposed to be called) never gets the generated code copied into it? 

I think I saw a bugzilla entry where there was mention that something else couldn't be called "thunderbird" because the mdimporter had gotten that name; perhaps someone was attempting to "correct" the naming?

I also think I saw somewhere in bugzilla, a suggestion that the mdimporter be removed from the testing for some reason; might that be why it isn't showing up in the betas and nightlies?

I hope this helps some. Sorry I haven't managed to find time to run through the nightlies to see exactly when things went wrong. It seems to me, though, that the mdimporter hasn't been functioning since TB release 24 (and, I think, release 31 was the next real release since 24.
Flags: needinfo?(Nomis101)
(Assignee)

Comment 27

4 years ago
(In reply to Zhora VandenBout from comment #26)
> I think I saw a bugzilla entry where there was mention that something else
> couldn't be called "thunderbird" because the mdimporter had gotten that
> name; perhaps someone was attempting to "correct" the naming?

I think you mean Bug 1045026.
Flags: needinfo?(Nomis101)

Comment 28

4 years ago
(In reply to Nomis101 from comment #27)
> (In reply to Zhora VandenBout from comment #26)
> > I think I saw a bugzilla entry where there was mention that something else
> > couldn't be called "thunderbird" because the mdimporter had gotten that
> > name; perhaps someone was attempting to "correct" the naming?
> 
> I think you mean Bug 1045026.

Yes, that's the one.

I let TB automaticly update my installed TB (31.4.?) to 31.5.0 and the TB mdimporter (still named "thunderbird") now contains code works correctly for me. The original error for this problem that showed up in the console log is also gone. I think that means that the current bug (Bug 1011449) is now basically resolved (except maybe for any testing that needs done or anything changes required to propagate it to later versions and since I don't know the procedure I'm not doing anything of that sort myself).

I see you opened Bug 1134867 about the missing mdimporter module in later versions but I'm mentioning it since it came up here and may interact with fixes to this bug. I'll add things about it there.

Question: this bug blocks Bug 1008543 - (TB31found) Bugs found prior to 31 release with betas from TB30 and TB31 [meta] - which doesn't block anything else and hasn't changed since last August. Is it still useful now? Or was it just an abandoned at something useful that was was found ineffective or just dropped along the way? It might be a standard procedure and I apologize for not being more familiar with such things or if I'm asking in the wrong place. I was wondering if it would be good to close or change it since it makes the bugs it depends on look like they are actually blocking TB in some way but really those blocks accomplish nothing? Maybe its useful as a way to group the set of both completed and uncompleted bugs that affect betas 30 and 31, now released, but it seems more like something that confuses things at this point.
(Assignee)

Comment 29

4 years ago
Zhora, could you check with the most recent nightly, which has Bug 1134867 fixed?

Comment 30

4 years ago
(In reply to Nomis101 from comment #29)
> Zhora, could you check with the most recent nightly, which has Bug 1134867
> fixed?

I'm still getting the hang of the organization of the various builds/versions/etc. so I'm including complete info about what I downloaded; so definitely make sure I checked the one you wanted me to .

Downloaded nightly: thunderbird-39.0a1.en-US.mac.dmg
Containing: Daily.app  Version: 39.0a1

Location: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2015-03-07-03-02-30-comm-central/thunderbird-39.0a1.en-US.mac.dmg

The application has all of the expected files for the thunderbird mdimporter. The mdimporter is using name "thunderbird.mdimporter" that I believe was changed in release 36.?.? (but not totally sure of that number). Though all of the files now exist and the mdimporter still shows the same version number as earlier (1.0), "otool -tV" displays no usable code in the file. I haven't tested it out (since I keep messing up my working TB app when I do) I'm pretty sure that the problem of this bugzilla report is still happening (note following!); I'll actually install the nightly and check out the mdimporter later since anything I find with my 5:30am brain would need to be questioned.

In double checking things while doing looking at the above I discovered that the mdimporter is still not working in 31.5 (though I had thought it was and said that earlier) and that when TB automatically updates one's working TB app there are no checks to see if the new version contains exactly the correct files that should be in the new release. After looking at why my auto updated release 31.5.0 version wasn't the same as the 31.5 version I manually downloaded I finally saw that mine contained an older mdimporter that I had grafted into the 31.4 release to get it to index files. For some reason my mdimporter wasn't replaced by the bad one in 31.5 though perhaps a correction was made which seems unlikely from a distribution point of view (but some dates are still confusing me so I can't be sure). Though the problem may have been caused by my changes to the app on my machine I think not having some checks of some sort (such as checksums) with perhaps the possibility to not write over the changes (should the user have made changes she wants to keep) could be considered a bug. Of course we don't have to be responsible for changes a user makes to an app that gets updated but if the changes caused the problem then the same issue would apply if the application bundle becomes corrupted, either accidentally or maliciously and at least the user should be informed that the resulting app is different than how it was created to be. (I'll file a bugzilla on that later.)

Anyway, due to the update fluke, my report that the mdimporter was now working turns out to be wrong and why there is no code in the midimporter module still needs to be determined and fixed.

Comment 31

4 years ago
Symptoms reproduced with Thunderbird 31.6.0 on OS X 10.9.5. 

Also, for a short while I tried Earlybird 39.0a2 on Yosemite with a ZFS home directory. I didn't take a Console view of all messages but I did find that Spotlight could not find individual messages. 

In both cases, as far as I recall, a Spotlight search for part of a subject line found the .msf file for the mailbox (not the message). That match is categorised as 'Other'. 

If it's relevant: I use ExQuilla for Microsoft Exchange <https://exquilla.zendesk.com/home> 31.0.1136. 

Question
========

Has anyone reproduced this bug with Mountain Lion or less?
(Assignee)

Comment 32

4 years ago
(In reply to Zhora VandenBout from comment #24)
  > /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.
> mdimporter/Contents/MacOS/thunderbird:
>     (__TEXT,__text) section
> ------------------------------------------
> 
> That's all…pretty much nothing! So it seems that something is going wrong in

After some testing I blame Bug 944952 for this issue. If I reverse this changes I get a normal output of this build with otool. But currently I have no idea whats wrong and how to fix this...
(Assignee)

Comment 33

4 years ago
Maybe this is the reason and it is needed?
http://hg.mozilla.org/comm-central/rev/24c3a939f322#l3.33
I will test this...

Comment 34

4 years ago
My Yosemite console error message is similar, but not identical:

2015-08-28 10:49:34.983 AM mdworker[15093]: (Import.Error:867) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Is this the same bug?

Comment 35

4 years ago
I get this on El Capitan on TB 38.3.0:
Oct 19 14:25:13 jarmac mdworker[7998]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7f9bf530dc20 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)
Oct 19 14:25:13 jarmac mdworker[7998]: (Import.Error:722) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
Oct 19 14:25:13 jarmac mdworker[7998]: (Import.Error:878) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Comment 36

4 years ago
Same problem here on 10.9.5 (Mavericks) with TB 38.3.0

mdworker[13147]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7fdc12515e00 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)

mdworker[13147]: (Error) Import: Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'

mdworker[13147]: (Error) Import: BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Comment 37

4 years ago
Confirming problem, TB 38.3.0 and El Capitan 10.11.1 --- console messages essentially identical to those found by Atlanx under Mavericks:

mdworker[1039]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7fe6984260c0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)

mdworker[1039]: (Import.Error:722) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'

mdworker[1039]: (Import.Error:878) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Comment 38

3 years ago
This still doesn't work for TB 44.0beta and El Capitan 10.11.2. 

The key issue is this log message:

mdworker[1039]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7fe6984260c0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)

That's indicating the Spotlight, when it goes to load Thunderbird importer plug in, can't locate the global symbol MetadataImporterPluginFactory because it isn't there.

If you examine /Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/Contents/MacOS/thunderbird-mdimport with nm -gU, you see that there are no global symbols defined. 

If you look at other Spotlight plugs, all have at least one global symbol defined, the one corresponding to whatever the Info.plist file specifies as the function name. For Thunderbird, it's this:

        <key>CFPlugInFactories</key>
        <dict>
                <key>37401ADE-1058-42DB-BBE5-F2AAB9D7C13E</key>
                <string>MetadataImporterPluginFactory</string>
        </dict>

Change the build options to keep the MetadataImporterPluginFactory symbol intact and you'll have a plugin that loads.

Comment 39

3 years ago
I'm getting the same problem, but this isn't just a symbol table problem, the thunderbird.mdimporter seems to be pretty empty - no entry points, no symbol table, basically a valid executable, but an empty one. It's only 34k in size.

Here's the error from the console:

19/01/2016 18:42:15.739 mdworker[14604]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7fd991517ed0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)

Here's the ouput of otool -f
Fat headers
fat_magic 0xcafebabe
nfat_arch 2
architecture 0
    cputype 16777223
    cpusubtype 3
    capabilities 0x0
    offset 4096
    size 13584
    align 2^12 (4096)
architecture 1
    cputype 7
    cpusubtype 3
    capabilities 0x0
    offset 20480
    size 13568
    align 2^12 (4096)

... and the output of nm -a
                 U dyld_stub_binder
(Assignee)

Comment 40

3 years ago
(In reply to Saul Tannenbaum from comment #38)
> Change the build options to keep the MetadataImporterPluginFactory symbol
> intact and you'll have a plugin that loads.

Do you think you can provide a patch with this suggestion?

Comment 41

3 years ago
I've recently begun seeing these messages in Console for TB 45.1.1.


6/30/16 4:53:21.296 AM mdworker[15104]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7ffb884111c0 </Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter> (bundle, loaded)
6/30/16 4:53:21.297 AM mdworker[15104]: (Import.Error:722) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
6/30/16 4:53:21.297 AM mdworker[15104]: (Import.Error:878) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/

Comment 42

3 years ago
I'm seeing it in TB 45.2.

7/13/16 3:53:31.949 AM mdworker[52146]: (Import.Error:878) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/
The issue still exists on 10.9.5 (El Capitan) with TB 52.0.1

Apr 12 14:26:12 BeBox mdworker[1766]: Cannot find function pointer MetadataImporterPluginFactory for factory 37401ADE-1058-42DB-BBE5-F2AAB9D7C13E in CFBundle/CFPlugIn 0x7ffd03e10840 </Applications/Thunderbird.app/Contents/Library/Spotlightthunderbird.mdimporter> (bundle, loaded)
Apr 12 14:26:12 BeBox mdworker[1766]: (Import.Error:722) Could not create instance for plugIn 'file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/'
Apr 12 14:26:12 BeBox mdworker[1766]: (Import.Error:878) BAD IMPORTER ~~~~~~~~~~~~~~~~~~ file:///Applications/Thunderbird.app/Contents/Library/Spotlight/thunderbird.mdimporter/


Also, it can be reproduce on 10.12.4 (Sierra) with TB 52.0.1.

Comment 44

2 years ago
Jcranmer, you seem to be the author of the suspect bug 944952, can you please look at this?
Flags: needinfo?(Pidgeot18)
Javi too perhaps
Blocks: tb-mac
Whiteboard: [regression:TB31]
I vote for reverting changes in bug 944952.

Don't know what advantage is providing removing the use of Xcode for this part of the code.

I was trying to backout those changes, but there were many merge conflicts and the building was failing. I have another priority right now in Calendar so cannot do anything else.

However, if :jcranmer or whoever provides me a patch which both applies and compiles, then I will be glad to test it.
(In reply to :aceman from comment #44)
> Jcranmer, you seem to be the author of the suspect bug 944952, can you
> please look at this?

I know almost nothing of OS X-specific build system details, particularly when it comes to this sort of code. The original patch was developed largely by begging the contents of the build system from some people, finding the relevant files, and using diffs until that portion of the build system saw no changes.

(In reply to Javi Rueda from comment #46)
> Don't know what advantage is providing removing the use of Xcode for this
> part of the code.

It means we're not reliant on an insane, fragile build system that probably blocks future automation improvements. (e.g., I'm quite sure it would have broken if we moved OS X to a cross-compile-from-Linux build environment). It probably also blocked the c-c/m-c merger work I was doing back then, although I don't recall the basic deatils.
Flags: needinfo?(Pidgeot18)
(Assignee)

Comment 48

2 years ago
(In reply to Javi Rueda from comment #46)
> However, if :jcranmer or whoever provides me a patch which both applies and
> compiles, then I will be glad to test it.

I tried very hard. But I don't get it to build. This attached patch applies, but it always fails with:

ld: entry point (_main) undefined. for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** [thunderbird-mdimport] Error 1
make[3]: *** [mail/components/search/mdimporter/target] Error 2
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [compile] Error 2
make[1]: *** [default] Error 2
make: *** [build] Error 2

I have no idea why this is. I tried a lot of different combinations, I every time get this linker error. If somebody can tell me why this fails, I can hopefully make a patch that also builds. Currently it just applies.
Assignee: nobody → Nomis101
You need to log in before you can comment on or make changes to this bug.