Thunderbird and Spotlight

I did a Google search for "Thunderbird Spotlight plugin" and found a deluge of comments from frustrated Thunderbird users, aching for someone to develop such a thing. If anyone knows of a Spotlight plugin for Thunderbird, please speak up! It's the one missing component from my otherwise-delightfully-functional mail set-up and, I'm guessing, I'm not alone.

Comments

[Nic]

Hi,

I have recently started investigations to create a spotlight plugin for Thunderbird. So far into my initial investigations things are not looking good. There is no problem creating the actual spotlight plugin the framework provided by apple is rather straightforward the problem lies in two areas.

1. The format which Thunderbird stores it mail messages (unix standard mBoxrd)
This is a single file and because of this there is a problem to index the messages in spotlight. Spotlight works by indexing certain meta data relating to a file and when you search it cross references your search term with the content in the index and then displays all files that meet the criteria. So if you have one file for all your email messages and you search for [email protected] you will find matches but in the spotlight results only one file the mailbox file will be shown not the individual messages. This is not the end of the world as it is possible to extract the messages and store them in a cache as individual .eml files. I also believe that there is an option to save all messages as individual files. This I guess is why Apple moved away from the mbox format present in Mail 1.x to the individual file method as used in Mail 2.0. The other problem could lie with IMAP folders im not sure if thunderbird stores the messaage or only an index of the header.

2. The problem with Thunderbird opening .eml files
When you double click an eml to open it with thunderbird it does not open correctly the message body is blank. If you open the same file from the menu with open saved message then the file opens perfectly. This I believe is a bug with Thunderbird and a major problem for us as when you click on a found item in spotlight it uses the LanuchServices application to determine which application to open the file with. At present I know of no fancy way to get arround this however I suspect that its possible to do something tricky with generating apple script, And there is always the possiblity to open the message in a text or with mail but this is not good.

My initial findings do not fill me with much hope and im not sure weather I will continue with the investigation. Problem 1 is easy to solve problem 2 may not be and if it is not solveable then the spotlight plugin will only search your messages you will not be able to open them.

I have created a blog to track the progress of my investigations which can be found at the following address : http://www.thatlondon.com/thunderbird

There is an RSS feed so you can keep updated as to the projects progress and I will be posting technical information as soon as I have written it up.

Nic

[Dave]

Thanks, Nic, this is WONDERFUL information!

I'm not sure that the saving all attachments option works because I believe they're saved only when you specifically select that per-attachment. *sigh*

I'm glad to see that someone (you!) have taken on this issue so seriously! You're aware that Google's Summer of Code had this project (Spotlight plugin for Thunderbird) as one of their wishlist??

[junjan]

I am not a programmer, but I was thinking that maybe some clues could be taken from "Google Desktop Search". Wonderful Windows tool (emulating Spotlight) that can search inside Thunderbird mailboxes.

I'm not sure how Google Desktop searches inside Thunderbird's mailboxes. Nic has been doing some research on this. Check out his site at http://www.thatlondon.com/thunderbird.

[eric]

I don't code this kind of stuff, but I can tell you that it is definitely feasible to create a plugin that will allow Spotlight to search and generate results within an MBox file.

I know that, because other vendors have solved the problem. For example, Mariner (http://marinersoftware.com/) has a Spotlight plugin for MacJournal, which natively stores documents as part of massive files. It works well. (I use MacJournal all the time.)

So it will probably happen. Maybe someone could have a quiet conversation with Logan Ryan at Mariner ;-).

[Dave]

If you know Logan, maybe you should have that quiet word? Thanks for your comment, Eric!

[luz]

Nic - thanks for your very helpful investigations. While considering to create such a plugin for Eudora (which has mbox based storage as well) and maybe for it's planned Thunderbird-based opensource successor, Penelope, I found this blog entry.

I also read through the Spotlight developer info, and came to similar conclusions. However, somehow I don't believe that the entire Spotlight concept is really restricted to a one-item-per-file paradigm. The comment on Mariner above suggests that there must be another way.

I also had a look at SearchKit, which does the content indexing for Spotlight as well. I don't really understand yet how EXACTLY these two interoperate (in particular: who calls SKIndexAddDocumentWithText to add the contents to the full text index? Is this done by Spotlight itself when passing back a kMDItemTextContent attribute from the importer? Or is the importer supposed to do that?)

But one thing is interesting - when adding something to the index via SearchKit, it will be related to an URL, not just a file. An URL could carry the mailbox name PLUS a message ID or index within that mailbox.

So maybe there's a way to have a mdimporter create multiple entries per file, and associate these entries with special Thunderbird message URLs (like tbrdmsg://mbox/msgid).
Then this URL type could be associated with thunderbird. And Thunderbird needed an extension to understand these URLs and open the right mbox/message.

just my thoughts to far...