Closed Bug 1901390 Opened 4 months ago Closed 4 months ago

local large language models do not work with thunderbird

Categories

(Thunderbird :: General, enhancement)

enhancement

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: jaan, Unassigned)

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:126.0) Gecko/20100101 Firefox/126.0

Steps to reproduce:

tried to use local large language models (demo from a talk at lightning.ai i did last week): https://colab.research.google.com/github/onefact/loving-the-baseline/blob/main/nearest-neighbors.ipynb

but this does not work because it is not compatible.

Actual results:

could not use local large language models or closed-source ones like claude.ai to complete or generate email messages or follow-ups in the thunderbird client.

Expected results:

able to use any LLM.

Where are you assuming this would be used and how?

This would be used for local large language models for clinicians, who cannot send data such as clinical notes or progress notes that communicate information about patient status that contains (or could contain) protected health information.

This is defined in the Health Insurance Portability and Accountability Act in the United States' federal law, and in the EU AI Act and GDPR among others - happy to send references.

A significant part of the burden for clinicians is due to vendor lock-in with platforms such as epic.com (they build MyChart, and the cost of procurement is ~$1B for academic medical centers like the ones I've worked at).

Thunderbird + this extension would help accelerate clinical workflows significantly for our patients, patient advocacy groups, care navigators, clinicians, and health systems we work with.

As another example, WhatsApp is a messaging platform backed by Meta (formerly Facebook), and uses the Signal Foundation's Whisper protocol (https://signalfoundation.org/). WhatsApp is one of the most common, if not the most common, way of transmitting patient records, patient data in several low- and middle-income country health systems I have done research in or worked with, from Mexico to rural India.

The resource constraints there mean that notes are often taken on paper, transcription is limited (due to lacking connectivity to 4G to send messages), which requires on-device transcription with tools like Whisper (another Whisper: https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.wasm that you can try on your browser) or WASMEdge (
https://github.com/LlamaEdge/LlamaEdge/blob/main/README.md).

We are building the equivalent for large language models for health care use cases at a non-profit we have formed for this purpose (EIN: 88-2145154; based on work during my PhD: http://arxiv.org/abs/1904.05342).

But the key blocker here is email, specifically inserting text generated from a large language model running on a user's computer. Happy to send more examples like this: https://colab.research.google.com/github/onefact/language-model-notebooks/blob/main/notebooks/getting-started.ipynb

If email is easier: jaan@onefact.org, happy to chat on Zoom too.

Also unrelated to this bug: I am in Gothenburg this week to run workshops here: https://mtflabs.net/just/goco/in case anyone is around :)

Ah so for add-on usage.

But since this is all done inside the add-on, what exactly is it that is not compatible? Thunderbird core doesn't use anything like that, so what would it be compatible with?

Component: Untriaged → General

(In reply to Magnus Melin [:mkmelin] from comment #4)

Ah so for add-on usage.

But since this is all done inside the add-on, what exactly is it that is not compatible? Thunderbird core doesn't use anything like that, so what would it be compatible with?

Thank you for clarifying :)

Most pressingly, the end-to-end encryption protocol to be able to sign and verify keys for sender and receiver, after processing by a large language model (inside the add-on, which you are right does most of the processing :).

Hope that helps! If this is already solved by another add-on, my apologies.

Most pressingly, the end-to-end encryption protocol to be able to sign and verify keys for sender and receiver, after processing by a large language model.

What encryption protocol are you using? Are you loading a JavaScript library into your add-on to sign data? Or are you referring to Thunderbirds native support to sign and encrypt emails (which works without additional add-ons, by enabling this feature in the compose window)?

(In reply to John Bieling (:TbSync) from comment #6)

Most pressingly, the end-to-end encryption protocol to be able to sign and verify keys for sender and receiver, after processing by a large language model.

What encryption protocol are you using? Are you loading a JavaScript library into your add-on to sign data? Or are you referring to Thunderbirds native support to sign and encrypt emails (which works without additional add-ons, by enabling this feature in the compose window)?

Thank you John! We are considering https://virgilsecurity.com/e3kit/ (code: https://github.com/VirgilSecurity/virgil-e3kit-js) and reached out out to their team to see if they have worked with local language models yet.

Do you have any advice on how to select this type of library? Grateful for any advice, we are new and novices to this area 🙇‍♂️🙏

Do you have any advice on how to select this type of library?

I have no experience with those libs, sorry. But other add-on developers might have, so I encourage you to reach out to them through our official add-on support channels: https://developer.thunderbird.net/add-ons/community

I am going to close this bug, as you are not experiencing a bug in Thunderbird.

Status: UNCONFIRMED → RESOLVED
Closed: 4 months ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.