1525274 - Support for raw message source in WebExtensions messages API

Status: RESOLVED FIXED

(Thunderbird :: Add-Ons: Extensions API, enhancement, P2)

Description

[liliakai]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36

Steps to reproduce:

In TB67, browser.messages.getFull will return a JS object representing the parsed mime message.

As an extension developer, I want an API to retrieve the original raw message source, meaning a single Blob containing the full headers and mime-encoded contents, as rendered when selecting "View Source" in the UI.

Comment 1

[Philipp Kewisch [:Fallen][📱📆]]

Can you provide some uses cases you are looking to fulfill with a raw API?

Comment 2

[liliakai]

I maintain a web extension which submits suspicious emails to a 3rd party service for analysis. The service expects raw mime-encoded emails as input.

Comment 3

[Marvin]

I want to do the same thing.... are you possibly talking about the postmark spam check?

Comment 4

[plieser]

I need it for DKIM verification. There I need to calculate a hash over the complete body, and some header fields.

For that I need the complete body as one big unparsed blob (with things like whitespace preserved).

The header fields I also need unparsed, with whitespace (including newlines if it goes over multiple lines) preserved. And if a header field exist multiple times, I need to get them in the same order as in the mail.
Haven't yet looked in detail at the existing web extension API for getting the headers. Could be that it is sufficient for me for getting the headers.

For me it is unimportant whether I get the complete mail as one block (and do the parsing of the headers myself), or just the complete body, and a separate API for getting the (unparsed) header values for the header fileds.

Comment 5

[Philipp Kewisch [:Fallen][📱📆]]

Thanks for the use cases folks. I'm wondering if it makes sense to consolidate get and getFull into one function, e.g.

messenger.messages.get(messageId);
messenger.messages.get(messageId, { mode: "full" });
messenger.messages.get(messageId, { mode: "raw" });

A similar option could be supported in the query and list functions. I'm also not sure what full in getFull means in this context. If raw is fuller than full, maybe it makes sense to use something more explicit?

Comment 6

[Marvin]

my guess is that "full" at least in this context means the complete message, which includes headers and body not just the body or the messageID

Comment 7

[Geoff Lankow (:darktrojan)]

I've stared on this and have a mostly working patch that adds a getRaw function. I don't think it's a good idea to return raw messages for queries as there's a cost to getting them (potentially including downloading from the server, depending on the settings).

Marvin's going to look at writing some tests if we can get him set up to build things.

Comment 8

[Geoff Lankow (:darktrojan)]

Attachment: 1525274-messages-getraw-1.diff

Comment 9

[Magnus Melin [:mkmelin]]

Comment on attachment 9107103 [details] [diff] [review]
1525274-messages-getraw-1.diff

Review of attachment 9107103 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/components/extensions/parent/ext-messages.js
@@ +184,5 @@
> +              },
> +              false,
> +              ""
> +            );
> +          });

Hmm, doesn't this need some error handling for reject?

::: mail/components/extensions/test/xpcshell/test_ext_messages_get.js
@@ +22,5 @@
> +var { PromiseTestUtils } = ChromeUtils.import(
> +  "resource://testing-common/mailnews/PromiseTestUtils.jsm"
> +);
> +
> +// Create a local mail account and add a message.

please use /** */ style comments for the documentations

Comment 10

[Geoff Lankow (:darktrojan)]

Attachment: 1525274-messages-getraw-2.diff

Comment 11

[Magnus Melin [:mkmelin]]

Comment on attachment 9107998 [details] [diff] [review]
1525274-messages-getraw-2.diff

Review of attachment 9107998 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/components/extensions/parent/ext-messages.js
@@ +180,5 @@
> +                {
> +                  OnStartRunningUrl() {},
> +                  OnStopRunningUrl(url, exitCode) {
> +                    if (exitCode !== 0) {
> +                      reject(exitCode);

and return

@@ +191,5 @@
> +              );
> +            });
> +          } catch (ex) {
> +            Cu.reportError(ex);
> +            throw new ExtensionError(`Error reading message ${messageId}`);

Not sure this is 100% correct, is it? Is that an exception getting thrown from the reject?

Could be nicer to just await new Promise(....).catch(reason => throw new ExtensionError(`Error reading message ${messageId}`); );

... or some such?

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/0658b2e48cab
Support for raw message source in WebExtensions messages API; r=mkmelin

Comment 13

[Geoff Lankow (:darktrojan)]

Comment on attachment 9107998 [details] [diff] [review]
1525274-messages-getraw-2.diff

This has been in beta since 72. Let's move it on.

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=a33fdff0980f8bad7810a1ca85f51a041e3deabc

Comment 14

[Wayne Mery (:wsmwk)]

Comment on attachment 9107998 [details] [diff] [review]
1525274-messages-getraw-2.diff

Approved for ESR

Comment 15

[Magnus Melin [:mkmelin]]

Thunderbird 68.7: https://hg.mozilla.org/releases/comm-esr68/rev/43c0d5aa25e181cbe41670cbef9b5844b1a5ce39