1644027 - messages.getFull and getRaw for news message

plieser

Reporter

Description

•

4 years ago

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Get a MessageHeader via browser.messageDisplay.onMessageDisplayed for news message inside a nntp account.
I tried to get the content by calling message.getFull(id) or message.getRaw(id)

Actual results:

The promises returned by getFull() and getRaw() never get out of the pending state.

Expected results:

The promises should get resolved with the message content. Or at least get rejected with an error.

jan.kiszka

Comment 1

•

4 years ago

Confirmed over 78.4.0 as well. In some cases, I get "TypeError: aPart is null MimeMessage.jsm:134:7" for getFull.
In addition, the MessageHeader is also not fully populated, lacking receivers, CC lists, possibly more.

Any chance that this get fixed? Makes the WebExtension API unusable for my add-on, enforcing own API extensions again.

max

Comment 2

•

3 years ago

nntp://news.gmane.io is remaining an extremely convenient interface
to mail lists even though HTTP interface of gmane.org was shut down
long time ago. It is possible to follow mail lists without
subscription. Archives are available, so there is no problem
to access from mail client messages sent before subscription to
a mail list.

My hope was that I could easily write an extension that extracts
useful headers from messages for my notes. Reality is that
only a few of headers are directly available. Even to get "To"
or "newsgroups" headers, it is necessary to query for a full message,
not to say about links to http archive or gmane message id.
I do not mind to have "Date" in the timezone of the sender.
Even "Message-Id" is available in nightly but not in the thunderbird-68.10
shipped with Ubuntu-20.04 focal (current long term support release).
The final disappointment is that messages.getFull(id) does not work at all
and messages.getRaw(id) is rather unstable.
I was surprised that attempt get details of message that is displayed
on the screen could fail due to timeout. It is a pity that the only
reliable way is to open message source and to cherry pick interesting
headers using mouse.

Steps to reproduce.

Setup news account for news.gmane.io
subscribe to some group, e.g. gmane.linux.debian.user.announce.
To factor out enigmail/OpenPGP issues, some other group could be used, e.g.
gmane.comp.mozilla.thunderbird.user
Load temporary extension
invoke extension action "getRaw" through the context menu for message list.
At first, let's try to do it without displaying the message
using left click on it.
inspect extension and thunderbird console messages.

Test extension manifest.json

{
	"manifest_version": 2,
	"name": "getFull test",
	"version": "0.1",
	"permissions": [
		"messagesRead", "menus"
	],
	"background": {
		"scripts": [ "background.js" ]
	}
}

background.js

async function testActionHandler(info, tab) {
	let selectedMessages = info && info.selectedMessages
		&& info.selectedMessages.messages;
	for (const message of selectedMessages) {
		if (info.menuItemId === "TEST_GET_FULL") {
			console.log("calling getFull() for message %o", message);
			const full = await browser.messages.getFull(message.id);
			console.log(full);
		} else if (info.menuItemId === "TEST_GET_RAW") {
			console.log("calling getRaw() for message %o", message);
			const raw = await browser.messages.getRaw(message.id);
			console.log(raw.substring(0, 1024));
		} else {
			console.error("testActionHandler: unsupported action %o", info.menuItemId);
		}
	}
}
function testMain() {
	browser.runtime.onInstalled.addListener(function testCreateMenu() {
		browser.menus.create({
			contexts: [ "message_list", "selection", "link", "page", "frame" ],
			id: "TEST_GET_FULL",
			title: "getFull test",
		});
		browser.menus.create({
			contexts: [ "message_list", "selection", "link", "page", "frame" ],
			id: "TEST_GET_RAW",
			title: "getRaw test",
		});
	});

	browser.menus.onClicked.addListener(function testMenuListener(info, tab) {
		console.debug("testMenuListener", info, tab);
		testActionHandler(info, tab).catch(ex => console.error("testMenuListener:", ex));
	});
}

testMain();

Extension console (thunderbird 87.0a1 2021-02-21)

testMenuListener 
Object { menuItemId: "TEST_GET_FULL", parentMenuItemId: 0, viewType: "tab", editable: false, pageUrl: undefined, selectedMessages: {…}, modifiers: [], button: 0 }
 
Object { id: 1, index: 0, windowId: 3, highlighted: true, active: true, status: "complete", width: 823, height: 664, mailTab: true }
background.js:32:11
calling getFull() for message  
Object { id: 1, date: Date Thu Jul 09 2020 16:24:11 GMT+0000 (Coordinated Universal Time), author:  … }
background.js:6:11
testMenuListener: Error: An unexpected error occurred background.js:33:52
    testMenuListener moz-extension://010f5daf-3f51-463e-9b7b-1818baa2cbc9/background.js:33
    (Async: promise callback)
    testMenuListener moz-extension://010f5daf-3f51-463e-9b7b-1818baa2cbc9/background.js:33

Thunderbird console

[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIMsgMessageService.streamMessage]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource:///modules/gloda/MimeMessage.jsm :: MsgHdrToMimeMessage :: line 236"  data: no] MimeMessage.jsm:236:16
    MsgHdrToMimeMessage resource:///modules/gloda/MimeMessage.jsm:236
    getFull chrome://messenger/content/parent/ext-messages.js:212
    getFull chrome://messenger/content/parent/ext-messages.js:210
    getFull self-hosted:1173
    result resource://gre/modules/ExtensionParent.jsm:956
    withPendingBrowser resource://gre/modules/ExtensionParent.jsm:489
    result resource://gre/modules/ExtensionParent.jsm:956
    callAndLog resource://gre/modules/ExtensionParent.jsm:918
    recvAPICall resource://gre/modules/ExtensionParent.jsm:955
    AsyncFunctionNext self-hosted:690

Does not work, but at least the promise has been rejected.

Now try to display some message by left click on the entry in the message list
Invoke "getRaw" context menu for the same item.

Silent failure, the promise is pending forever. Another error in logged
to thunderbird console:

Did you try to stream an nttp message? Doens't work. See bug 545365. MimeMessage.jsm:121:13
    onDataAvailable resource:///modules/gloda/MimeMessage.jsm:121
TypeError: right-hand side of 'in' should be an object, got nullext-messages.js:54:9
    convertMessagePart chrome://messenger/content/parent/ext-messages.js:54
    getFull chrome://messenger/content/parent/ext-messages.js:216
    onStopRequest resource:///modules/gloda/MimeMessage.jsm:95

Try extension "getRaw" action twice for the same message.
For the first time it works, next attempts leads to silent failures.

My expectations:

messages.getFull and messages.getRaw works for news (NNTP) messages.
no hanged promises, any failures cause promise rejection.
Error messages are informative enough to figure out what has happened,
not just something like "unexpected error".

P.S. In my particular use case, I do not need message body or attachments,
it is enough to fetch just message headers.

max

Comment 3

•

3 years ago

As to the following error

TypeError: right-hand side of 'in' should be an object, got nullext-messages.js:54:9
    convertMessagePart chrome://messenger/content/parent/ext-messages.js:54
    getFull chrome://messenger/content/parent/ext-messages.js:216
    onStopRequest resource:///modules/gloda/MimeMessage.jsm:95

that appears in thunderbird (not extension) console, it is due to insufficient error handling in convertMessagePart https://hg.mozilla.org/comm-central/file/tip/mail/components/extensions/parent/ext-messages.js#l54 To get such message, try "getFull" (e.g. using test extension from the previous comment) for a NNTP message that is displayed or for a NNTP message for that "getRaw" was called earlier. The function is not ready that its argument could be null. I suppose, it should just return in this case.

There is more severe problem with getFull implementation https://hg.mozilla.org/comm-central/file/tip/mail/components/extensions/parent/ext-messages.js#l216 It is not ready that convertMessagePart() could throw an exception. Such exception must cause rejection of the promise. If mimeMsg at this point is null than it could be considered as a reason to reject the promise per se. I am unsure if try-catch around msgService.streamMessage is enough in https://hg.mozilla.org/comm-central/file/tip/mailnews/db/gloda/modules/MimeMessage.jsm#l236 or return value should be checked as well.

I believe, neither fulfilled nor rejected promise is a serious error.

I have filed the Bug #1694988 "nsNntpService::StreamMessage does not support aConvertData argument". It is the actual origin of the errors described above. Without a fix of #1694988, messages.getFull() would not work. The only hope is to get at least an error reliably.

max

Comment 4

•

3 years ago

In my opinion, unconditionally passing null as the aAllowDownload argument of MsgHdrToMimeMessage is questionable, see https://hg.mozilla.org/comm-central/file/tip/mail/components/extensions/parent/ext-messages.js#l218 If an NNTP message has not been retrieved before (message neither has been displayed nor e.g. message.getRaw() has been called), then the following error appears in thunderbird console

[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIMsgMessageService.streamMessage]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource:///modules/gloda/MimeMessage.jsm :: MsgHdrToMimeMessage :: line 236"  data: no] MimeMessage.jsm:236:16
    MsgHdrToMimeMessage resource:///modules/gloda/MimeMessage.jsm:236
    getFull chrome://messenger/content/parent/ext-messages.js:212
    getFull chrome://messenger/content/parent/ext-messages.js:210
    getFull self-hosted:1173
    result resource://gre/modules/ExtensionParent.jsm:956
    withPendingBrowser resource://gre/modules/ExtensionParent.jsm:489
    result resource://gre/modules/ExtensionParent.jsm:956
    callAndLog resource://gre/modules/ExtensionParent.jsm:918
    recvAPICall resource://gre/modules/ExtensionParent.jsm:955
    InterpretGeneratorResume self-hosted:1480
    AsyncFunctionNext self-hosted:690

Promise returned to an extension is rejected but with obscure error "Error: An unexpected error occurred"

WIP: Bug 1644027 - Fix messages.getFull() for news. 3 years ago John Bieling (:TbSync) 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1644027 - Fix messages.getFull() and attachment support for news by adding a full MIME emitter to jsmime.js based MimeParser.jsm. r=mkmelin 3 years ago John Bieling (:TbSync) 48 bytes, text/x-phabricator-request		Details \| Review