816987 - [email/IMAP] bug in mimelib dep on consecutive base64-encoded mime encoded-words breaks IMAP synchronization due to our Buffer shim's angry base64 decoding

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P2)

Description

[Andrew Sutherland [:asuth] (he/him)]

A recent e-mail from Vivien to the dev-gaia mailing list encoded Stas's e-mail address as cc'ed like so:

=?UTF-8?B?U3RhxZsgTWHFgm9sZQ==?= =?UTF-8?B?cHN6eQ==?= <[e-mail]>

The node "mimelib" library attempts to join mime-words together in decodeMimeWords(), ignoring the whitespace separating them.  While this is correct, it has a sketchy optimization that actually fuses the contents of the mime-encoded words into a single new encoded word.  The bit of RFC 2047 that says "The 'encoded-text' in an 'encoded-word' must be self-contained" could be interpreted as an argument for or against this depending on how you want to deal with people violating that rule.  The key thing here is that concatenating base64 strings is not safe under window.atob or under node's Buffer implementation.

The result of this logic is the allegedly base64 encoded string:

U3RhxZsgTWHFgm9sZQ==cHN6eQ==


Our Buffer shim tries to run window.atob() on this which explodes angrily, as it tends to do.

There are 2 problems here:
1) mimelib shouldn't be trying to concatenate base64 strings like that.  Arguably it shouldn't try concatenating quoted-printable either, although that's up for debate. 
2) our Buffer shim needs to not throw a deadly exception when it sees base64 it doesn't like.  Node's Buffer implementation just skips non-base64 characters.  I think the easiest thing is probably to use a regexp as a scanner to find regions of legal base64 characters.  This will be helpful as we can abandon our (minor) mailparser fork which only remains forked because of that base64 problem.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

pull request to upstream mimelib:
https://github.com/andris9/mimelib/pull/18

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Pull request for the fix is up at:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/91

The github helper extension thing is broken again :(

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

The mimelib upstream pull request was merged.

The gelam patch has r=squib.

Blocking or approval is required to land this patch :( :( :(

I'm going to wait on blocking since that has triage rounds and then go panhandle on IRC for authority.

Comment 5

[Axel Hecht [:Pike]]

I'm very much in favor of blocking+ here.

We're going for international markets, and this blocks people from reading their mail at all. Like, in bug 817402, the issue is that I can't access complete mail folders, as soon as there's one mail in it that triggers this bug.

Comment 6

[Andrew Sutherland [:asuth] (he/him)]

Thanks for the blocking approval!  Fix landed on gelam/master:
https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/91

And then landed on gaia/master:
https://github.com/mozilla-b2g/gaia/pull/6789