336049 - S/MIME signature is removed from messages sent via mail to a mailing list

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task, P1)

Description

[Frank Wein [:mcsmurf]]

Since the binary filter was turned on for the mailing lists, S/MIME signatures are removed from the message and this leads to confusion for some newsreaders. One example for such a message is news://news.mozilla.org/[e-mail]. I don't think those signatures should be removed, PGP/GPG ones are allowed, too :). I think the same is true when posting directly to the newsgroups via news.mozilla.org, but (at least with the Gecko-based newsreaders) posting a S/MIME signed news message is not possible at all or very difficult anyway. Maybe that can also be fixed.

Comment 1

[:Gavin Sharp [email: [e-mail]]]

See also bug 335409.

Comment 2

[Chris Ilias [:cilias]]

Moving to Server Ops

Comment 3

[Dave Miller [:justdave]]

It's all done by mime type.  I need a mime-type.

Comment 4

[Frank Wein [:mcsmurf]]

Something like Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms020204010205030401020004" is the mime type. So maybe it should look for multipart/signed; protocol="application/x-pkcs7-signature"

Comment 5

[Mike Shaver (:shaver emeritus)]

I think multipart/signed and multipart/mixed should be handled at the outer level, and then application/x-pkcs7-signature to preserve the signature as the component of the multipart/mixed.

Comment 6

[Chris Ilias [:cilias]]

If the end-goal is to align the lists.mozilla.org filters with the news.mozilla.org filters, it should be pointed out that Giganews filters out S/MIME signed posts. It doesn't remove the sig, but rejects the entire post. Google Groups does the same.

Comment 7

[Mike Shaver (:shaver emeritus)]

Hrm.  http://groups.google.com/group/mozilla.dev.planning/msg/3ed2d39b584f710f is S/MIME signed, and is right there in groups.google.com.  Hard as hell to read, though!

If we can't preserve the S/MIME stuff when we propagate to news from mail, though, then we should strip it more effectively/cleanly, and put in a comment indicating that the signature was removed.  We should talk to giganews about getting the S/MIME stuff through to at least the news server, even if we can't get it to groups.google as well.

Comment 8

[Dave Miller [:justdave]]

multipart/signed is already allowed, we added that one so it would let GPG/MIME get through.  The mime type off the actual signature part is what I need.  I will add that type on our gateway, then we can test and see if they get through or not, and if they don't, I'll open a ticket with Giganews to see if we can get them to add it as an accepted type on their end.

Comment 9

[Dave Miller [:justdave]]

This had been sitting here waiting for an answer to my question in comment 8, but I just re-read the message and realized my question had already been answered in comment 5.

application/x-pkcs7-signature has been added to the allowed attachment types on the list server, and I have opened a ticket with Giganews to request they whitelist that type.

Comment 10

[Dave Miller [:justdave]]

Giganews would like some test messages posted that they can look at for examples.  They said they'd be happy to do this but it'll probably require code changes, because they aren't actually looking at mime types apparently, and it's probably tripping something on accident.  Even if the posts don't actually show up on news.mozilla.org, it should still go into the queue on the back end, and they can get it from there with a message ID.

So if someone could volunteer to post a few (both through a mail gateway and a direct post) and paste the message IDs here, that'd be great.

Comment 11

[Chris Ilias [:cilias]]

Okay, I had to set mail.identity.default.generate_news_message_id=true, to do it in Thunderbird. :-)

Post to mozilla.test, via news.mozilla.org:
Message-ID: <[e-mail]>

Post to [e-mail]:
Message-ID: <[e-mail]>
(although, the [e-mail] message doesn't appear as signed)

Comment 12

[Dave Miller [:justdave]]

received word from Giganews that the code changes needed to support this have been completed and tested, and they expect to push it into production in about a week.

Comment 13

[Dave Miller [:justdave]]

got this from Giganews:

> We pushed this out today.  I apologize for the delay, but we had to hold it
> while we finished pushing out the retention upgrade.  It's out.  We've tested
> with the e-mail sent to us.  If you notice any problems with PGP signed posting, 
> please let us know.

Comment 14

[Chris Ilias [:cilias]]

I did some S/MIME tests in mozilla.test.

S/MIME signed messages made via news.mozilla.org or lists.mozilla.org will show up on news.mozilla.org, with the S/MIME sig intact.

S/MIME signed messages made via news.mozilla.org or lists.mozilla.org will get to mailing list subscribers, but the S/MIME sig will instead show as an attachment.
(named "Part 1.2" in Mozilla apps)

S/MIME signed messages made via news.mozilla.org or lists.mozilla.org will not show up on Google Groups at all.

Comment 15

[Mike Shaver (:shaver emeritus)]

(In reply to comment #14)
> S/MIME signed messages made via news.mozilla.org or lists.mozilla.org will get
> to mailing list subscribers, but the S/MIME sig will instead show as an
> attachment.
> (named "Part 1.2" in Mozilla apps)

Reopening, since getting the signature as a non-useful attachment is really not much better than having it being removed.  Probably need another bug for the google groups thing, though...

Comment 16

[Dave Miller [:justdave]]

Yeah, the Google thing is probably a separate issue.  File a new bug for that.

The rest of this sounds like MailMan is screwing something up on the way through.  Could you possibly attach the full headers from one of these, both the list version and the news version of the same message (or mail them to me)?

Comment 17

[Chris Ilias [:cilias]]

Dave, I've emailed you the message source of both the list and news version.
(in case it gets caught by any spam filters, and you don't know I sent it) 

Comment 18

[Dave Miller [:justdave]]

OK, I did a little tweaking on the mailman config with the mime types it'll allow, someone want to try it again and see if it gets through?  If you could CC me on test mails so I see what it should have looked like when it got there it might help, too.

Comment 19

[Dave Miller [:justdave]]

OK, I saw some test messages in my inbox, but I don't appear to be subscribed to the list they went to so I don't know if the list copy succeeded.  Did they?

Comment 20

[Chris Ilias [:cilias]]

They did not succeed. They appear the same a before ("Part 1.2" attachment). Sorry I didn't post earlier. There was a /severe/ latency in my email last night.

Comment 21

[Dave Miller [:justdave]]

ok, I tried to subscribe to the test list again, and my subscription mail hasn't gone through yet.  guess this will sit while I wait for it (or figure out where it went).  I need to do some things this weekend anyway, so I'll hit this again on Monday and have you try again once I can see both pieces of it.

Comment 22

[Dave Miller [:justdave]]

OK, got my subscription stuff settled, care to send the test mail again? (to the list and CC me).  One of these days I'll have to go get me a thawte cert so I can do it myself (I use Enigmail personally)

Comment 23

[Dave Miller [:justdave]]

OK, upon examining the headers of the message I got directly, and the one CCed to the mailing list delivered by Mailman, I'm going to say Mailman is doing exactly what it's supposed to and this is a bug in Thunderbird.

Mailman is adding the list footer to the message.  Because the message is signed, and it doesn't want to interfere with the signature it's wrapping the entire signed portion of the email in a multipart/mixed, with the first part being the existing multipart/signed message, and the second part being the text/plain with the list footer.

It does the same thing with PGP-signed messages, and Enigmail handles this correctly, by saying "part of this message is signed" and having "*** BEGIN SIGNED PORTION ***" and "*** END SIGNED PORTION ***" inserted in the UI when displaying the message.

I'm re-resolving this as fixed since the original problem was the messages not making it to Giganews, and that part's been dealt with.  I suggest filing a bug against Thunderbird for the way it handles partially-signed S/MIME messages.