553900 - Messages with signatures not archived at groups.google.com

Status: RESOLVED INVALID

(Infrastructure & Operations :: Infrastructure: Mail, task)

Description

[Kai Engert [:KaiE:]]

On Friday 21th I sent two emails from my @kuix.de account 
to dev-security-policy at lists.mozilla.org

I received both messages by email, via my own subscription to that list.

However, the messages didn't show up at

http://groups.google.com/group/mozilla.dev.security.policy/browse_thread/thread/17be3bd7e0b33e8c/d7ee76708d247eef

Is the email-to-newsgroup gateway broken?

Comment 1

[Kai Engert [:KaiE:]]

Attachment: Received message 1 including headers

Comment 2

[Kai Engert [:KaiE:]]

Attachment: Received message 2 including headers

Comment 3

[Eddy Nigg (StartCom)]

Confirming - received one message via CC by mail from Kai, but news doesn't show the message.

Comment 4

[Shyam Mani [:fox2mike]]

Content-Transfer-Encoding: base64

That's your issue. 

Unfortunately, we cannot do anything about base64 encoded messages, they will be dropped by news.mozilla.org.

Comment 5

[Kai Engert [:KaiE:]]

I disagree with your explanation, because:

- the messages contain a plain text part
- I can show that similar messages work

Yes, the message contained a base64 portion, but it was a secondary portion, containing a digital signature.

Just two days earlier I had sent a very similar message, 
sent from the same source account,
sent to dev-tech-crypto at lists.mozilla.org
and it successfully arrived at
http://groups.google.com/group/mozilla.dev.tech.crypto/browse_thread/thread/fb6e52c772b7db42/a19682fe27bc6ddd

That message had the same structure as the dropped messages attached to this bug.
I'll attach the working message (to the other group) that got delivered to me.

It appears the base64 encoded signature got stripped in the working message.

Are lists dev-security-policy and dev-tech-crypto configured differently, perhaps one is configured to filter out signatures to make things work?

Comment 6

[Kai Engert [:KaiE:]]

Attachment: Similar message to different group that worked fine

Comment 7

[Shyam Mani [:fox2mike]]

Kai, 

I'm sorry, but I misread the subject. 

So here's the issue. We don't control Google Groups. In fact, we don't even get replies from them when we report issues. The google groups sync portion of the newsgroups setup is automated. Did the messages arrive on news.mozilla.org? If they did and didn't appear on Google Groups, it's something Google will have to look into and so far I've had zero success getting someone to look into them.

If the messages didn't hit news.mozilla.org, that's because of base64. news.mozilla.org has been setup and run by Giganews and they've told me that due to some technical reasons, base64 encoded stuff is dropped.

In either case, my hands are tied. Both of these things are something that we don't manage directly and hence my ability to help is severely limited.

Also, I don't agree that the messages were the exact same. I don't see a base64 signature or content in your last attached email? While the previous two attachments did have that.

Comment 8

[Kai Engert [:KaiE:]]

(In reply to comment #7)
>
> Also, I don't agree that the messages were the exact same. I don't see a base64
> signature or content in your last attached email? While the previous two
> attachments did have that.

The messages I had sent all contained the base64 part.
(If you wish I can attach my outgoing messages from my Sent folder.)

The messages attached are the format I had received. You don't see the base64 portion in the latest one, probably because some agent filtered it.


Shyam, is it possible to compare the configuration for both lists on the news.mozilla.org server? Maybe there is a difference.


If you are sure that both lists at mozilla are configured identically, then we could conclude it's Google who does random behavior (sometimes strip away signature, sometimes drop) and I could stop sending signatures in the future.

(But it really seems wrong that messages are simply being dropped. Maybe Google concluded I'm sending spam.)

Comment 9

[Shyam Mani [:fox2mike]]

(In reply to comment #8)
> Shyam, is it possible to compare the configuration for both lists on the
> news.mozilla.org server? Maybe there is a difference.

So is the bug about news.mozilla.org or groups.google.com?

Comment 10

[Shyam Mani [:fox2mike]]

And you didn't answer this :

(In reply to comment #7)

> newsgroups setup is automated. Did the messages arrive on news.mozilla.org? If
> they did and didn't appear on Google Groups, it's something Google will have to
> look into and so far I've had zero success getting someone to look into them.
> 
> If the messages didn't hit news.mozilla.org, that's because of base64.
> news.mozilla.org has been setup and run by Giganews and they've told me that
> due to some technical reasons, base64 encoded stuff is dropped.

So to rephrase, did the messages appear on news.mozilla.org or not?

Comment 11

[Kai Engert [:KaiE:]]

(In reply to comment #9)
> (In reply to comment #8)
> > Shyam, is it possible to compare the configuration for both lists on the
> > news.mozilla.org server? Maybe there is a difference.
> 
> So is the bug about news.mozilla.org or groups.google.com?

I don't know.

I would like to kindly ask you to have a look at the configuration file on news.mozilla.org, and compare the configuration for lists

  dev-security-policy

and

  dev-tech-crypto

and see if there is a difference in attachment handling.

If both lists are identical, then google is the culprit and I'm OK to close this bug.

Comment 12

[Shyam Mani [:fox2mike]]

Opened a ticket with GigaNews.

Comment 13

[Kai Engert [:KaiE:]]

(In reply to comment #10)
> 
> So to rephrase, did the messages appear on news.mozilla.org or not?

Thanks for your idea to look.
To my surprise, I see more inconsistency.

Attachment message 1:
  Can't find it, apparently dropped.

Attachment message 2:
  Yes! I see it on news.mozilla.org

Comment 14

[Kai Engert [:KaiE:]]

(In reply to comment #11)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > Shyam, is it possible to compare the configuration for both lists on the
> > > news.mozilla.org server? Maybe there is a difference.
> > 
> > So is the bug about news.mozilla.org or groups.google.com?
> 
> I don't know.


I should say that I can't tell whether the bug is about news.mozilla.org or groups.google.com, because I don't know the path of the routing.

If the path is:
 mail -> news.mozilla.org -> groups.google.com
then the bug is about news.mozilla.org

If the path is:
       /- news.mozilla.org
  mail-
       \- groups.google.com
then
  the bug is about the mail-to-news gateway?

But given the inconsistency listed in comment 13, I don't know what to suspect.

Comment 15

[Eddy Nigg (StartCom)]

(In reply to comment #9)
> (In reply to comment #8)
> > Shyam, is it possible to compare the configuration for both lists on the
> > news.mozilla.org server? Maybe there is a difference.
> 
> So is the bug about news.mozilla.org or groups.google.com?

News - it didn't appear there.

Comment 16

[Shyam Mani [:fox2mike]]

GigaNews gave me this response :

Hello Shyam,

These two groups are configured the same way.  Your explanation was correct, the large base64 block triggered the binary filter.  None of the user's posts to mozilla.dev.tech.crypto have that block.

There is one post to mozilla.dev.security.policy that does have the block, and still reached the group (<[e-mail]>).  I'm not positive why that was not dropped -- it should have triggered the filter.

-- Bradley Giganews ISP Support

Comment 17

[Kai Engert [:KaiE:]]

Thanks for the clarification.

I'll refrain from sending signatures to the lists in the future.
Thanks

Comment 18

[Dave Miller [:justdave]]

FWIW, I filed a ticket with Giganews several months back about GPG/SMIME signatures getting treated as attachments, and was told they were going to make an exception for those because they agreed they ought to be let through.

Comment 19

[Dave Miller [:justdave]]

OK, I just went back looking for it, and that was more than a few months ago, it was in May of 2006. :)  And I actually got told at the time that there was already supposed to be an exception for GPG-signed messages.  S/MIME signatures were not mentioned in the thread though.

Comment 20

[Kai Engert [:KaiE:]]

(In reply to Dave Miller [:justdave] from comment #18)
> FWIW, I filed a ticket with Giganews several months back about GPG/SMIME
> signatures getting treated as attachments, and was told they were going to
> make an exception for those because they agreed they ought to be let through.

Dave, did you ever get a reply?
It seems, they still aren't allowed.

At least the following content types should be covered by the exception:

application/pgp-signature
application/x-pkcs7-signature
application/pkcs7-signature

Comment 21

[Kai Engert [:KaiE:]]

Changing the subject to better describe the issue.

Comment 22

[Kai Engert [:KaiE:]]

I'm reopening this bug. Other people are sending signed messages to the list, too, so we should have a real solution.

Nevertheless I've updated the list information page to recommend using other list archives.
See my post at
http://www.mail-archive.com/[e-mail]/msg11154.html

Comment 23

[Amy Rich [:arr] [:arich]]

deleted bogus whiteboard tag

Comment 24

[Greg Cox [:gcox]]

Mailman and the news gateway have been decom'ed, closing.