Last Comment Bug 329781 - news.mozilla.org not getting all messages
: news.mozilla.org not getting all messages
Status: RESOLVED DUPLICATE of bug 327300
:
Product: mozilla.org Graveyard
Classification: Graveyard
Component: Server Operations (show other bugs)
: other
: All Other
: P1 critical (vote)
: ---
Assigned To: Dave Miller [:justdave] (justdave@bugzilla.org)
: Justin Fitzhugh
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-03-08 07:55 PST by Deb Richardson [:dria] (plz NEEDINFO)
Modified: 2015-03-12 08:17 PDT (History)
11 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description Deb Richardson [:dria] (plz NEEDINFO) 2006-03-08 07:55:15 PST
Noticed this morning that a response to a thread didn't seem to make it to Google Groups for archiving, which is a bit annoying because I'm trying to reference that thread in a blog post.

There are three messages in the thread.  The first and third made it to Google Groups, mailing list, and the news groups just fine.

The second message only made it to the mailing list, and didn't make it to Google Groups or the newsgroup.

Link to incomplete Google groups thread:
http://groups.google.com/group/mozilla.dev.mdc/browse_frm/thread/5c27038fddc34dfe/608581ca2a2dfc60#608581ca2a2dfc60

Headers from the missing message:

From: "Justin Wood (Callek)" <callek@gmail.com>
Date: March 8, 2006 12:56:26 AM EST (CA)
To: "Deb Richardson" <deb@dria.org>
Cc: dev-mdc@lists.mozilla.org
Subject: Re: Topic/Category changes summary
Return-Path: <callek@gmail.com>
X-Original-To: deb@dria.org
Delivered-To: deb@mail.off.net
Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.207]) by  
mail.off.net (Postfix) with ESMTP id 5B78A77049F for <deb@dria.org>;  
Wed,  8 Mar 2006 00:56:28 -0500 (EST)
Received: by xproxy.gmail.com with SMTP id s10so73304wxc for  
<deb@dria.org>; Tue, 07 Mar 2006 21:56:26 -0800 (PST)
Received: by 10.70.39.7 with SMTP id m7mr412052wxm; Tue, 07 Mar 2006  
21:56:26 -0800 (PST)
Received: by 10.70.6.8 with HTTP; Tue, 7 Mar 2006 21:56:26 -0800 (PST)
Domainkey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;  
h=received:message-id:date:from:to:subject:cc:in-reply-to:mime- 
version:content-type:content-transfer-encoding:content- 
disposition:references; b=Cixpfx9yfPMhG8f6nX3/HVKLdJ/ 
fyKU9S2gt8WJFKG6Lw8v+nm/Y5FBaRODnZbPtmSxq9tbPYF1Ni7/ 
bVUtqLcwUgJgInM3hyTQpY+pnMC+rYg0falF9QjXhoudFyfqe2 
+hXGGYssdAlEUXVNR6VBDiJA6W9GnNtAqO8VWSWp/4=
Message-Id:  
<2e4780750603072156l27c8e7f6q7ae1f212097f92c3@mail.gmail.com>
In-Reply-To: <7B706523-51C2-4C8A-BF59-08ACEA01029F@dria.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: base64
Content-Disposition: inline
References: <7B706523-51C2-4C8A-BF59-08ACEA01029F@dria.org>
Comment 1 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-08 08:20:39 PST
Google Groups gets its feed from Giganews, so if it didn't make it to news.mozilla.org, it's not going to make it to Google.  Can you forward me the entire message out-of-band?
Comment 2 Deb Richardson [:dria] (plz NEEDINFO) 2006-03-08 11:40:50 PST
(In reply to comment #1)
> Google Groups gets its feed from Giganews, so if it didn't make it to
> news.mozilla.org, it's not going to make it to Google.  Can you forward me the
> entire message out-of-band?
> 

Forwarded.
Comment 3 Chris Ilias [:cilias] 2006-03-08 11:46:22 PST
I noticed the message was sent to Deb's address. Is it possible the message didn't make it to the mailing list?  Did anyone else on the list receive the message (via lists.mozilla.org)?
Comment 4 Deb Richardson [:dria] (plz NEEDINFO) 2006-03-08 12:05:55 PST
(In reply to comment #3)
> I noticed the message was sent to Deb's address. Is it possible the message
> didn't make it to the mailing list?  Did anyone else on the list receive the
> message (via lists.mozilla.org)?
> 

the CC is to the list: "Cc: dev-mdc@lists.mozilla.org"
Comment 5 Chris Ilias [:cilias] 2006-03-08 12:11:20 PST
(In reply to comment #4)
> the CC is to the list: "Cc: dev-mdc@lists.mozilla.org"

list.mozilla.org may have filtered it, thus never passing it on to Giganews, thus not making it to Google.

I don't don't why list.m.o would filter it (maybe it was a large message), but finding out if it made it to the list would help in finding the cause.
Comment 6 Mike Beltzner [:beltzner, not reading bugmail] 2006-03-08 12:52:32 PST
(In reply to comment #5)
> I don't don't why list.m.o would filter it (maybe it was a large message), but
> finding out if it made it to the list would help in finding the cause.

Yes, it made it to the list. I received this message with the following headers:

From:   callek@gmail.com
Subject: Re: Topic/Category changes summary
Date: March 8, 2006 12:56:26 AM EST (CA)
To:   deb@dria.org
Cc:   dev-mdc@lists.mozilla.org
X-Eon-Dm: dm30
Return-Path: <dev-mdc-bounces@lists.mozilla.org>
Received: from lists.mozilla.org (207.126.111.250 [207.126.111.250]) by dm04.mta.everyone.net (EON-INBOUND) with ESMTP id dm04.44040ac6.5148b63 for <beltzner@mozilla.com>; Tue, 7 Mar 2006 21:56:29 -0800 (PST)
Received: from notorious.mozilla.org (localhost.localdomain [127.0.0.1]) by lists.mozilla.org (Postfix) with ESMTP id EAF0F5800E; Tue,  7 Mar 2006 21:56:28 -0800 (PST)
Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.205]) by lists.mozilla.org (Postfix) with ESMTP id AFDE058004 for <dev-mdc@lists.mozilla.org>; Tue,  7 Mar 2006 21:56:26 -0800 (PST)
Received: by xproxy.gmail.com with SMTP id s12so69365wxc for <dev-mdc@lists.mozilla.org>; Tue, 07 Mar 2006 21:56:26 -0800 (PST)
Received: by 10.70.39.7 with SMTP id m7mr412052wxm; Tue, 07 Mar 2006 21:56:26 -0800 (PST)
Received: by 10.70.6.8 with HTTP; Tue, 7 Mar 2006 21:56:26 -0800 (PST)
X-Original-To: dev-mdc@lists.mozilla.org
Delivered-To: dev-mdc@lists.mozilla.org
Domainkey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Cixpfx9yfPMhG8f6nX3/HVKLdJ/fyKU9S2gt8WJFKG6Lw8v+nm/Y5FBaRODnZbPtmSxq9tbPYF1Ni7/bVUtqLcwUgJgInM3hyTQpY+pnMC+rYg0falF9QjXhoudFyfqe2+hXGGYssdAlEUXVNR6VBDiJA6W9GnNtAqO8VWSWp/4=
Message-Id: <2e4780750603072156l27c8e7f6q7ae1f212097f92c3@mail.gmail.com>
In-Reply-To: <7B706523-51C2-4C8A-BF59-08ACEA01029F@dria.org>
Mime-Version: 1.0
References: <7B706523-51C2-4C8A-BF59-08ACEA01029F@dria.org>
X-Beenthere: dev-mdc@lists.mozilla.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "Discussion list for all things related to the Mozilla Developer Center, including developer documentation." <dev-mdc.lists.mozilla.org>
List-Unsubscribe: <https://lists.mozilla.org/listinfo/dev-mdc>, <mailto:dev-mdc-request@lists.mozilla.org?subject=unsubscribe>
List-Post: <mailto:dev-mdc@lists.mozilla.org>
List-Help: <mailto:dev-mdc-request@lists.mozilla.org?subject=help>
List-Subscribe: <https://lists.mozilla.org/listinfo/dev-mdc>, <mailto:dev-mdc-request@lists.mozilla.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1766732003=="
Sender: dev-mdc-bounces@lists.mozilla.org
Errors-To: dev-mdc-bounces@lists.mozilla.org
Comment 7 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-13 14:58:12 PST
(In reply to comment #2)
> (In reply to comment #1)
> > Google Groups gets its feed from Giganews, so if it didn't make it to
> > news.mozilla.org, it's not going to make it to Google.  Can you forward me the
> > entire message out-of-band?
> > 
> 
> Forwarded.

I never got this, could you try again? (have you seen any more?)

I suspect Giganews' spamfilters probably nailed it.
Comment 8 Deb Richardson [:dria] (plz NEEDINFO) 2006-03-15 05:15:59 PST
Forwarded again.
Comment 9 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-20 14:42:05 PST
Have you seen any more that haven't shown up?  I don't see any information in the mail you forwarded that looks like something giganews would block...  hopefully it was a fluke...
Comment 10 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-24 00:11:02 PST
I'm going to close this out for lack of being able to reproduce.  If you run across any others, feel free to reopen it.
Comment 11 Mike Beltzner [:beltzner, not reading bugmail] 2006-03-24 12:18:55 PST
Yup, I got some more examples: I started a thread (via email) yesterday in mozilla.dev.planning. Later, I replied to a reply of that thread. The original message was delivered to Google Groups and by mailman, but did not show up in NNTP. The later reply was only delivered in mailman, and doesn't appear in NNTP or Google Groups:

ORIGINAL MESSAGE -- EMAIL HEADERS:

 X-Eon-Dm: dm10
Return-Path: <dev-planning-bounces@lists.mozilla.org>
Received: from lists.mozilla.org (207.126.111.250 [207.126.111.250])
	by dm10.mta.everyone.net (EON-INBOUND) with ESMTP
	id dm10.4419136a.1e159c3; Thu, 23 Mar 2006 15:17:09 -0800 (PST)
Received: from notorious.mozilla.org (localhost.localdomain [127.0.0.1])
	by lists.mozilla.org (Postfix) with ESMTP id 1F5B05801C;
	Thu, 23 Mar 2006 15:17:09 -0800 (PST)
X-Original-To: dev-planning@lists.mozilla.org
Delivered-To: dev-planning@lists.mozilla.org
Received: from smtp02.bis.na.blackberry.com (smtp02.bis.na.blackberry.com
	[216.9.248.49])
	by lists.mozilla.org (Postfix) with ESMTP id 59AAF1336D
	for <dev-planning@lists.mozilla.org>;
	Thu, 23 Mar 2006 15:17:07 -0800 (PST)
Message-ID: <1092075651-1143155826-cardhu_blackberry.rim.net-3248-@engine11-cell02>
Sensitivity: Normal
Importance: Normal
To: "bonecho" <bonecho@mozilla.org>,
	"dev.planning" <dev-planning@lists.mozilla.org>
From: "Mike Beltzner" <beltzner@mozilla.com>
Date: Thu, 23 Mar 2006 23:15:55 +0000 GMT
MIME-Version: 1.0
Cc: 
Subject: Fx2 Planning Centre (was PRD review)
X-BeenThere: dev-planning@lists.mozilla.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dev-planning.lists.mozilla.org
List-Unsubscribe: <https://lists.mozilla.org/listinfo/dev-planning>,
	<mailto:dev-planning-request@lists.mozilla.org?subject=unsubscribe>
List-Post: <mailto:dev-planning@lists.mozilla.org>
List-Help: <mailto:dev-planning-request@lists.mozilla.org?subject=help>
List-Subscribe: <https://lists.mozilla.org/listinfo/dev-planning>,
	<mailto:dev-planning-request@lists.mozilla.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1585748431=="
Sender: dev-planning-bounces@lists.mozilla.org
Errors-To: dev-planning-bounces@lists.mozilla.org

ORIGINAL MESSAGE - GOOGLE GROUPS URI: http://groups.google.com/group/mozilla.dev.planning/browse_frm/thread/363d9531af2dad10/69a04ae0b1032661#69a04ae0b1032661

ORIGINAL MESSAGE - NNTP: missing in action!

FOLLOW-ON MESSAGE - EMAIL HEADERS:

X-Eon-Dm: dm01
Return-Path: <dev-planning-bounces@lists.mozilla.org>
Received: from lists.mozilla.org (207.126.111.250 [207.126.111.250])
	by dm01.mta.everyone.net (EON-INBOUND) with ESMTP
	id dm01.441b0d44.30505a8; Thu, 23 Mar 2006 17:05:38 -0800 (PST)
Received: from notorious.mozilla.org (localhost.localdomain [127.0.0.1])
	by lists.mozilla.org (Postfix) with ESMTP id 1585F5808D;
	Thu, 23 Mar 2006 17:05:38 -0800 (PST)
X-Original-To: dev-planning@lists.mozilla.org
Delivered-To: dev-planning@lists.mozilla.org
Received: from smtp03.bis.na.blackberry.com (smtp03.bis.na.blackberry.com
	[216.9.248.50])
	by lists.mozilla.org (Postfix) with ESMTP id 2DFFF13E85
	for <dev-planning@lists.mozilla.org>;
	Thu, 23 Mar 2006 17:05:35 -0800 (PST)
Message-ID: <195139708-1143162334-cardhu_blackberry.rim.net-10754-@engine27-cell02>
References: <1092075651-1143155826-cardhu_blackberry.rim.net-3248-@engine11-cell02>
	<f4e041d00603231639t4d7a420esca70b5fbb8fe1bf8@mail.google.com>
In-Reply-To: <f4e041d00603231639t4d7a420esca70b5fbb8fe1bf8@mail.google.com>
Sensitivity: Normal
Importance: Normal
From: "Mike Beltzner" <beltzner@mozilla.com>
Date: Fri, 24 Mar 2006 01:05:28 +0000 GMT
MIME-Version: 1.0
Cc: "dev.planning" <dev-planning@lists.mozilla.org>,
	bonecho <bonecho@mozilla.org>
Subject: Re: Fx2 Planning Centre (was PRD review)
X-BeenThere: dev-planning@lists.mozilla.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: dev-planning.lists.mozilla.org
List-Unsubscribe: <https://lists.mozilla.org/listinfo/dev-planning>,
	<mailto:dev-planning-request@lists.mozilla.org?subject=unsubscribe>
List-Post: <mailto:dev-planning@lists.mozilla.org>
List-Help: <mailto:dev-planning-request@lists.mozilla.org?subject=help>
List-Subscribe: <https://lists.mozilla.org/listinfo/dev-planning>,
	<mailto:dev-planning-request@lists.mozilla.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============0353668301=="
Sender: dev-planning-bounces@lists.mozilla.org
Errors-To: dev-planning-bounces@lists.mozilla.org
To: undisclosed-recipients: ;

FOLLOW-ON MESSAGE - GOOGLE GROUPS URI: missing in action!

FOLLOW-ON MESSAGE - NNTP: missing in action!

some notes: the follow on message was "To: undisclosed recipients;" (which is another wierd thing that happens a lot) and CC'd to dev-planning@lists.mozilla.org ... could that be part of the (but obviously not the entire) problem?
Comment 12 Mike Beltzner [:beltzner, not reading bugmail] 2006-03-24 12:22:41 PST
cc'ing a bunch of people; justin to make sure he's getting the email on this, dbaron, ben, cbeard and mitchell because it's very much related to ongoing decisions about moving discussion to forums that use this infrastructure. Obviously we need to ensure that if we SAY that people can get to these discussions through email, NNTP or Google Groups, then we need to make sure that they are getting those discussions reliably.
Comment 13 Robert Strong [:rstrong] (use needinfo to contact me) 2006-03-24 12:55:11 PST
This may or not be related - I just received several messages on the mailing list from yesterday... may be related or it may just be latency
Comment 14 David Baron :dbaron: ⌚️UTC-10 2006-03-24 12:57:24 PST
That can also be because of someone going through the mailman moderation queue -- I just cleared the moderation queue on dev-planning, if that's the list you're talking about.
Comment 15 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-24 13:28:51 PST
(In reply to comment #12)
> cc'ing a bunch of people; justin to make sure he's getting the email on this

he's already the qa contact ;)

Beltzner's additional info was enough to prove there's something real going on here, I've opened a ticket with Giganews.
Comment 16 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-24 15:37:38 PST
response from Giganews:

---8<---
That article was filtered from the text group as a binary article, since the article was base64 encoded.  We'll still feed that stuff on to Google (we feed everything to Google and let them filter what they want), but that will be seen as a binary and filtered.

If you could, recommend to people that they not base64-encode their text articles.  Post them as unencoded text when posting to text groups.  We don't check the actual content of the binary to see if it's really text or not (at the volume we feed our servers, that'd be crushing), and we don't trust content-type headers, since those can easily be forged.  Basically, if it looks like binary, regardless of how "binary" it truly is, it will be dropped in a text group.
---8<---

That makes this a duplicate of bug 327300, but I'll leave this open for a day or so for additional comment first.
Comment 17 Mike Shaver (:shaver -- probably not reading bugmail closely) 2006-03-24 16:26:19 PST
There's no attachment here, so it's not a dup of that bug, and some mail clients (like that on the blackberry) don't give a choice about how the body of the message is encoded.  (I suspect it uses base64 so that it doesn't have to worry about gateways that don't like high-bit characters, since it's a perfectly legal and reasonable way to send mail supported by pretty much every mail client the recipient might have.)

Can we get giganews to turn off this "protection" for us on our groups, or to honour MIME types, or to send it to the moderator address (thus permitting posts from people who are subbed to the list, at least, though that is only marginally less ridiculously painful) or _something_?
Comment 18 Mike Shaver (:shaver -- probably not reading bugmail closely) 2006-03-24 16:28:30 PST
Or, heck, _we_ could decode base64 for messages that have text MIME types before we send it to the news gateway, which wouldn't require us to get Giganews to do anything at all.

(That moderator-address thing won't work, I now realize; I apologize for the dumbness there.)
Comment 19 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-24 17:54:51 PST
(In reply to comment #18)
> Or, heck, _we_ could decode base64 for messages that have text MIME types
> before we send it to the news gateway, which wouldn't require us to get
> Giganews to do anything at all.

If you actually read the other bug, that's part of it ;)

Comment 20 Mike Shaver (:shaver -- probably not reading bugmail closely) 2006-03-24 18:01:09 PST
I did read the other bug, and it sounds like it should be WONTFIXED, or summarizes another problem entirely.
Comment 21 Dave Miller [:justdave] (justdave@bugzilla.org) 2006-03-27 13:51:50 PST
OK, I asked Giganews, and they will not budge on this.  Bug 327300 is the only solution as long as we're still using them.

*** This bug has been marked as a duplicate of 327300 ***

Note You need to log in before you can comment on or make changes to this bug.