Last Comment Bug 716007 - Newsgroups are not being propagated to Google Groups
: Newsgroups are not being propagated to Google Groups
Status: RESOLVED FIXED
[waiting for Google Groups]
:
Product: mozilla.org
Classification: Other
Component: Discussion Forums (show other bugs)
: other
: All All
: P1 normal with 1 vote (vote)
: ---
Assigned To: Michael Burns [:mburns]
: Dave Miller [:justdave] (justdave@bugzilla.org)
:
Mentors:
Depends on:
Blocks: 701127 708511 711473 712978 717696 728054 735584 735787 737380 738477 739655 740674 745772 747101 761273 761883 764361 765741 765769 769219 770075 772872 789418 791630 791666 802891 804370 817014 824945
  Show dependency treegraph
 
Reported: 2012-01-06 12:20 PST by Ian Bicking (:ianb)
Modified: 2013-02-28 18:57 PST (History)
31 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
dev-webapps message that didn't propagate (4.24 KB, message/rfc822)
2012-07-25 10:18 PDT, Myk Melez [:myk] [@mykmelez]
no flags Details

Description Ian Bicking (:ianb) 2012-01-06 12:20:31 PST
Recent posts to mozilla.dev.platform, mozilla.dev.webapps, and mozilla.dev.b2g are not being propagated to their respective Google Group mirrors (maybe all groups, those are just the ones we checked, all of which have had missing posts).  The confirmed missing post is by fabrice@mozilla.com though I believe there are other missing posts.
Comment 1 Gervase Markham [:gerv] 2012-01-07 03:20:33 PST
Someone will need to contact Google.

Gerv
Comment 2 Michael Burns [:mburns] 2012-01-17 20:33:38 PST
Sent a second email to Google Groups admins noting sync issues. Will wait a couple business days before further escalation up the ladder.
Comment 3 Ian Bicking (:ianb) 2012-01-17 20:40:09 PST
I'm also subscribed here, and haven't been gotten any messages: https://lists.mozilla.org/listinfo/dev-webapps - unfortunately I don't know if I should have gotten any messages (well, I at least have written to that email address, but I'm not sure I get my own messages, and there's no archive except the Google Group).
Comment 4 Ian Bicking (:ianb) 2012-01-18 11:22:46 PST
After checking via a newsreader, mail sent to the dev-webapps list goes to the group, but traffic on the webapps group does not come back to me on the dev-webapps list.
Comment 5 Michael Burns [:mburns] 2012-01-23 18:21:35 PST
mozilla.dev.marketplace has also reported this problem.
Comment 6 Irvin (MozTW) 2012-01-29 20:33:33 PST
mozilla.dev.general.zh has faced a similar problem that google group page stayed empty weather we posted via the mailing-list, news-server or directly post on google group page, while the mailing-list and news-server works normally.
Comment 7 Ragavan S [:rags] 2012-02-09 01:14:18 PST
Has this been escalated, per comment #2 above? It is a bit ridiculous how long this is taking. Is this what every new group we create has to go through if we want it gatewayed to google groups?
Comment 8 Irvin (MozTW) 2012-02-09 01:21:24 PST
(In reply to Ragavan S [:rags] from comment #7)
+1, we've been waiting for 4+ month, it's ridiculous that Google always not reply.
Comment 9 Gervase Markham [:gerv] 2012-02-09 03:44:40 PST
mburns: ping?

Gerv
Comment 10 Ragavan S [:rags] 2012-02-17 23:38:44 PST
mburns: any update?
Comment 11 Ian Bicking (:ianb) 2012-02-18 13:06:10 PST
The real problem is that we don't have any public archives of the group, the only way to read old messages is to connect with a newsreader.  Can we setup pipermail or some other simple mail archiver so at least we can have something visible?
Comment 12 David Ascher (:davida) 2012-02-18 17:03:40 PST
Can't we do a news-mailman gateway if the google groups gateway's broken?

Upping prio, as this is going to become problematic for the apps group with the upcoming marketplace launch.
Comment 13 Ian Bicking (:ianb) 2012-02-18 17:22:38 PST
We do have a working mail gateway (I don't have the link on me) but it is missing archives (it just points to google groups)
Comment 14 David Ascher (:davida) 2012-02-18 18:04:12 PST
Ah, right.  The link is https://lists.mozilla.org/listinfo/dev-webapps.
Comment 15 Irvin (MozTW) 2012-02-19 05:10:59 PST
(In reply to Ian Bicking (:ianb) from comment #11)
> The real problem is that we don't have any public archives of the group, the
> only way to read old messages is to connect with a newsreader.  Can we setup
> pipermail or some other simple mail archiver so at least we can have
> something visible?

Agree, we just need an working archive of the discussion on mailing list, if the Google solution is so lacking of maintaining (for monthes!), find another way is better solution.
Comment 16 Ragavan S [:rags] 2012-02-22 00:42:03 PST
Can someone from IT/serverops reply here? The last update I see is from Jan 23rd. Can you at least help with a news-mailman gateway like David suggests in comment #12? I don't care it's on google groups, just get me an archive.

Bumping the priority to P1 to see if we can at least get a response from someone.
Comment 17 Michael Burns [:mburns] 2012-02-22 01:00:03 PST
We've sent in a number of emails (ever couple weeks), and rbryce called at least once, as I understand it. The source of delay here was that we (I) was too optimistic toward Google's eventual response.

AFAIK, the next escalation step is having Mitchell call up a google contact and have them get the ball rolling from that end.

CC'ing Phong: what needs to happen to get action from Google Groups on this?
Comment 18 Michael Burns [:mburns] 2012-03-09 14:31:48 PST
I google groups admin responded to my latest email poke:

"We're aware of the problem, and a solution should be in place by the end of the month - we had to rewrite some ancient nntp code to make it compatible with current google infrastructure.
-Andrew Shebanow"
Comment 19 Michael Burns [:mburns] 2012-03-14 01:12:01 PDT
YOU get a newsgroup that syncs to Google Groups and YOU get a newsgroup that syncs to Google Groups. EVERYBODY GETS NEWSGROUPS THAT SYNCS TO GOOGLE GROUPSSSS! http://bit.ly/gcSoTx



https://groups.google.com/group/mozilla.dev.platform/topics

https://groups.google.com/group/mozilla.dev.webapps/topics

https://groups.google.com/group/mozilla.dev.b2g/topics

https://groups.google.com/group/mozilla.dev.marketplace/topics

https://groups.google.com/group/mozilla.dev.general.zh/topics

et cetera
Comment 20 Myk Melez [:myk] [@mykmelez] 2012-03-15 10:10:04 PDT
It looks like some of us get a newsgroup that syncs to Google Groups only messages from March 9 onwards (at least webapps, but I suspect also marketplace and general.zh), while others get one that syncs all messages (platform, b2g).  Might all of us get the latter type of newsgroup?

Webapps in particular has messages dating back at least to January that would be super-awesome to have available for posterity (and because people keep asking us what we said back then).
Comment 21 Dan Mosedale (:dmose) 2012-03-19 14:14:56 PDT
mozilla.community.games appears to not exist on Google Groups at all.  

Perhaps there would be some advantages to driving our use of Google groups via the mailing list instantiations of our fora rather than the newsgroup versions.  Some that come to mind include:

* it would be possible to easily set these up by hand individually rather the depending on third-parties
* it shouldn't, in theory, be too hard to automate that setup ourselves
* given the relative prominence of mailing lists versus newsgroups in the world at large (ie not just among techies), I would bet that Google gives implementation, testing, and maintenance priority to the parts of the Groups code that touches email over the parts that touch NNTP.
Comment 22 Myk Melez [:myk] [@mykmelez] 2012-03-19 16:08:18 PDT
(In reply to Dan Mosedale (:dmose) from comment #21)

> Perhaps there would be some advantages to driving our use of Google groups
> via the mailing list instantiations of our fora rather than the newsgroup
> versions.

Some Labs projects have set up Google Groups, which gives them mailing lists and web interfaces but not newsgroups.  It works great.  We should do more of that.

Nevertheless, we should also make sure the existing mailing list/newsgroup groups are properly archived.
Comment 23 Robert Kaiser 2012-03-20 05:02:13 PDT
(In reply to Myk Melez [:myk] [@mykmelez] from comment #22)
> Some Labs projects have set up Google Groups, which gives them mailing lists
> and web interfaces but not newsgroups.  It works great.  We should do more
> of that.

Actually, it closes out a part of the community, people like me who hate signing up for a ton of lists and get their accounts spammed with stuff they only want to glance over, which NTTP allows beautifully. It also closes out anyone who doesn't want to possess or use Google accounts, which after the recent privacy changes at Google is a growing audience.
Comment 24 Dan Mosedale (:dmose) 2012-03-20 09:54:14 PDT
I framed my suggestion the way I did specifically because I don't think it's productive to get into a "usefulness of NNTP" discussion here; that's very much off-topic for this bug.  

I've specifically suggested changing the technical detail of which channel gets used to populate the Google groups.  I'd be interested to hear from someone in IT about the technical merits of that strategy w.r.t. solving the particular problem we're having here.
Comment 25 Michael Burns [:mburns] 2012-03-20 21:37:31 PDT
You are quite right.

I've sent in an email to the Google Groups admin. Will let you know if they do/don't respond.
Comment 26 Dan Mosedale (:dmose) 2012-03-21 10:17:15 PDT
Great; thanks!
Comment 27 Michael Burns [:mburns] 2012-03-28 22:09:47 PDT
No response as of yet. I just sent in another request for help.
Comment 28 Dan Mosedale (:dmose) 2012-03-29 09:57:54 PDT
Thanks muchly for staying on top of the Google folks, getting this fixed will be helpful to a bunch of folks.
Comment 29 Dan Mosedale (:dmose) 2012-06-05 09:32:22 PDT
We've just had someone from the community ask why the games archive link is broken, what's the next step in escalating this?  We really need a more functional discussion system than we've currently got.
Comment 30 Martin Best (:mbest) 2012-06-05 09:37:08 PDT
Agreed, this has now been down for almost 2 quarters.  I think it's time to start considering a wider range of solutions.
Comment 31 Ian Bicking (:ianb) 2012-06-05 09:38:28 PDT
We also cannot have unarchived mail, it makes it very hard to bring people into conversations, and decisions are essentially unaccountable when there's no artifact of the discussion.  If Google Groups isn't entirely reliable we need a backup, such as pipermail.  There's no reason we can't be archiving to both at the same time.
Comment 32 Myk Melez [:myk] [@mykmelez] 2012-06-05 10:02:25 PDT
Google Groups proper is actually entirely reliable, it's the Mozilla-specific newsgroup/mailing list/Google Group triad that isn't reliable.  And Google seems little interested in continuing to maintain it, much less enhance it, based on the lack of response to the issues raised in this bug.

Mozilla projects that use just Google Groups, like Jetpack <https://groups.google.com/forum/#!forum/mozilla-labs-jetpack>, have none of these problems, and we should use Google Groups by itself for new projects (and switch to it for existing ones having problems with the triad).
Comment 33 [:fabrice] Fabrice Desré 2012-06-05 10:16:27 PDT
Do people need a google account to use google groups?
Comment 34 Dan Mosedale (:dmose) 2012-06-05 11:41:50 PDT
I believe for reading google groups, one does not need an account.  To subscribe or post one probably does.
Comment 35 Robert Kaiser 2012-06-05 13:02:12 PDT
There's actually two different things that are both called "Google Groups" and hosted on the same site: A Google-hosted mailing list service and a web interface to newsgroups. The former is working pretty well and AFAIK even needs an account to read messages, of course needs one to subscribe and post. The latter, which we are using here, has always been sucky, but has been degrading even more over time, and has free access to reading messages - needs an account to post as well, though.
Unfortunately, nobody seems to ever have built a really good web interface to newsgroups (though I hear the gmane interface is pretty usable for their own newsgroups that are mirroring mailing lists), which is probably one reason why their usage is on a steady decline. Standalone clients for messaging are the only really good access to NNTP/newgroups and standalone client usage for messaging is declining - even if the Thunderbird team doesn't want to hear that, understandably.
Comment 36 Lawrence Mandel [:lmandel] (use needinfo) 2012-07-11 10:43:47 PDT
Kev, I see that you're already cced. Do you have any Google contacts who can help escalate this growing issue?
Comment 37 Brandon Long 2012-07-19 13:00:33 PDT
I just happened to stumble upon this bug after reading a Google Alert pointing at a blog post about it.

I run part of the service at Google that this depends on, and although we've gotten your emails in the past several months, and I made sure your groups were added, apparently there was a second place they needed to get added to work.

I fixed that yesterday, and I see that the mozilla.community.games group has now been created.

The usenet portion of Groups has been kept together with spit and bailing wire, and we ran out of spit about 9 months ago.  We're still in the middle of moving the whole system over to a new system... but there's still something quite hacky about the way that new groups are transferred as control messages over usenet, and it may take us some time to get everything right in the new system.  We hope things will be stable on the new system in the next 4-6 weeks.

You might have more luck with premium Google Apps for Your Domain, which would give you access to siloed mailing lists with an API for managing them (afaik), and Groups also has an "external archive" mode that may work as well.  We're happy to try and keep the current system working, though.

We've added monitoring for our at least the portion of the system that was broken that I just fixed... but the whole deal still depends on a lot of steps, so please continue to notify us if things aren't working.  We'll try and do a better job of making sure the exact group gets created and messages are flowing.
Comment 38 Lawrence Mandel [:lmandel] (use needinfo) 2012-07-23 09:18:03 PDT
Thanks for jumping in here Brandon. I see that at least some groups still haven't been created. Can you follow up on the rest of the specific requests in this bug? From the dependencies, I think the complete list is:

mozilla.community.antarctica (not sure of the status of this list on our end)
mozilla.community.morocco
mozilla.community.vietnam
mozilla.compatibility
mozilla.dev.chat
mozilla.dev.sumo
mozilla.support.instantbird
mozilla.webapps
mozilla.webapps.pt-br
Comment 39 Josh Matthews [:jdm] 2012-07-23 10:14:16 PDT
mozilla.education has been revived, but only appears as an archived newsgroup on Google Groups.
Comment 40 Myk Melez [:myk] [@mykmelez] 2012-07-24 13:56:40 PDT
(In reply to Brandon Long from comment #37)
> I run part of the service at Google that this depends on, and although we've
> gotten your emails in the past several months, and I made sure your groups
> were added, apparently there was a second place they needed to get added to
> work.
> 
> I fixed that yesterday, and I see that the mozilla.community.games group has
> now been created.

In addition to the problem of groups not being created, at least dev-webapps has two other issues:

1. messages from before the group was created never made it to the group;
2. some messages from after the group was created don't make it to the group.

An example of the latter problem is the One-app-per-origin thread <https://groups.google.com/forum/#!topic/mozilla.dev.webapps/lILc00-owaQ>, which contains 30 messages, only 18 ever made it to the group.

That's a big problem, because such conversations lead to decisions that entrain implementations (in this case bug 775847), and we need to be able to refer back to them to know what we're doing and how to do it.


> You might have more luck with premium Google Apps for Your Domain, which
> would give you access to siloed mailing lists with an API for managing them
> (afaik), and Groups also has an "external archive" mode that may work as
> well.  We're happy to try and keep the current system working, though.

We did something like that for the Mozilla Labs groups, and it worked great, as I noted in comment 22.  I think we should do it for new groups, even though some Mozillians don't like that approach (cf. comment 23).

However, we should still make sure our existing groups work properly.


> We've added monitoring for our at least the portion of the system that was
> broken that I just fixed... but the whole deal still depends on a lot of
> steps, so please continue to notify us if things aren't working.  We'll try
> and do a better job of making sure the exact group gets created and messages
> are flowing.

Thanks Brandon, I really appreciate it!  In addition to making sure the new groups get created, can you look into the issues of messages not making it to them?
Comment 41 Brandon Long 2012-07-25 10:01:32 PDT
With usenet, messages sent before the newsgroup is created are rejected, so we don't have them.

The newsgroup creation itself is handled asynchronously, ie in a batch job I think once a day, so any messages sent immediately after the control message are also going to be rejected.

That thread is from April and I have no way of looking into the missing messages from that thread, any logs are long gone.

Do you have a more recent example of messages that didn't propagate?
Comment 42 Myk Melez [:myk] [@mykmelez] 2012-07-25 10:18:56 PDT
Created attachment 645805 [details]
dev-webapps message that didn't propagate

(In reply to Brandon Long from comment #41)
> Do you have a more recent example of messages that didn't propagate?

Here's a recent message that didn't propagate.
Comment 43 Mohamed Amarochan 2012-08-01 15:04:27 PDT
Still waiting for the group creation!
mozilla.community.morocco
Comment 44 Brandon Long 2012-09-06 13:22:03 PDT
If someone can get me a list of newsgroups that need to be added/deleted and their moderation status, I'm happy to update our list.

Also, the new system is mostly in place at the moment, so I can better look for propagation errors, at least for anything in the last 8 days (the length of our logs).
Comment 45 Lawrence Mandel [:lmandel] (use needinfo) 2012-09-06 13:25:02 PDT
Thanks for helping out Brandon. I listed 9 groups in comment 38 that still need to be created. Can you clarify what you need for moderation status?
Comment 46 Michael Burns [:mburns] 2012-09-07 04:57:08 PDT
Brandon et al.,

I've sent you a list of all the newsgroups I have with open bugs (those listed in comment 38, plus a couple from other bugs I own), including their moderation status. Hopefully this is an exhaustive list (15, for those playing along at home). Thank you!
Comment 47 Brandon Long 2012-09-14 14:07:44 PDT
I received the list of groups and updated them in our datastore and verified they were all active/available from the Groups web interface.  Things should be working now.
Comment 48 Michael Burns [:mburns] 2012-09-17 13:37:55 PDT
mozilla.community.morocco
mozilla.community.vietnam
mozilla.compatibility

all have had messages show up correctly and appear to be working fine. A few  more newsgroups have been created on Google Groups, but I don't see messages flowing yet. Still investigating.
Comment 49 Michael Burns [:mburns] 2012-11-08 04:11:33 PST
m.c.morocco, m.c.vietnam and m.compatibility have all been working nicely.

We only have a couple outstanding lists (.usa and .switzerland, .comms and .education) that have issues. Waiting to hear back from Google on why those haven't been picked up in their Monday scans.
Comment 50 Michael Burns [:mburns] 2012-11-20 16:42:23 PST
Brandon Long from Google has been in contact with me, and should be adding all outstanding lists to date. Yay!
Comment 51 fredyrouge 2012-12-14 21:17:05 PST
Hi

The mailing list now works, thanks!

I put the language in French and the list info page lost the mozilla look, is not the most important but, can you help me?

Finaly, can any add the mailing list to www.mozilla.org/about/forums/

Best regards
Comment 52 Michael Burns [:mburns] 2012-12-18 21:31:08 PST
I believe the problem you saw with setting French as the language is bug 815466, which I hope to fix. I'm not sure where mozilla.org/about/forums/ gets populated from. It might be by way of Google Groups, in fact. Let me ask around.
Comment 53 Gervase Markham [:gerv] 2012-12-19 01:25:03 PST
http://www.mozilla.org/about/forums is populated by a script run over a list of newsgroups which I maintain; I have help from other scripts to maintain it, and those scripts consult the NNTP server to see what groups are valid.

I still need to update it manually, though. I'll add it to my ToDo list.

Gerv
Comment 54 fredyrouge 2012-12-20 06:35:06 PST
Hi, Tanks to all for your help, now google groups work and the template is in the bug 815466, I think that the only issue for close this tiket is the inclusion of Québec community in /about/forums :)
Comment 55 Gervase Markham [:gerv] 2012-12-20 07:21:50 PST
Check about/forums :-)

Gerv
Comment 56 fredyrouge 2013-01-02 19:00:02 PST
Thanks and happy new year!!

Note You need to log in before you can comment on or make changes to this bug.