582454 - Adopt Weekly Updates as a Mozilla-owned web tool

Status: NEW

(Websites :: Other, defect)

Description

[Benjamin Smedberg]

Overview: Weekly Updates is a dynamic website I've been running at http://benjamin.smedbergs.us/weekly-updates.fcgi/ which is a low-touch way for members of the Mozilla community to post and consume status updates. It runs and is useful to many people, and I'd like to turn it over to Mozilla webdev/IT to run on Mozilla infrastructure.

Project Scope: * At a minimum, I'd like it to run using the current code, and migrate the existing data.
* There are also a set of patches which I've been unable to test because I don't have a staging environment/twitter account/etc which might be nice to test and integrate.

Dependencies: 
	Security: The only private data in the database is the user password. This is currently stored in cleartext in the DB: I expect there might be a better way to do this, but I haven't looked.
	App: The code is currently at http://hg.mozilla.org/users/bsmedberg_mozilla.com/weekly-updates/
It is written in Python/cherrypy/genshi/python-markdown2 with a MySQL database backend (sqlite used to work for testing, but that may be broken now). It can be deployed into any WSGI-capable environment.

Assumptions: morgamic told me to file this at the Summit, so I'm assuming webdev is still willing to take on maintenance of this project.

Deliverables: Phase I: deploy http://benjamin.smedbergs.us/weekly-updates.fcgi/ exactly as it is, migrating the data to Mozilla infrastructure.
Phase Next: test and accept patches from dietrich to forward status updates to a twitter account. See if there are any other patches pending.

Comment 1

[Mike Alexis [:malexis]]

Thanks Benjamin. I'll regroup with Morgamic and see about assigning over to a dev and what kind of timeline we're looking at. 

BTW, your the first to use the project initiation form!

Comment 2

[Les Orchard [:lorchard]]

I can start taking a look at this. Some questions off the top of my head:

* What's the urgency? End of year, whenever; end of quarter; yesterday, because the server's dying?

* Is webdev (ie. me) taking this over entirely, or do you (Benjamin) still want to play?

* Should I assume we want a full hosting loadout - eg. staging server with regular updates, prod cluster with multiple webheads / dbs?

* Can I get a dump of the existing DB to play with? Feel free to garble passwords, if need be, though we'll need those eventually to migrate the thing.

* How does this differ from what we're doing with Rypple? Is this an alternate system that people are just more comfortable with?

* Is this something we might eventually want to hook up to Secret Squirrel for SSO?

http://github.com/mozilla/secret-squirrel
https://intranet.mozilla.org/Webdev:SSO

* Could this fit somehow into dria's extranet proposal? (eg. open up to more community members)

https://intranet.mozilla.org/forum/viewtopic.php?f=4&t=183&start=0&sid=61351422e890c450b155a926c80d77bd

Comment 3

[Benjamin Smedberg]

No particular urgency: it's running just fine right now. The problem for me is that people keep sending me patches, and I don't really have or want to create a staging environment to test them.

I'm happy to answer questions, but I have plans to add any new features. I know of several people from the community who would like to add some features.

Sure... I'll have to figure out how to do that.

What are we doing with rypple? I never knew rypple had any sort of status-updating feature, just feedback, and I didn't think it produced public feeds or anything useful like that.

I have no clue about secret-squirrel.

The site is already open to anyone who knows the spamfilter password, which is pretty much anyone in the community. I'd be happy for it to be worked into the extranet, although it sounded like that was still designed for community-private communication, and this is fully-public.

Comment 4

[Les Orchard [:lorchard]]

(In reply to comment #3)
> No particular urgency: it's running just fine right now. The problem for me is
> that people keep sending me patches, and I don't really have or want to create
> a staging environment to test them.
> 
> I'm happy to answer questions, but I have plans to add any new features. I know
> of several people from the community who would like to add some features.

Fair enough - so then we'll just kind of adopt the status app as a webdev project.

> Sure... I'll have to figure out how to do that.

Cool. I see there's a dbschema.sql in the repo - as long as that's current, I can start playing with the app. A SQL dump will be most needed at the point where we're ready to switch over to a new install.

> What are we doing with rypple? I never knew rypple had any sort of
> status-updating feature, just feedback, and I didn't think it produced public
> feeds or anything useful like that.

Rypple's also got goals / tasks / status updates, etc. I'm not sure how public it is / can be, though.

> I have no clue about secret-squirrel.

That's okay - that was more a general webdev question to the Mikes on the bug CC list.

> The site is already open to anyone who knows the spamfilter password, which is
> pretty much anyone in the community. I'd be happy for it to be worked into the
> extranet, although it sounded like that was still designed for
> community-private communication, and this is fully-public.

Hmm, I don't have the spamfilter password myself, so I'm not in the set of pretty much everyone. :)  But, good point about fully-public vs community-private.

Comment 5

[Les Orchard [:lorchard]]

I think the app repo is also missing a sample config file. Any chance you could check one in with some example settings (eg. bogus DB passwords) and a -dist filename (eg. config.py-dist)? I'm not sure what all needs to go in there

Comment 6

[Benjamin Smedberg]

Done http://hg.mozilla.org/users/bsmedberg_mozilla.com/weekly-updates/rev/d4fe0f2a63ad

Comment 7

[Les Orchard [:lorchard]]

(In reply to comment #6)
> Done
> http://hg.mozilla.org/users/bsmedberg_mozilla.com/weekly-updates/rev/d4fe0f2a63ad

Cool beans, thanks! Between the config and the DB dump I just got, I've got the app running on my laptop now. That should get me going

Comment 8

[Les Orchard [:lorchard]]

A small status update: I've started working with ianb's Silver Lining to deploy this app to prod and staging servers on Rackspace VMs. Since this is a pretty small app, that seemed like a good trial case for the tool.

I've opened bug 586815 to get DNS entries pointed at the prod and staging VMs. Beyond that, this might be pretty close to usable after one more import of the database just before we want to switch over.

Comment 9

[Les Orchard [:lorchard]]

(In reply to comment #8)
> A small status update: I've started working with ianb's Silver Lining to deploy
> this app to prod and staging servers on Rackspace VMs. Since this is a pretty
> small app, that seemed like a good trial case for the tool.
> 
> I've opened bug 586815 to get DNS entries pointed at the prod and staging VMs.
> Beyond that, this might be pretty close to usable after one more import of the
> database just before we want to switch over.

Oh, and I also still need to make some tweaks to the app itself to switch over hardcoded URLs to the original app, but I want to make sure I've got the DNS entries straight before I do that.

Comment 10

[Les Orchard [:lorchard]]

Further status update: Looks like this might not be just as simple as getting a couple of DNS entries to point at some Rackspace VMs. Asking for the DNS entries seems to have tickled some concerns with regard to managing non-IT VM's, security, etc. 

Stay tuned...

Comment 11

[Les Orchard [:lorchard]]

It's been awhile, but I have progress to report!  Looks like the DNS entries have propagated, and now we have a prod and staging server:

http://weeklyupdates.mozillacloud.org/
http://stage-weeklyupdates.mozillacloud.org/

Need to swap my head back into this and see where I was at.

Comment 12

[David Boswell]

Component www.mozilla.org -> Other since this isn't a www.mozilla.org issue.

Comment 13

[Benjamin Smedberg]

ping? When can I give you a final data dump and actually get this completely switched over to the moz server?

Comment 14

[Les Orchard [:lorchard]]

(In reply to comment #13)
> ping? When can I give you a final data dump and actually get this completely
> switched over to the moz server?

Since we've reorg'd in webdev a bit and I'm working mainly on MDN now, I think this is due for a reboot. I'm not sure who would be the first contact for a project like this, now, though. Is this a webtool? Is this for the rapid dev team?

FWIW:

* I haven't touched this since last August - it fell off my radar as Fx4 projects piled on;

* I don't think the mozillacloud.org VMs are running any more;

* my laptop had a hard drive failure last year, so I no longer have the API keys from ianb to spin up the Rackspace VMs using Silver Lining;

* I suspect the new way of things is to see if webdev can revamp this tool as a Django / Playdoh app. That, or I think there's been some talk about community incubation hosting? Not sure.

* if Silver Lining is still a thing we want to play with, my hg repos are here:

http://hg.mozilla.org/users/lorchard_mozilla.com/weekly-updates/
http://hg.mozilla.org/users/lorchard_mozilla.com/weekly-updates-lib/

Comment 15

[Benjamin Smedberg]

If you're going to have to rewrite the app, don't bother... I just want the existing app deployed with slightly more maintenance/uptime/security reliability than self-hosting it with dreamhost. When I first started it as cherrypy+genshi I was told that this was a deployable solution (it's what I use for almost all my self-written webapps).

Comment 16

[Laura Thomson :laura]

Will work on getting this deployed in early Q3 via Webtools team.

Comment 17

[:Atoll]

(In reply to Laura Thomson :laura from comment #16)
> Will work on getting this deployed in early Q3 via Webtools team.

Hi, what's the status of this deployment? I'm seeing false-positive reports from Services-Eng users that would best be resolved by the 2011Q3 goal described above.

Comment 18

[Laura Thomson :laura]

It fell off the end of the prio list.  I don't think we'll get to it in the short term, just too many other things that are more critical.