380177 - Set up developer-stage to sync to MediaWiki's tip

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Eric Shepherd [:sheppy]]

In order to keep better abreast of what's going on with MediaWiki development, and to not be caught by surprise by any changes, I'd like to have an MDC clone set up that syncs to the MediaWiki trunk nightly.

That way we can be prepared for updates better, since from now on we're going to be trying to update whenever a new release comes out instead of a year or two later like we've been doing.

This probably shouldn't be on developer-stage, since we'll be using that for other development work that will likely not be on the same schedule as MediaWiki upgrades.

Comment 1

[Eric Shepherd [:sheppy]]

Actually, on second thought, after discussing this further with shaver, let's go ahead and do this on developer-stage.  Will get more testing that way, and since we're going to be trying to keep in sync with MW updates regularly anyway, this makes sense.

He says I can blame him if it blows up on us sometime down the line. :)

Comment 2

[Jeremy Orem [:oremj]]

Since this is going to be developer-stage.mozilla.org and that is being autopulled from SVN the devmo team should be able to merge in new versions of mediawiki.  The only run in we are going to have is major revisions will need database updates. 
 
Since this does not happen very often I think the best strategy would be to file a bug when a database update needs to occur.

Comment 3

[Eric Shepherd [:sheppy]]

How do we merge in new versions of MediaWiki then?

Comment 4

[Jeremy Orem [:oremj]]

It's actually not super easy.  Here is a good article on doing something similar http://svnbook.red-bean.com/en/1.1/ch07s05.html

Maybe shaver or someone else has a better idea on how to do this.

Comment 5

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

I don't understand why this was marked as "INVALID" -- the bug's request seems pretty valid to me, even if it's not trivial.  We want to get the SVN merge done automatically, and I'm pretty sure that running the maintenance scripts is idempotent, so the db changes shouldn't be a problem either.  The wikipedia guys do exactly this, so they might be able to offer suggestions on the mechanics.

Comment 6

[Jeremy Orem [:oremj]]

I'm not completely sure why this was marked invalid, but I think it was because this is more of a developer bug.

We also have a few local code modifications, so there may need to be someone watching it to resolve conflicts if any occur.

Comment 7

[Reed Loden [:reed]]

(In reply to comment #5)
> The wikipedia guys do exactly this, so they might be able to offer suggestions on the mechanics.

Where did you get this (false) idea? What makes you think the Wikipedia guys have Subversion branches that automerge (because they don't). Wikipedia itself does not use Subversion at all. MediaWiki uses Subversion for development, but they don't use any auto-merging of the sort.

Comment 8

[Jeremy Orem [:oremj]]

Please attach or commit an svn autocommit script and I'll will set it up on cron on developer-stage.


Closing as incomplete until the script is ready.

Comment 9

[Eric Shepherd [:sheppy]]

OK, so who knows how to make such a script?  I have a huge problem closing this as incomplete just because it's not done yet.  I have no idea what an autocommit script is, so whose job is it to create one?  I assumed it was IT's job to do stuff like this.

Comment 10

[Jeremy Orem [:oremj]]

Autocommit script probably wasn't the best term.  What I meant is a script that auto-merges Mediawiki's tip into the devmo repository.  I'm not sure who should take job this either, but it doesn't seem like an IT thing since it requires code changes and code commits.

This will also require some engineering on the developers part to figure out how the merging should be done.  Should it be merged directly in to trunk, in to a branch, tagged each time?  Lots of things probably need to be figured out.

Anyways, we can reopen the bug, but should probably first figure out where it should be reassigned.

Comment 11

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

(In reply to comment #7)
> (In reply to comment #5)
> > The wikipedia guys do exactly this, so they might be able to offer suggestions on the mechanics.
> 
> Where did you get this (false) idea? What makes you think the Wikipedia guys
> have Subversion branches that automerge (because they don't). Wikipedia itself
> does not use Subversion at all. MediaWiki uses Subversion for development, but
> they don't use any auto-merging of the sort.

Wikipedia uses the tip of the trunk for their operation, which is what we want to be doing here.

Comment 12

[Justin Fitzhugh]

I'd agree with oremj here.  We try to stay away from large code changes and merges, rather maintain the site, keep it operational and push updates we get from developers. Good practice to keep dev and production ops separate as well (with qa hopefully somewhere in between).  

Perhaps we should ask webdev to help with the code changes and merges if you
aren't able to?  I'll reopen given the open task of finding a devmo developer
who can work on this.

Comment 13

[Reed Loden [:reed]]

Is this blocking bug 395839?

Comment 14

[Eric Shepherd [:sheppy]]

Oops, got that backward.  Yes, it is.

Comment 15

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

(In reply to comment #12)
> I'd agree with oremj here.  We try to stay away from large code changes and
> merges, rather maintain the site, keep it operational and push updates we get
> from developers.

That's not how it's worked with MDC so far, and without access to the staging environment (how are the maint scripts to be run? how does change control of the configs work? how do we know when/how to do db cloning?) or a high-fidelity replica, I don't think anyone else.  All of the configuration system and layout and update model for MDC at this point came from IT; all we did was get involved when it had gone too long between updates and it needed to be put into SVN.

And we're not talking about large code changes.  We're specifically talking about taking _small_ code changes frequently, so that we can avoid having to do big code changes.  I don't know how IT wants to deal with effectively auto-running code out of a foreign repository, what checks they want to perform to ensure the integrity of an update, etc., which is why I think it's an IT bug to resolve.

> Good practice to keep dev and production ops separate as well
> (with qa hopefully somewhere in between).  

Where are the dev ops, then?  We have developer-stage, for which we don't see database errors or suchlike, and have (same old story) no ability to work with MW's debug logging, etc., and we have the production site.

> Perhaps we should ask webdev to help with the code changes and merges if you
> aren't able to?  I'll reopen given the open task of finding a devmo developer
> who can work on this.

We're not asking for someone to help with merges, we're asking for someone to set it up to run on its own so that developer-stage is always updated according to the MW trunk (like Wikipedia itself).  _All_ the auto-update scripts I've ever worked with for Mozilla apps have been maintained and developed by IT, and it's rare that I've even been able to see them.  I don't think anyone other than IT could actually write them, given that nobody else can actually tell how things are laid out or configured on those machines, or efficiently test their work.  If you're concerned about the core changes (very few at this point, I believe, because the large ones have been upstreamed) breaking the update, then I'm more than fine with the script updating from MW tip, applying our code change patch, and bailing (with the old stuff still in place, natch) if there's a patch application error.  We can work with webdev or contractors or volunteers or whoever to get that patch updated when (rarely, IMO) it's needed.

(What was INCOMPLETE about this bug, and why would it have been RESOLVED in this state?  I really don't understand that; we've tried to be pretty clear about what we need, and answer questions when they're asked. :( )

Comment 16

[Justin Fitzhugh]

I see a lot of your points, but still would like to talk through how we *should* maintain devmo going forward rather than how we have done it, as I think we can all agree it's been less than perfect.  

Can I setup a meeting with everyone (should be quick) to hash this out rather than doing it over bugmail?  Hopefully we can address everyone's concerns and come out the other side with a much better setup.

Sound OK?

Comment 17

[Eric Shepherd [:sheppy]]

A meeting sounds like a good idea.

Comment 18

[Eric Shepherd [:sheppy]]

We have a page of meeting notes regarding the ideas that came out of our meeting:

http://wiki.mozilla.org/DevmoIdeas

Comment 19

[Jeremy Orem [:oremj]]

I've been trying to figure out how to make this all pull out of one directory before I do this, so the scripts aren't as annoying and the directory isn't so bulky.  It looks like we can just pull the language off PHP_SELF or something.  It looks like for maintenance scripts it is possible to specify which db and lang the operation should happen on based on command line arguments.  I will look in to this more tomorrow and try to set up a test server.

Comment 20

[Jeremy Orem [:oremj]]

Right now http://developer-stage.mozilla.org is all running out of one mediawiki directory.  I imported my changes into svn at http://viewvc.svn.mozilla.org/vc/projects/developer.mozilla.org/branches/mediawikionedir/

Comment 21

[Jeremy Orem [:oremj]]

Can we work on pushing this one directory change live?  Mostly, the site just needs to be QA'd.  Does anyone have an opinion on this?

Comment 22

[Jeremy Orem [:oremj]]

Still need a QA of stage devmo.  I assume that the onedir stuff is working since no one has been complaining.

Also, I don't think that it will be possible to auto sync with tip via subversion.  The subversion tools that allow this to happen are interactive.  http://svnbook.red-bean.com/en/1.1/ch07s05.html see "svn_load_dirs.pl".

Comment 23

[Eric Shepherd [:sheppy]]

The site seems to be working to me; I've been doing some work on it recently and it seems to be fine.  I'd say to go for it.

Comment 24

[Jeremy Orem [:oremj]]

I set up a developer box (sm-devmo01) that syncs with mediawiki's trunk.  Right now Eric has root access.  I updated http://wiki.mozilla.org/DevmoIdeas with instruction on how to make changes, sync with the production db, and sync with mediawiki tip.