721895 - merge github and hg graphserver codebases

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect)

Description

[Robert Helmer [:rhelmer]]

From bug 721884 comment 0:

"""
Right now graphs-old.mozilla.org ("1.0") runs the old codebase (http://hg.mozilla.org/graphs branch 1.0) and this is where talos traffic is sent to right now.

graphs.mozilla.org ("2.0, https://github.com/mozilla/graphs branch master) was built using the hg-git plugin, so it is based on the same history, but contains a completely rewritten UI and some improvements to the server-side code (wsgifying some of the CGIs for example).
"""

Once the above is done, let's unify the codebases. I think we should:

1) merge master from github onto default branch in hg
2) consider hg to be canonical, but merge to github (perhaps automatically)

We get most "casual" contributions via github, so I would like to maintain that even if just as a mirror for people not as familiar with mozilla-specific infra.

Comment 1

[Robert Helmer [:rhelmer]]

Merged github to default branch on hg and pushed back up to github:

https://github.com/mozilla/graphs/commit/1f18d27e9213d8334ff122d7ab8550cd8909d39e

Comment 2

[Robert Helmer [:rhelmer]]

Next steps are:

1) merge hg 1.0 branch to default branch (and push to github)
2) move backend to new prod/staging per bug 721884
3) update perfomatic wiki docs to point to proper repos
4) mark hg 1.0 branch "closed"

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #2)
> Next steps are:
> 
> 1) merge hg 1.0 branch to default branch (and push to github)

Doesn't look like we have any changes that need merging! I think we're just waiting on bug 721884 to do the remaining steps from comment 2.

Comment 4

[Robert Helmer [:rhelmer]]

Hrm actually this looks easier than it is, since the 1.0 branch diverged at one point and wasn't resolved. I'll just do that work by hand on the 1.0 branch now.

Comment 5

[Robert Helmer [:rhelmer]]

Attachment: merge graphs 1.0 and 2.0 onto hg default branch

Sorry for the size of this diff :) This is an attempt to merge these three separate codebases:

1) ibicking's work (from his hg repo)
2) graphs 2.0 frontend work from chowse and myself
3) the 1.0 branch in hg (currently used to receive production pushes from talos)

This work is a precursor to moving talos writes from the old server to the new (bug 721884).

This massive patch is really just wrangling to get the history in line; if you apply this and look at an actual diff with the 1.0 branch, you'll see the changes in server/ and sql/ are mostly cosmetic.

Once this is applied, we should move graphs-old staging/prod to the default branch, and we can move talos traffic over to the new servers ASAP after that (I am going to suggest that we copy the old deployment and do a new clone on the servers, just in case there are manual changes etc. we're missing on the old servers).

Comment 6

[Robert Helmer [:rhelmer]]

Comment on attachment 594307 [details]
merge graphs 1.0 and 2.0 onto hg default branch

Actually you know what scratch this - it's going to make 1.0 branch incompatible, which defeats the purpose of graphs-old

What I'll do is:

1) push this to my own hg repo
2) update 2.0 stage/prod to #1
3) get talos writes going to stage/prod 2.0
4) ask for my hg repo to be merged to graphs
5) switch 2.0 stage/prod to using official hg repo

That way we leave alone the 1.0 servers until they are ready for decommissioning.

Comment 7

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #6)
> Comment on attachment 594307 [details]
> merge graphs 1.0 and 2.0 onto hg default branch
> 
> Actually you know what scratch this - it's going to make 1.0 branch
> incompatible, which defeats the purpose of graphs-old
> 
> What I'll do is:
> 
> 1) push this to my own hg repo

OK I have this done and am maintaining it at http://hg.mozilla.org/users/robert_roberthelmer.com/

> 2) update 2.0 stage/prod to #1
> 3) get talos writes going to stage/prod 2.0
> 4) ask for my hg repo to be merged to graphs
> 5) switch 2.0 stage/prod to using official hg repo
> 
> That way we leave alone the 1.0 servers until they are ready for
> decommissioning.

Comment 8

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #7)
> (In reply to Robert Helmer [:rhelmer] from comment #6)
> > Comment on attachment 594307 [details]
> > merge graphs 1.0 and 2.0 onto hg default branch
> > 
> > Actually you know what scratch this - it's going to make 1.0 branch
> > incompatible, which defeats the purpose of graphs-old
> > 
> > What I'll do is:
> > 
> > 1) push this to my own hg repo
> 
> OK I have this done and am maintaining it at
> http://hg.mozilla.org/users/robert_roberthelmer.com/

Actually here's an even better version, using my @mozilla.com account, which resets the 1.0 branch so it's identical to what production is using now:

http://hg.mozilla.org/users/rhelmer_mozilla.com/graphs

Comment 9

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #8)
> http://hg.mozilla.org/users/rhelmer_mozilla.com/graphs

staging is now using this branch, and I have written down instructions for switching from git to hg (keeping a backup of the git clone just in case).

(In reply to Robert Helmer [:rhelmer] from comment #6) 
> 1) push this to my own hg repo
> 2) update 2.0 stage/prod to #1

Staging is ready for testing, following up on that in dependent bug 721884

> 3) get talos writes going to stage/prod 2.0
> 4) ask for my hg repo to be merged to graphs

Technically we could do this now since I reset the 1.0 branch on my repo, but I thin it's probably easiest for everybody if we just wait and I'll keep things merged in the meantime.

> 5) switch 2.0 stage/prod to using official hg repo

Comment 10

[Robert Helmer [:rhelmer]]

https://hg.mozilla.org/graphs is now merged.

Please use the default branch, not the 1.0 branch!

I have kept the 1.0 branch the same (different changesets but same content), just for historical purposes.

I've also taken a copy of the repo from before I modified anything for comparison purposes, and everything looks as expected. I'll push that somewhere in case there are any concerns.

Comment 11

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/94f42a6ec031787edb4309c2796f46b34b4673da
bug 721895 - merge hg 1.0 branch into default

--HG--
branch : 1.0

https://github.com/mozilla/treeherder/commit/e9e0e1244b89521bf0c5cf5582c0f43482a0003f
bug 721895 - merge hg 1.0 branch to default