448014 - Support pushing changes to the try server

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Jason Orendorff [:jorendorff]]

Kicking off a try server build isn't exactly hard, but what if it could be as simple as:

  hg push -f ssh://hg.mozilla.org/try

It would be pretty cool if you didn't have to create a user repo or export a patch, and you didn't have to visit the try server web page.

Here's what I think would have to be done:

- Create the repo and make sure it allows multiple heads.

- The repo would need a push hook (or whatever) that identifies the new head and sends it along to the try server.

- (nice to have) The repo should enforce a rule that you're only allowed to push one *new* head at a time (otherwise, which new head should it build?)

I wouldn't mind trying this.

Comment 1

[[e-mail] (:bhearsum)]

This is a pretty neat idea....

The commit-hook will need to generate an info file like this: http://mxr.mozilla.org/mozilla/source/webtools/buildbot-try/sendchange.cgi#226 and drop it in the patches directory, same as the web interface would do. After that the script on the tryserver master will pull it down and do its thing.

Comment 2

[(not currently active) Ted Mielczarek]

I'm sure you've already seen it, but the pushlog hook lives here:
http://hg.mozilla.org/users/bsmedberg_mozilla.com/index.cgi/hghooks/file/259dc6ae87e6/mozhghooks/pushlog.py

and the hgrc for how to hook it up is there too:
http://hg.mozilla.org/users/bsmedberg_mozilla.com/index.cgi/hghooks/file/259dc6ae87e6/example-hgrc

the HG book has a pretty good section on writing hooks:
http://hgbook.red-bean.com/hgbookch10.html#x14-20900010.5

Comment 3

[Dirkjan Ochtman (:djc)]

Heh, neat idea. You know where to find me if you need help from the hg side.

Comment 4

[Jason Orendorff [:jorendorff]]

Hmm, ok, that looks easy enough.

First we need an hg repository set up on build.m.o (or some other server that can write to that directory).  Separate bug?

Comment 5

[Jason Orendorff [:jorendorff]]

I need some server admin type help with this.  The options I see are:

1.  Host the repo on hg.m.o.  The hook can use ssh, NFS, or SMB (or something) to put the try-server-request file into the appropriate directory on build.m.o.  Is any of that already in place?  Can we set it up?

2.  Host the repo on build.m.o.  Then the hook can just write the file directly to the local filesystem.

On my end, the hook is pretty trivial to code up either way.  (The hg part turned out to be really easy; the rest is logistics.)

Comment 6

[Benjamin Smedberg]

That's a question for aravind+bhearsum. I tend to think we should use hg.mozilla.org for this.

Comment 7

[[e-mail] (:bhearsum)]

Yeah, I guess we should since we already have all the authentication set-up properly. Rather than ssh, NFS, or SMB, can we just have a script run in a crontab on build.m.o that pulls changes in and generates the .info files? Not sure if it'd work that way, though.

Comment 8

[Jason Orendorff [:jorendorff]]

bsmedberg suggested a different approach.  We'll host the repo on hg.mozilla.org.  We'll make buildbot monitor that repo's push-log feed and automatically build each tip someone pushes.

bhearsum, if you have a spare half hour, I'd like to get together and do this.

Comment 9

[[e-mail] (:bhearsum)]

Attachment: [checked in] make HgPoller support ?tipsonly

Comment 10

[[e-mail] (:bhearsum)]

Attachment: [checked in] special case PATCH_TRY, HG_TRY, to make it possible to pull changes from hg.m.o/try

Comment 11

[[e-mail] (:bhearsum)]

Attachment: minor master.cfg update

Comment 12

[alice nodelman [:alice] [:anode]]

Comment on attachment 331737 [details] [diff] [review]
[checked in] special case PATCH_TRY, HG_TRY, to make it possible to pull changes from hg.m.o/try

Other than abuse of the term 'branch', looks good.

Comment 13

[[e-mail] (:bhearsum)]

Attachment: [checked in] again, with a scheduler this time

Comment 14

[[e-mail] (:bhearsum)]

Comment on attachment 331677 [details] [diff] [review]
[checked in] make HgPoller support ?tipsonly

Checking in hgpoller.py;
/cvsroot/mozilla/tools/buildbotcustom/changes/hgpoller.py,v  <--  hgpoller.py
new revision: 1.3; previous revision: 1.2
done

Comment 15

[[e-mail] (:bhearsum)]

Comment on attachment 331737 [details] [diff] [review]
[checked in] special case PATCH_TRY, HG_TRY, to make it possible to pull changes from hg.m.o/try

Checking in steps.py;
/cvsroot/mozilla/tools/buildbotcustom/tryserver/steps.py,v  <--  steps.py
new revision: 1.5; previous revision: 1.4
done

Comment 16

[[e-mail] (:bhearsum)]

Comment on attachment 331742 [details] [diff] [review]
[checked in] again, with a scheduler this time

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/tryserver/master.cfg,v  <--  master.cfg
new revision: 1.17; previous revision: 1.16
done

Comment 17

[[e-mail] (:bhearsum)]

Attachment: [checked in] add an HgPoller so this thing actually works

I missed this in my first master.cfg, wtf?

Comment 18

[[e-mail] (:bhearsum)]

Comment on attachment 332517 [details] [diff] [review]
[checked in] add an HgPoller so this thing actually works

pre-landed this:
Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/tryserver/master.cfg,v  <--  master.cfg
new revision: 1.18; previous revision: 1.17
done

Comment 19

[[e-mail] (:bhearsum)]

Had to land a bustage fix (use urljoin instead of 'foo + bar' to create the hg.m.o url) because Mercurial barfs when there is extra slasheys in the URL.

Other than that, looks like this is working fine.