643324 - Enable the shared checkouts from CCMercurialBuildFactory

Status: RESOLVED FIXED

(SeaMonkey :: Release Engineering, defect)

Description

[Justin Wood (:Callek)]

Attachment: Use hg shared on CC Builds

This patch mimics the shared-checkout code from mozilla-try.

Differences, we (on our normal Mercurial checkout) have mode="update" not "clobber" so I did not define an explicit clobber step here.

I only chose to share mozilla-* and comm-* for now, I have an idea on how to make this code more sharable even with mozilla proper and will expand our list of shared repos then.

This only shares the repo if there is NOT an existing repo in |dest| so will need a clobber to work.

Requires newer hg (I have 1.7.5 on our windows slaves; will upgrade other OS's soon) and share and purge enabled in hgrc. But the tools script this uses falls back to a normal clone/update if share is not enabled, so we should be safe for MoMo until they get around to this.

tested that our client.py stuff works fine on a shared repo, without any worries.

Due to an unrelated issue with our master setup, I cannot yet update to buildbotcustom-default-tip, so if this gets review, I will double-land onto seamonkey-production branch and default.

Requesting feedback from gozer, incase he has any strong objections to the choices I have made here.

Comment 1

[Justin Wood (:Callek)]

Attachment: and enable it on windows [Checked in: Comment 6]

Comment 2

[Robert Kaiser]

Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

Let's try it - I don't understand all of this and I trust you did c&p correctly - we'll see if it breaks or not ;-)

Comment 3

[Serge Gautherie (:sgautherie)]

It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
Could you investigate?

Comment 4

[Justin Wood (:Callek)]

(In reply to comment #3)
> It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
> Could you investigate?

Ok, our problem here is a little complicated. First a bit of background:

We poll the mozilla repo, and the comm* repo and let the Poller fill in the |branch| (which buildbot needs to know what type of build this is). ex. comm-central-trunk.

The poller fills in the |revision| property of the ChangeSource, (actually using the revision of the "newest" changesource if there are multiple).

the hgtool script actually fails if the revision in the ChangeSource is not in the repo we are trying to clone from/update to.

This could fail in both directions, comm-* with a mozilla* Change, or mozilla* with a comm change.

The end result, is that I think we'll need to revisit how we poll from mozilla-central for our builds, and a short term fix would be to disable the mozilla-central poller. [we Do NOT even set the mozilla-central revision when we run a poll for it, so it is mostly useless except in having us do extra builds]. And disable them now (so we can also use HG_SHARED)

Thoughts?

Comment 5

[Serge Gautherie (:sgautherie)]

Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

http://hg.mozilla.org/build/buildbotcustom/rev/f931ca29fe6a (seamonkey-production)
http://hg.mozilla.org/build/buildbotcustom/rev/da039c446a3d
+
http://hg.mozilla.org/build/buildbotcustom/rev/58e9247785c0 (seamonkey-production)
Remove mozilla dir hg shared until I can look more carefully
+
http://hg.mozilla.org/build/buildbotcustom/rev/4a7c77b09ed8 (seamonkey-production)
Backed out changeset 58e9247785c0
+
http://hg.mozilla.org/build/buildbotcustom/rev/206656b2a258 (seamonkey-production)
more backing out of CC-based hg share
+
http://hg.mozilla.org/build/buildbotcustom/rev/79dee6c0f11f
Backed out changeset da039c446a3d
http://hg.mozilla.org/build/buildbotcustom/rev/173c0c31577a
merge backout of Bug 643324 with tip of default

Comment 6

[Serge Gautherie (:sgautherie)]

Comment on attachment 520556 [details] [diff] [review]
and enable it on windows
[Checked in: Comment 6]

http://hg.mozilla.org/build/buildbot-configs/rev/bbbd0aa60454 (seamonkey-production)
http://hg.mozilla.org/build/buildbot-configs/rev/335c4b07474c
Merge seamonkey-production to default

Comment 7

[Robert Kaiser]

(In reply to comment #4)
> The end result, is that I think we'll need to revisit how we poll from
> mozilla-central for our builds, and a short term fix would be to disable the
> mozilla-central poller.

I think disabling that one is dangerous, as we'd not get informed any more when m-c changes break us, esp. as c-c changes are of a very much lower frequency.

Comment 8

[Justin Wood (:Callek)]

(In reply to comment #7)
> (In reply to comment #4)
> > The end result, is that I think we'll need to revisit how we poll from
> > mozilla-central for our builds, and a short term fix would be to disable the
> > mozilla-central poller.
> 
> I think disabling that one is dangerous, as we'd not get informed any more when
> m-c changes break us, esp. as c-c changes are of a very much lower frequency.

I admit I came up with the same thought; so I decided to attempt to correct our underlying issue here with Bug 645437

Comment 9

[Justin Wood (:Callek)]

Attachment: Use hg shared on CC Builds

I got rs+=KaiRo over IRC to deploy this to seamonkey, so am landing on seamonkey-production shortly.

Since this is shared with TB, I am asking John for review, Note that it may be useful/necessary for TB to port Bug 645437 for this. (I'm not 100% sure)

Other changes I included pulled "set_got_revision" out of the if buildRevision, so that the tinderboxprint works even without a Mercurial step (otherwise it would fail, since no prop named got_revision)

Set us to always --skip-comm in client.py.
Set us to only use client.py with a --mozilla-repo if we are NOT using shared checkouts.

I'll land this on |default| when I get review.

Comment 10

[Justin Wood (:Callek)]

Attachment: Use hg shared on CC Builds v3

v3, fixes a typo I missed in v2.

This whole thing could probably use some cleanup too, but this works, and is no worse than was here already.

Comment 11

[Justin Wood (:Callek)]

v2 pushed to seamonkey-production in:
https://hg.mozilla.org/build/buildbotcustom/rev/a61a8a3fe8ce

With the followup that spawned patch v3:
https://hg.mozilla.org/build/buildbotcustom/rev/25b275fbaed2

Not yet landed on default, so leaving bug open.

Comment 12

[John Hopkins (:jhopkins)]

Looking at this now

Comment 13

[Edmund Wong (:ewong)]

Closing this bug as:

1. Firefox no longer uses Buildbot for release
2. The patch as been pushed to our seamonkey-production branch (which has been migrated to its own
   tree : https://hg.mozilla.org/SeaMonkey/buildbotcustom