Last Comment Bug 643324 - Enable the shared checkouts from CCMercurialBuildFactory
: Enable the shared checkouts from CCMercurialBuildFactory
Status: NEW
:
Product: SeaMonkey
Classification: Client Software
Component: Release Engineering (show other bugs)
: unspecified
: x86 Windows XP
: -- normal (vote)
: ---
Assigned To: Justin Wood (:Callek) (Away until Aug 29)
:
Mentors:
Depends on: 645437
Blocks:
  Show dependency treegraph
 
Reported: 2011-03-20 14:53 PDT by Justin Wood (:Callek) (Away until Aug 29)
Modified: 2011-04-13 14:15 PDT (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Use hg shared on CC Builds (2.26 KB, patch)
2011-03-20 14:53 PDT, Justin Wood (:Callek) (Away until Aug 29)
kairo: review+
gozer: feedback+
Details | Diff | Splinter Review
and enable it on windows [Checked in: Comment 6] (1.48 KB, patch)
2011-03-20 14:59 PDT, Justin Wood (:Callek) (Away until Aug 29)
kairo: review+
Details | Diff | Splinter Review
Use hg shared on CC Builds (4.33 KB, patch)
2011-04-02 19:52 PDT, Justin Wood (:Callek) (Away until Aug 29)
no flags Details | Diff | Splinter Review
Use hg shared on CC Builds v3 (4.51 KB, patch)
2011-04-02 22:55 PDT, Justin Wood (:Callek) (Away until Aug 29)
jhopkins: review+
Details | Diff | Splinter Review

Description Justin Wood (:Callek) (Away until Aug 29) 2011-03-20 14:53:32 PDT
Created attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

This patch mimics the shared-checkout code from mozilla-try.

Differences, we (on our normal Mercurial checkout) have mode="update" not "clobber" so I did not define an explicit clobber step here.

I only chose to share mozilla-* and comm-* for now, I have an idea on how to make this code more sharable even with mozilla proper and will expand our list of shared repos then.

This only shares the repo if there is NOT an existing repo in |dest| so will need a clobber to work.

Requires newer hg (I have 1.7.5 on our windows slaves; will upgrade other OS's soon) and share and purge enabled in hgrc. But the tools script this uses falls back to a normal clone/update if share is not enabled, so we should be safe for MoMo until they get around to this.

tested that our client.py stuff works fine on a shared repo, without any worries.

Due to an unrelated issue with our master setup, I cannot yet update to buildbotcustom-default-tip, so if this gets review, I will double-land onto seamonkey-production branch and default.

Requesting feedback from gozer, incase he has any strong objections to the choices I have made here.
Comment 1 Justin Wood (:Callek) (Away until Aug 29) 2011-03-20 14:59:57 PDT
Created attachment 520556 [details] [diff] [review]
and enable it on windows
[Checked in: Comment 6]
Comment 2 Robert Kaiser 2011-03-20 17:10:31 PDT
Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

Let's try it - I don't understand all of this and I trust you did c&p correctly - we'll see if it breaks or not ;-)
Comment 3 Serge Gautherie (:sgautherie) 2011-03-22 08:38:55 PDT
It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
Could you investigate?
Comment 4 Justin Wood (:Callek) (Away until Aug 29) 2011-03-26 19:13:01 PDT
(In reply to comment #3)
> It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
> Could you investigate?

Ok, our problem here is a little complicated. First a bit of background:

We poll the mozilla repo, and the comm* repo and let the Poller fill in the |branch| (which buildbot needs to know what type of build this is). ex. comm-central-trunk.

The poller fills in the |revision| property of the ChangeSource, (actually using the revision of the "newest" changesource if there are multiple).

the hgtool script actually fails if the revision in the ChangeSource is not in the repo we are trying to clone from/update to.

This could fail in both directions, comm-* with a mozilla* Change, or mozilla* with a comm change.

The end result, is that I think we'll need to revisit how we poll from mozilla-central for our builds, and a short term fix would be to disable the mozilla-central poller. [we Do NOT even set the mozilla-central revision when we run a poll for it, so it is mostly useless except in having us do extra builds]. And disable them now (so we can also use HG_SHARED)

Thoughts?
Comment 5 Serge Gautherie (:sgautherie) 2011-03-26 20:40:07 PDT
Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

http://hg.mozilla.org/build/buildbotcustom/rev/f931ca29fe6a (seamonkey-production)
http://hg.mozilla.org/build/buildbotcustom/rev/da039c446a3d
+
http://hg.mozilla.org/build/buildbotcustom/rev/58e9247785c0 (seamonkey-production)
Remove mozilla dir hg shared until I can look more carefully
+
http://hg.mozilla.org/build/buildbotcustom/rev/4a7c77b09ed8 (seamonkey-production)
Backed out changeset 58e9247785c0
+
http://hg.mozilla.org/build/buildbotcustom/rev/206656b2a258 (seamonkey-production)
more backing out of CC-based hg share
+
http://hg.mozilla.org/build/buildbotcustom/rev/79dee6c0f11f
Backed out changeset da039c446a3d
http://hg.mozilla.org/build/buildbotcustom/rev/173c0c31577a
merge backout of Bug 643324 with tip of default
Comment 6 Serge Gautherie (:sgautherie) 2011-03-26 20:46:36 PDT
Comment on attachment 520556 [details] [diff] [review]
and enable it on windows
[Checked in: Comment 6]

http://hg.mozilla.org/build/buildbot-configs/rev/bbbd0aa60454 (seamonkey-production)
http://hg.mozilla.org/build/buildbot-configs/rev/335c4b07474c
Merge seamonkey-production to default
Comment 7 Robert Kaiser 2011-03-27 16:15:37 PDT
(In reply to comment #4)
> The end result, is that I think we'll need to revisit how we poll from
> mozilla-central for our builds, and a short term fix would be to disable the
> mozilla-central poller.

I think disabling that one is dangerous, as we'd not get informed any more when m-c changes break us, esp. as c-c changes are of a very much lower frequency.
Comment 8 Justin Wood (:Callek) (Away until Aug 29) 2011-03-27 16:18:25 PDT
(In reply to comment #7)
> (In reply to comment #4)
> > The end result, is that I think we'll need to revisit how we poll from
> > mozilla-central for our builds, and a short term fix would be to disable the
> > mozilla-central poller.
> 
> I think disabling that one is dangerous, as we'd not get informed any more when
> m-c changes break us, esp. as c-c changes are of a very much lower frequency.

I admit I came up with the same thought; so I decided to attempt to correct our underlying issue here with Bug 645437
Comment 9 Justin Wood (:Callek) (Away until Aug 29) 2011-04-02 19:52:00 PDT
Created attachment 523844 [details] [diff] [review]
Use hg shared on CC Builds

I got rs+=KaiRo over IRC to deploy this to seamonkey, so am landing on seamonkey-production shortly.

Since this is shared with TB, I am asking John for review, Note that it may be useful/necessary for TB to port Bug 645437 for this. (I'm not 100% sure)

Other changes I included pulled "set_got_revision" out of the if buildRevision, so that the tinderboxprint works even without a Mercurial step (otherwise it would fail, since no prop named got_revision)

Set us to always --skip-comm in client.py.
Set us to only use client.py with a --mozilla-repo if we are NOT using shared checkouts.

I'll land this on |default| when I get review.
Comment 10 Justin Wood (:Callek) (Away until Aug 29) 2011-04-02 22:55:06 PDT
Created attachment 523848 [details] [diff] [review]
Use hg shared on CC Builds v3

v3, fixes a typo I missed in v2.

This whole thing could probably use some cleanup too, but this works, and is no worse than was here already.
Comment 11 Justin Wood (:Callek) (Away until Aug 29) 2011-04-02 22:57:54 PDT
v2 pushed to seamonkey-production in:
https://hg.mozilla.org/build/buildbotcustom/rev/a61a8a3fe8ce

With the followup that spawned patch v3:
https://hg.mozilla.org/build/buildbotcustom/rev/25b275fbaed2

Not yet landed on default, so leaving bug open.
Comment 12 John Hopkins (:jhopkins) 2011-04-13 12:12:58 PDT
Looking at this now

Note You need to log in before you can comment on or make changes to this bug.