Enable the shared checkouts from CCMercurialBuildFactory

NEW
Assigned to

Status

SeaMonkey
Release Engineering
6 years ago
6 years ago

People

(Reporter: Callek, Assigned: Callek)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 2 obsolete attachments)

(Assignee)

Description

6 years ago
Created attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

This patch mimics the shared-checkout code from mozilla-try.

Differences, we (on our normal Mercurial checkout) have mode="update" not "clobber" so I did not define an explicit clobber step here.

I only chose to share mozilla-* and comm-* for now, I have an idea on how to make this code more sharable even with mozilla proper and will expand our list of shared repos then.

This only shares the repo if there is NOT an existing repo in |dest| so will need a clobber to work.

Requires newer hg (I have 1.7.5 on our windows slaves; will upgrade other OS's soon) and share and purge enabled in hgrc. But the tools script this uses falls back to a normal clone/update if share is not enabled, so we should be safe for MoMo until they get around to this.

tested that our client.py stuff works fine on a shared repo, without any worries.

Due to an unrelated issue with our master setup, I cannot yet update to buildbotcustom-default-tip, so if this gets review, I will double-land onto seamonkey-production branch and default.

Requesting feedback from gozer, incase he has any strong objections to the choices I have made here.
Attachment #520555 - Flags: review?(kairo)
Attachment #520555 - Flags: feedback?(gozer)
(Assignee)

Updated

6 years ago
Attachment #520555 - Attachment is patch: true
Attachment #520555 - Attachment mime type: application/octet-stream → text/plain
(Assignee)

Comment 1

6 years ago
Created attachment 520556 [details] [diff] [review]
and enable it on windows
[Checked in: Comment 6]
Attachment #520556 - Flags: review?(kairo)

Updated

6 years ago
Attachment #520556 - Flags: review?(kairo) → review+

Comment 2

6 years ago
Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

Let's try it - I don't understand all of this and I trust you did c&p correctly - we'll see if it breaks or not ;-)
Attachment #520555 - Flags: review?(kairo) → review+
It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
Could you investigate?
Attachment #520555 - Flags: feedback?(gozer) → feedback+
(Assignee)

Comment 4

6 years ago
(In reply to comment #3)
> It seems SeaMonkey trunk Windows boxes are failing on this since last "night".
> Could you investigate?

Ok, our problem here is a little complicated. First a bit of background:

We poll the mozilla repo, and the comm* repo and let the Poller fill in the |branch| (which buildbot needs to know what type of build this is). ex. comm-central-trunk.

The poller fills in the |revision| property of the ChangeSource, (actually using the revision of the "newest" changesource if there are multiple).

the hgtool script actually fails if the revision in the ChangeSource is not in the repo we are trying to clone from/update to.

This could fail in both directions, comm-* with a mozilla* Change, or mozilla* with a comm change.

The end result, is that I think we'll need to revisit how we poll from mozilla-central for our builds, and a short term fix would be to disable the mozilla-central poller. [we Do NOT even set the mozilla-central revision when we run a poll for it, so it is mostly useless except in having us do extra builds]. And disable them now (so we can also use HG_SHARED)

Thoughts?
(Assignee)

Updated

6 years ago
Depends on: 645437
Comment on attachment 520555 [details] [diff] [review]
Use hg shared on CC Builds

http://hg.mozilla.org/build/buildbotcustom/rev/f931ca29fe6a (seamonkey-production)
http://hg.mozilla.org/build/buildbotcustom/rev/da039c446a3d
+
http://hg.mozilla.org/build/buildbotcustom/rev/58e9247785c0 (seamonkey-production)
Remove mozilla dir hg shared until I can look more carefully
+
http://hg.mozilla.org/build/buildbotcustom/rev/4a7c77b09ed8 (seamonkey-production)
Backed out changeset 58e9247785c0
+
http://hg.mozilla.org/build/buildbotcustom/rev/206656b2a258 (seamonkey-production)
more backing out of CC-based hg share
+
http://hg.mozilla.org/build/buildbotcustom/rev/79dee6c0f11f
Backed out changeset da039c446a3d
http://hg.mozilla.org/build/buildbotcustom/rev/173c0c31577a
merge backout of Bug 643324 with tip of default
Comment on attachment 520556 [details] [diff] [review]
and enable it on windows
[Checked in: Comment 6]

http://hg.mozilla.org/build/buildbot-configs/rev/bbbd0aa60454 (seamonkey-production)
http://hg.mozilla.org/build/buildbot-configs/rev/335c4b07474c
Merge seamonkey-production to default
Attachment #520556 - Attachment description: and enable it on windows. → and enable it on windows [Checked in: Comment 6]

Comment 7

6 years ago
(In reply to comment #4)
> The end result, is that I think we'll need to revisit how we poll from
> mozilla-central for our builds, and a short term fix would be to disable the
> mozilla-central poller.

I think disabling that one is dangerous, as we'd not get informed any more when m-c changes break us, esp. as c-c changes are of a very much lower frequency.
(Assignee)

Comment 8

6 years ago
(In reply to comment #7)
> (In reply to comment #4)
> > The end result, is that I think we'll need to revisit how we poll from
> > mozilla-central for our builds, and a short term fix would be to disable the
> > mozilla-central poller.
> 
> I think disabling that one is dangerous, as we'd not get informed any more when
> m-c changes break us, esp. as c-c changes are of a very much lower frequency.

I admit I came up with the same thought; so I decided to attempt to correct our underlying issue here with Bug 645437
(Assignee)

Comment 9

6 years ago
Created attachment 523844 [details] [diff] [review]
Use hg shared on CC Builds

I got rs+=KaiRo over IRC to deploy this to seamonkey, so am landing on seamonkey-production shortly.

Since this is shared with TB, I am asking John for review, Note that it may be useful/necessary for TB to port Bug 645437 for this. (I'm not 100% sure)

Other changes I included pulled "set_got_revision" out of the if buildRevision, so that the tinderboxprint works even without a Mercurial step (otherwise it would fail, since no prop named got_revision)

Set us to always --skip-comm in client.py.
Set us to only use client.py with a --mozilla-repo if we are NOT using shared checkouts.

I'll land this on |default| when I get review.
Attachment #520555 - Attachment is obsolete: true
Attachment #523844 - Flags: review?(john.hopkins)
(Assignee)

Updated

6 years ago
Attachment #523844 - Attachment is patch: true
Attachment #523844 - Attachment mime type: application/octet-stream → text/plain
(Assignee)

Comment 10

6 years ago
Created attachment 523848 [details] [diff] [review]
Use hg shared on CC Builds v3

v3, fixes a typo I missed in v2.

This whole thing could probably use some cleanup too, but this works, and is no worse than was here already.
Attachment #523844 - Attachment is obsolete: true
Attachment #523844 - Flags: review?(john.hopkins)
Attachment #523848 - Flags: review?(john.hopkins)
(Assignee)

Comment 11

6 years ago
v2 pushed to seamonkey-production in:
https://hg.mozilla.org/build/buildbotcustom/rev/a61a8a3fe8ce

With the followup that spawned patch v3:
https://hg.mozilla.org/build/buildbotcustom/rev/25b275fbaed2

Not yet landed on default, so leaving bug open.
Looking at this now
Attachment #523848 - Flags: review?(john.hopkins) → review+
You need to log in before you can comment on or make changes to this bug.