659767 - Buildbot: add support to run redux branches in the Main Build (Phase 1 & 2) Builders

Status: RESOLVED FIXED

(Tamarin Graveyard :: Tools, defect)

Description

[Chris Peyer]

We currently only run tamarin-redux in the Main Build.  We need to be able to also run redux branches (eg Serrano) through a full buildbot pass.

Comment 1

[Chris Peyer]

Attachment: Modify hg_buildbot_trigger.py to be compatible with SandboxClone step.

hg_buildbot_trigger.py is called by hgbuildbot_trigger.sh which is called using a hg hook.

This patch adds a requirement of also passing in the calling repos url which is prepended to the changefile so that the SandboxClone step (defined in shellAddons.py) can pull the url out.

The name of the changefile is also prepended to the file list so that the SandboxClone step knows which changefile to open.

Comment 2

[Chris Peyer]

Attachment: Change tamarinredux to clone using sync_clone_sandbox instead of sync_clone

Note that I haven't removed the sync_clone step from commonsteps.py yet.  If all goes well we could also remove that code.

Comment 3

[Chris Peyer]

In addition to the changes attached to the bug, we will also have to add the repo url to the hg hook on asteam as the last parameter.

For serrano, the priority should be set to 1, the highest priority (but changes can still be batched).

Comment 4

[Chris Peyer]

Attachment: Set the priority1 branch to serrano.

Comment 5

[Chris Peyer]

Attachment: Set the priority1 branch to serrano.

Resubmit patch since I forgot to also update deep.

Comment 7

[Tamarin Bot]

changeset: 6351:b5b4f5ce595b
user:      Brent Baker <[e-mail]>
summary:   Bug 659767: add support to run redux branches in the Main Build (Phase 1 & 2) Builders (p=cpeyer, r=brbaker)

http://hg.mozilla.org/tamarin-redux/rev/b5b4f5ce595b

Comment 8

[Tamarin Bot]

changeset: 6352:2cd215ced7c5
user:      Brent Baker <[e-mail]>
summary:   Bug 659767: couple more tweaks for getting multiple branches working in buildbot  (r=brbaker)

http://hg.mozilla.org/tamarin-redux/rev/2cd215ced7c5

Comment 9

[Brent Baker]

Attachment: Handle multiple branches

Ability for schedulers to handle multiple branches instead of a single branch.

I needed to pull in the base Scheduler.addChange() so that I could override the branch check.

Comment 10

[Brent Baker]

(In reply to comment #9)
> Created attachment 535365 [details] [diff] [review] [review]
> Handle multiple branches
> 
> Ability for schedulers to handle multiple branches instead of a single
> branch.
> 
> I needed to pull in the base Scheduler.addChange() so that I could override
> the branch check.

Chris,

Do you think that we could actually leverage this change so that we could ditch the sandbox.py? We could just add "sandbox" to the branches that the "test" phase scheduler would respond to and not the deep.

Comment 11

[Chris Peyer]

Comment on attachment 535365 [details] [diff] [review]
Handle multiple branches

Review of attachment 535365 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.

Comment 12

[Tamarin Bot]

changeset: 6357:377ba538bec1
user:      Brent Baker <[e-mail]>
summary:   Bug 659767: handle multiple branches in a single scheduler (r=cpeyer)

http://hg.mozilla.org/tamarin-redux/rev/377ba538bec1

Comment 13

[Chris Peyer]

Attachment: pass branch/priority info to PhaseOneListener

Comment 14

[Brent Baker]

Comment on attachment 536523 [details] [diff] [review]
pass branch/priority info to PhaseOneListener

I also needed to add the priorities info to the PhaseTwoScheduler since it needs to know the priority for the branch so that it can open up the build request file (I added that in with the changeset that I pushed)

Comment 15

[Tamarin Bot]

changeset: 6363:4cc7eaabff29
user:      Brent Baker <[e-mail]>
summary:   Bug 659767: pass branch/priority information to the Listeners and also set it on the PhaseTwo scheduler (p=cpeyer, r=brbaker)

http://hg.mozilla.org/tamarin-redux/rev/4cc7eaabff29

Comment 16

[Brent Baker]

Attachment: do not submit unimportant changes to the scheduler

Do not submit unimportant changes to the scheduler. When building multiple branches it is possible that there will be an unimportant change submitted from one branch, which will not be built, and then an important change will come in for a different branch. Previously the unimportant change from a different branch would be added as a change to build, but this causes the wrong repo to be synced.

Comment 17

[Brent Baker]

Comment on attachment 536917 [details] [diff] [review]
do not submit unimportant changes to the scheduler

Review of attachment 536917 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/buildbot/master/custom/buildbot_ext/scheduler.py
@@ +75,5 @@
>          # clear out our state
>          self.timer = None
>          self.nextBuildTime = None
> +        # Only build changes that are deemed important by self.fileIsImportant
> +       changes = self.importantChanges

I have fixed this indentation locally

Comment 18

[Tamarin Bot]

changeset: 6366:29a2cced7d48
user:      Brent Baker <[e-mail]>
summary:   Bug 659767: do not submit unimportant changes to the scheduler (r=cpeyer)

http://hg.mozilla.org/tamarin-redux/rev/29a2cced7d48