Port talos master code to schedulerdb

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
8 years ago
4 years ago

People

(Reporter: catlee, Assigned: catlee)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos][buildbot])

Attachments

(3 attachments)

Comment hidden (empty)
(Assignee)

Updated

8 years ago
Depends on: 568617
(Assignee)

Comment 1

7 years ago
Created attachment 459511 [details] [diff] [review]
WIP

Updated

7 years ago
Blocks: 563458
(Assignee)

Updated

7 years ago
Depends on: 584039
(Assignee)

Updated

7 years ago
Whiteboard: [talos][buildbot]
(Assignee)

Comment 2

7 years ago
Created attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

Notable changes:

* The previously generic "Rev3 Fedora mozilla-central talos" suite is now named
  "Rev3 Fedora mozilla-central talos chrome"

* builddirs are now named $branch_$platform_test-$suite (used to be
  $branch-$platform$suite)

* scheduler names have a 'test-' prefix.  This shouldn't affect anything, and
  just ensures that the test schedulers don't interfere with the builder
  schedulers.

* talos sendchanges have a '-talos' suffix for the branch name
Attachment #462435 - Flags: review?(bhearsum)
Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

Went through this a bit on IRC with Catlee. Looks good.
Attachment #462435 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 4

7 years ago
Created attachment 462454 [details] [diff] [review]
New configs for mozilla-tests masters

These configs should be similar to the mozilla/ configuration.

Things of note:

* master_common.py is shared

* sendchanges for builds should go to only 1 master that writes to the db.  The tests scheduler master will see the changes and create build requests.  The PBChangeSource in the tests scheduler master is for convenience, if you want/need to sendchange from the tests masters.
Attachment #462454 - Flags: review?(bhearsum)
Comment on attachment 462454 [details] [diff] [review]
New configs for mozilla-tests masters

Looks good for the most part. A couple of comments:
Are you sure you want to put the scheduler master on pm01? It looks to be the most heavily loaded of pm01, 02, and 03, with twice the average load of 02.

I see that you're sticking with the ACTIVE_PLATFORMS split for now, do you any plans to change that soon? It'd be great to survive master explosions better on this side of things.

r=me regardless
Attachment #462454 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 6

7 years ago
(In reply to comment #5)
> Comment on attachment 462454 [details] [diff] [review]
> New configs for mozilla-tests masters
> 
> Looks good for the most part. A couple of comments:
> Are you sure you want to put the scheduler master on pm01? It looks to be the
> most heavily loaded of pm01, 02, and 03, with twice the average load of 02.

Where do you see that?  My intent is to put the scheduler master on pm02.  The sendchanges can still go to pm01 though.

> I see that you're sticking with the ACTIVE_PLATFORMS split for now, do you any
> plans to change that soon? It'd be great to survive master explosions better on
> this side of things.

No plans right now.  Let's get this deployed and see how things stand first.
(Assignee)

Updated

7 years ago
Blocks: 583819
(In reply to comment #6)
> (In reply to comment #5)
> > Comment on attachment 462454 [details] [diff] [review] [details]
> > New configs for mozilla-tests masters
> > 
> > Looks good for the most part. A couple of comments:
> > Are you sure you want to put the scheduler master on pm01? It looks to be the
> > most heavily loaded of pm01, 02, and 03, with twice the average load of 02.
> 
> Where do you see that?  My intent is to put the scheduler master on pm02.  The
> sendchanges can still go to pm01 though.
> 

The localconfig's filename says "pm02":
diff --git a/mozilla-tests/production_tests_scheduler_master_pm02_localconfig.py b/mozilla-tests/production_tests_scheduler_master_pm02_localconfig.py
(Assignee)

Comment 8

7 years ago
Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

changeset:   851:edba7275dec1
Attachment #462435 - Flags: checked-in+
(Assignee)

Comment 9

7 years ago
Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

Whoops, that should be:
changeset:   852:2ccf1930cb52
(Assignee)

Comment 10

7 years ago
Comment on attachment 462454 [details] [diff] [review]
New configs for mozilla-tests masters

changeset:   2787:67bbba13a116
Attachment #462454 - Flags: checked-in+
(Assignee)

Comment 11

7 years ago
Seems to be working well.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED

Updated

7 years ago
Blocks: 584857

Updated

7 years ago
Blocks: 585098

Updated

7 years ago
Blocks: 590369
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.