568568 - Port talos master code to schedulerdb

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Comment 1

[Chris AtLee [:catlee]]

Attachment: WIP

Comment 2

[Chris AtLee [:catlee]]

Attachment: buildbotcustom changes for talos on 080

Notable changes:

* The previously generic "Rev3 Fedora mozilla-central talos" suite is now named
  "Rev3 Fedora mozilla-central talos chrome"

* builddirs are now named $branch_$platform_test-$suite (used to be
  $branch-$platform$suite)

* scheduler names have a 'test-' prefix.  This shouldn't affect anything, and
  just ensures that the test schedulers don't interfere with the builder
  schedulers.

* talos sendchanges have a '-talos' suffix for the branch name

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

Went through this a bit on IRC with Catlee. Looks good.

Comment 4

[Chris AtLee [:catlee]]

Attachment: New configs for mozilla-tests masters

These configs should be similar to the mozilla/ configuration.

Things of note:

* master_common.py is shared

* sendchanges for builds should go to only 1 master that writes to the db.  The tests scheduler master will see the changes and create build requests.  The PBChangeSource in the tests scheduler master is for convenience, if you want/need to sendchange from the tests masters.

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 462454 [details] [diff] [review]
New configs for mozilla-tests masters

Looks good for the most part. A couple of comments:
Are you sure you want to put the scheduler master on pm01? It looks to be the most heavily loaded of pm01, 02, and 03, with twice the average load of 02.

I see that you're sticking with the ACTIVE_PLATFORMS split for now, do you any plans to change that soon? It'd be great to survive master explosions better on this side of things.

r=me regardless

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #5)
> Comment on attachment 462454 [details] [diff] [review]
> New configs for mozilla-tests masters
> 
> Looks good for the most part. A couple of comments:
> Are you sure you want to put the scheduler master on pm01? It looks to be the
> most heavily loaded of pm01, 02, and 03, with twice the average load of 02.

Where do you see that?  My intent is to put the scheduler master on pm02.  The sendchanges can still go to pm01 though.

> I see that you're sticking with the ACTIVE_PLATFORMS split for now, do you any
> plans to change that soon? It'd be great to survive master explosions better on
> this side of things.

No plans right now.  Let's get this deployed and see how things stand first.

Comment 7

[[e-mail] (:bhearsum)]

(In reply to comment #6)
> (In reply to comment #5)
> > Comment on attachment 462454 [details] [diff] [review] [details]
> > New configs for mozilla-tests masters
> > 
> > Looks good for the most part. A couple of comments:
> > Are you sure you want to put the scheduler master on pm01? It looks to be the
> > most heavily loaded of pm01, 02, and 03, with twice the average load of 02.
> 
> Where do you see that?  My intent is to put the scheduler master on pm02.  The
> sendchanges can still go to pm01 though.
> 

The localconfig's filename says "pm02":
diff --git a/mozilla-tests/production_tests_scheduler_master_pm02_localconfig.py b/mozilla-tests/production_tests_scheduler_master_pm02_localconfig.py

Comment 8

[Chris AtLee [:catlee]]

Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

changeset:   851:edba7275dec1

Comment 9

[Chris AtLee [:catlee]]

Comment on attachment 462435 [details] [diff] [review]
buildbotcustom changes for talos on 080

Whoops, that should be:
changeset:   852:2ccf1930cb52

Comment 10

[Chris AtLee [:catlee]]

Comment on attachment 462454 [details] [diff] [review]
New configs for mozilla-tests masters

changeset:   2787:67bbba13a116

Comment 11

[Chris AtLee [:catlee]]

Seems to be working well.