1421062 - [meta] Setup scriptworkers for thunderbird builds.

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, enhancement)

Comment 1

[Johan Lorenzo [:jlorenzo]]

Talked with Aki. I'll help on it. His instructions were:

> - We need to update our scopes / tree info. These are currently in scriptworker/constants.py, but they're overrideable in the puppet scriptworker.yaml.
>   - we need new `valid_vcs_rules` [1] to allow for the comm-* branch paths on hg.m.o. Ideally this would be indexed by cot_product, but since we're overriding this value, that's ok.
>   - we need new `cot_restricted_scopes` [2], to specify which scopes are restricted. We're thinking about replacing `project/releng/` with `project/thunderbird/` or something. It's probably best to discuss exact naming with the taskcluster team. Tom Prince has an idea of how to specify the `project/releng/` vs `project/thunderbird/` in-tree.
>   - we need new `cot_restricted_trees` [3], to specify which trees can have access to which scopes. Tom Prince probably knows best here; these will probably be fairly similar to the firefox tree layout, but comm-*.
>   - we currently have a single scriptworker.yaml template [4]. We could make this shared, but I think it might be easiest if we allow for an alternate template, and hardcode the thunderbird cot info in it.
> 
> - we'll need new worker clientIds with appropriate scopes.
> 
> - we need a new `comm-dep` (or something, tb-dep? cc-dep?) env_config for signing_scriptworker [5], balrog_scriptworker, and beetmover_scriptworker. These won't have gpg sig verification or signing, and will go to the dep [signing/balrog] servers and s3 buckets, I think. We should double check on balrog and beetmover. These will use the moco signing servers for now.
> 
> - we need to spin up an instance per type, and puppetize.
> 
> - we need in-tree patches to allow for tb support while defaulting to firefox scopes.
> 
> 
> [1] https://github.com/mozilla-releng/scriptworker/blob/513fe573feb5eb4b6f9c4780b44586a057d85ebc/scriptworker/constants.py#L189
> [2] https://github.com/mozilla-releng/scriptworker/blob/513fe573feb5eb4b6f9c4780b44586a057d85ebc/scriptworker/constants.py#L207
> [3] https://github.com/mozilla-releng/scriptworker/blob/513fe573feb5eb4b6f9c4780b44586a057d85ebc/scriptworker/constants.py#L228
> [4] https://hg.mozilla.org/build/puppet/file/64367087754a/modules/scriptworker/templates/scriptworker.yaml.erb
> [5] https://hg.mozilla.org/build/puppet/file/64367087754a/modules/signing_scriptworker/manifests/settings.pp#l25

Comment 2

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> >   - we need new `valid_vcs_rules` [1] to allow for the comm-* branch paths on hg.m.o. Ideally this would be indexed by cot_product, but since we're overriding this value, that's ok.
Keying by cot_product is handled by :tomprince in https://github.com/mozilla-releng/scriptworker/pull/170

Comment 3

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> >   - we need new `cot_restricted_scopes` [2], to specify which scopes are restricted. We're thinking about replacing `project/releng/` with `project/thunderbird/` or something. It's probably best to discuss exact naming with the taskcluster team. Tom Prince has an idea of how to specify the `project/releng/` vs `project/thunderbird/` in-tree.
> ...
> > - we'll need new worker clientIds with appropriate scopes.

The clients name and scopes aren't defined yet. However, I dug #taskcluster and found (UTC time):

> [2017-11-03 19:42:29] <tomprince> jonasfj: We are talking about how to seperate out comm-central stuff from mozilla-central stuff.
> [2017-11-03 19:42:37] <jonasfj> if it's not gecko related... then a project.<project>. namespace is likely appropriate, or if it's related to a github repo, maybe we come up with a new namespace pattern
> [2017-11-03 19:42:49] <jonasfj> ahm
> [2017-11-03 19:43:08] <dustin> yeah
> [2017-11-03 19:43:11] <dustin> somewhere in between all those things
> [2017-11-03 19:43:12] <jonasfj> comm-central == thunderbird, right?
> [2017-11-03 19:43:13] <dustin> gecko? yeah
> [2017-11-03 19:43:16] <dustin> firefox/ no
> [2017-11-03 19:43:28] <tomprince> jonasfj: Yeah.
> [2017-11-03 19:43:48] <jonasfj> maybe, project.thunderbird.*
> [2017-11-03 19:44:07] <jonasfj> giving thunderbird a dedicated namespace
> [2017-11-03 19:44:13] <tomprince> The tree also has seamonkey and instabird, but they have separate build infra.
> [2017-11-03 19:44:32] <jonasfj> well, project.comm-central is also a decent name
> [2017-11-03 19:44:52] <jonasfj> granted, if this uses all the gekco in-tree decision logic... then it'll have to be paramterized quite a few places
> [2017-11-03 19:45:02] <nalexander> comm-central encodes a particular repo; the equivalent would be mozilla-central, which doesn't scale to mozilla-{beta,release}.
> [2017-11-03 19:45:14] <tomprince> It is using the in-tree decision logic.
> [2017-11-03 19:45:47] <jonasfj> so is it a fork of mozilla-central?
> [2017-11-03 19:45:53] <tomprince> I was thinking about parameterizing the gecko prefix to something like comm
> [2017-11-03 19:46:06] <jonasfj> that could work too

I see artifacts are already stored under `comm.v2.comm-central.latest.thunderbird`[1]. As a consequence, I would suggest:
* clientIds: project/comm/thunderbird/scriptworker/{depsigning,beetmoverworker-dev,balrogworker-dev}
* scopes: replace / by :

That way, we know about the comm project, and the fact that infra is meant for thunderbird (and not another application living under comm). Does this nomenclature sound good to you, guys? 

[1] https://tools.taskcluster.net/index/artifacts/comm.v2.comm-central.latest.thunderbird
[2] https://tools.taskcluster.net/auth/clients/

Comment 4

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> > - we need a new `comm-dep` (or something, tb-dep? cc-dep?) 

Per comment 3, I think "comm-thunderbird-dep" makes things explicit:
* "cc-dep" (i.e. "comm-centra-dep") ties the scriptworker instance to one branch.
* "comm-dep" doesn't say what project within comms is built. Seamonkey may have different instances, for the sake of security,
* "tb-dep" might not show that Thunderbird and Seamonkey share more common configuration than Gecko. I don't have a strong opinion on this one. I don't mind taking it.

What would you prefer, Tom?

Comment 5

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1421062 - Add Thunderbird dev instances {depsigning,balrog,beetmover}_scriptworker

Review commit: https://reviewboard.mozilla.org/r/206302/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/206302/

Comment 6

[Johan Lorenzo [:jlorenzo]]

Attachment 8935401 [details] is WIP. It contains the overall idea of what's needed, but hasn't been tested yet. This requires to agree on the naming convention (comment 3 and 4)

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

I think it makes sense for this to follow the [trust domains](https://reviewboard.mozilla.org/r/200396/diff/1#index_header).  So, s/gecko/comm/.

I'm not terribly inclined to worry about seamonkey.. sorry :(

Comment 8

[Johan Lorenzo [:jlorenzo]]

After a discussion on IRC (#tcmigration), Tom suggested we put releng in the clientId, so that we have 'project/comm/thunderbird/releng/*'. Dustin agreed on it.

Comment 9

[Tom Prince [:tomprince]]

https://github.com/mozilla-releng/scriptworker/pull/171 has code that allows verifying actual comm-central tasks.

Comment 10

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8935401 [details]
Bug 1421062 - Add Thunderbird dev instances {depsigning,balrog,beetmover}_scriptworker

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/206302/diff/1-2/

Comment 11

[Jonas Finnemann Jensen (:jonasfj)]

I agree,

Generally, whatever comes under:
  project/comm/
can be fairly free form. Granted aiming for consistency never hurt anyone.

Comment 12

[Johan Lorenzo [:jlorenzo]]

Status update

Machines[1] were created on Friday. All 3 are attached to my personal puppet environment so that I could test attachment 8935401 [details]. Scriptworker isn't fully configured yet. Nonetheless, puppet is happy and doesn't report any error. Next actions are:
1. Craft an sdist package with [2].
2. Create Taskcluster clientIds with the right scopes. This depends in bug 1424254.
3. Update attachment 8935401 [details] to reflect the client ids/scopes.
4. Update the comm-central to trigger signing, balrog and beetmover jobs.
5. Test the whole chain and repeat steps above for any necessary fix.

[1] tb-balrogworker-dev1.srv.releng.use1.mozilla.com, tb-beetmover-dev1.srv.releng.use1.mozilla.com, tb-depsigning-worker1.srv.releng.use1.mozilla.com
[2] https://github.com/mozilla-releng/scriptworker/pull/170 and https://github.com/mozilla-releng/scriptworker/pull/171

Comment 13

[Tom Prince [:tomprince]]

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=c7a9a2d30b5a10c0e61fa33350b030cca0787ccb

It looks like either isn't using https://github.com/mozilla-releng/scriptworker/pull/171 or not passing the correct cot-product to test.

Testing verification locally, with

  verify_cot --task-type signing --cleanup --cot-product thunderbird Unwpf7TwTiGXonRbXEWIIA 

it appears to verify.

Comment 14

[Johan Lorenzo [:jlorenzo]]

You're right. It was missing `cot_product: thunderbird` in scriptworker.yaml[1]. Next type of error is [2]

[1] https://github.com/mozilla-releng/scriptworker/blob/ef7da86de374cc74c582bbebde56d20ec37922df/scriptworker.yaml.tmpl#L48
[2] https://public-artifacts.taskcluster.net/Unwpf7TwTiGXonRbXEWIIA/8/public/logs/live_backing.log

Comment 15

[Johan Lorenzo [:jlorenzo]]

Attachment: [signingscript] Support tb dep-signing

These changes are needed in signingscript to solve comment 14. Now we're hitting what seems a firewall issue: https://public-artifacts.taskcluster.net/Unwpf7TwTiGXonRbXEWIIA/14/public/logs/live_backing.log

Comment 16

[Johan Lorenzo [:jlorenzo]]

Attachment: [puppet] Allow tb-depsigning-worker1.srv.releng.use1.m.c to reach dep-signing servers

You were right about the firewall, Aki. It seems it's the only place that should be changed. Does that look correct to you?

Comment 17

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8941854 [details] [diff] [review]
[puppet] Allow tb-depsigning-worker1.srv.releng.use1.m.c to reach dep-signing servers

default: https://hg.mozilla.org/build/puppet/rev/9315204f688ac4841feb01504c605d19b905ce25
prod: https://hg.mozilla.org/build/puppet/rev/4c3ab2366d47b2544d2b7a0bc2f653d949cea301

Comment 18

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8935401 [details]
Bug 1421062 - Add Thunderbird dev instances {depsigning,balrog,beetmover}_scriptworker

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/206302/diff/2-3/

Comment 19

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8935401 [details]
Bug 1421062 - Add Thunderbird dev instances {depsigning,balrog,beetmover}_scriptworker

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/206302/diff/3-4/

Comment 20

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8935401 [details]
Bug 1421062 - Add Thunderbird dev instances {depsigning,balrog,beetmover}_scriptworker

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/206302/diff/4-5/

Comment 21

[Aki Sasaki (not active)]

Some issues:

- tb-depsigning-worker1 appears to have 3 ips? 109 appears to be accurate.

(swenv) akimoz: /src/tc [13:44:36]
10935$ nslookup tb-depsigning-worker1
Server:         10.22.72.136
Address:        10.22.72.136#53

Name:   tb-depsigning-worker1.srv.releng.use1.mozilla.com
Address: 10.134.30.183
Name:   tb-depsigning-worker1.srv.releng.use1.mozilla.com
Address: 10.134.30.109
Name:   tb-depsigning-worker1.srv.releng.use1.mozilla.com
Address: 10.134.30.152

- I modified https://tools.taskcluster.net/auth/clients/project%2Freleng%2Fscriptworker%2Fdepsigning to include queue:worker-id:signing-linux-v1/tb-depsigning-*

- now it's complaining about taskcluster_certificate_prefix not being set in context.config. I'm wondering if this used to be pinned to your puppet env, and isn't anymore?

Comment 22

[Aki Sasaki (not active)]

tb-depsigning-worker1 started taking autoland signing tasks:
https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&fromchange=df6485ef321aa3705c92f8f5b2498037b32e44f0&selectedJob=158064743&filter-searchStr=build-signing

Re-pinned to your puppet env. Going to rerun the failed tasks.

Comment 23

[Tom Prince [:tomprince]]

It looks like it is now taking TB jobs again, but not using the tb-cot branch: https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=32a25748090a38b097aa07acb39cc49227389380&filter-searchStr=sign

Comment 24

[Intermittent Failures Robot]

4 failures in 701 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 3
* mozilla-inbound: 1

Platform breakdown:
* windows2012-32: 3
* windows2012-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421062&startday=2018-01-22&endday=2018-01-28&tree=all

Comment 25

[Johan Lorenzo [:jlorenzo]]

Attachment: [puppet] Define and default cot_product to Firefox

Comment 26

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8947164 [details] [diff] [review]
[puppet] Define and default cot_product to Firefox

https://hg.mozilla.org/build/puppet/rev/1639617bee6b43fa99c366567a060fe0fe803043

Comment 27

[Johan Lorenzo [:jlorenzo]]

Attachment: [gecko] Change comm signing format scopes

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/fdecefbc7f55
Make signing format dependent of scope-prefix r=tomprince

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ae73697f72b7
fixup linter issue r=bustage

Comment 30

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/fdecefbc7f55
https://hg.mozilla.org/mozilla-central/rev/ae73697f72b7

Comment 31

[Johan Lorenzo [:jlorenzo]]

Attachment: [puppet] Add Thunderbird depsigning_scriptworker

Comment 32

[Simon Fraser [:sfraser] ⌚️GMT]

Comment on attachment 8947767 [details] [diff] [review]
[puppet] Add Thunderbird depsigning_scriptworker

Good call anchoring the regular expressions! 

I worry a little bit that having both worker_type and worker_group with the same name, 'signing-linux-v1', may cause confusion later, but it's probably going to be fine. 

Go for it!

Comment 33

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8947767 [details] [diff] [review]
[puppet] Add Thunderbird depsigning_scriptworker

(In reply to Simon Fraser [:sfraser] ⌚️GMT from comment #32)
> I worry a little bit that having both worker_type and worker_group with the
> same name, 'signing-linux-v1', may cause confusion later, but it's probably
> going to be fine. 

I agree. I left the group names as is, because it would require us to update scopes on roles. I wanted to avoid such a burden. I'll file a follow up to fix this.


Landed on:
* default: https://hg.mozilla.org/build/puppet/rev/175f43ac030d96bfcc5f2dc53523c99585138524
* production https://hg.mozilla.org/build/puppet/rev/4216eaff1a9c66564ff8e162567ff0fb2bcd3147

Comment 34

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8947539 [details] [diff] [review]
[gecko] Change comm signing format scopes

See comment 30

Comment 36

[Tom Prince [:tomprince]]

It appears that mar_sha384 signing is working on the depsigning worker. See for example https://tools.taskcluster.net/groups/RML7xz2LTMm03D7UjJ3U4Q/tasks/bcuzWmZDRfyqZr5gHzFA0g/runs/0/logs/public%2Flogs%2Flive_backing.log#L21

Comment 37

[Phabricator Automation]

Attachment: Bug 1421062: Use correct scope prefixes in repackage signing tasks; r?jlorenzo

Comment 38

[Phabricator Automation]

Comment on attachment 8952286 [details]
Bug 1421062: Use correct scope prefixes in repackage signing tasks; r?jlorenzo

Johan Lorenzo [:jlorenzo] has approved the revision.

https://phabricator.services.mozilla.com/D611

Comment 39

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e3ad5ddda361
Use correct scope prefixes in repackage signing tasks; r=jlorenzo

Comment 40

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/e3ad5ddda361

Comment 41

[Tom Prince [:tomprince]]

Can I get

- project:comm:thunderbird:releng:beetmover:bucket:dep
- project:comm:thunderbird:releng:beetmover:action:*

added to https://tools.taskcluster.net/auth/roles/moz-tree%3Alevel%3A1:comm

Comment 42

[Tom Prince [:tomprince]]

Attachment: [beetmoverscript] Add support for the thunderbird scope prefixes.

Comment 43

[Johan Lorenzo [:jlorenzo]]

Attachment: Bug 1421062 - Spin up tb-beetmoverworker-dev

Review commit: https://reviewboard.mozilla.org/r/225416/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/225416/

Comment 44

[Tom Prince [:tomprince]]

Johan, Bug 1443588 has patches that I think will make it pick up the manifest template based on appname, rather than indirecting through https://github.com/mozilla-releng/beetmoverscript/pull/105/files#diff-67ea3dc94b3e0a4aaeb35e229f9254bdL39 and the config file.

Comment 45

[Johan Lorenzo [:jlorenzo]]

Server was deployed with patches in Bug 1443588, 2 days ago by 8pm UTC.

Comment 46

[Phabricator Automation]

Attachment: Bug 1421062: Configure beetmover workers via global taskgraph config; r?aki

Comment 47

[Phabricator Automation]

Attachment: Bug 1421062: Use nightly beetmover config by default on try; r?aki

Comment 48

[Phabricator Automation]

Attachment: Bug 1421062: Use `Thunderbird` as the product in comm-central beetmover tasks; r?aki

Comment 49

[Phabricator Automation]

Comment on attachment 8960748 [details]
Bug 1421062: Configure beetmover workers via global taskgraph config; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D781

Comment 50

[Phabricator Automation]

Comment on attachment 8960749 [details]
Bug 1421062: Use nightly beetmover config by default on try; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D782

Comment 51

[Phabricator Automation]

Comment on attachment 8960750 [details]
Bug 1421062: Use `Thunderbird` as the product in comm-central beetmover tasks; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D783

Comment 52

[Tom Prince [:tomprince]]

Attachment: Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review commit: https://reviewboard.mozilla.org/r/232312/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/232312/

Comment 53

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/bcc822db5775
Configure beetmover workers via global taskgraph config; r=aki
https://hg.mozilla.org/integration/mozilla-inbound/rev/cbab4fd64be0
Use nightly beetmover config by default on try; r=aki
https://hg.mozilla.org/integration/mozilla-inbound/rev/147d5a5eb3ac
Use `Thunderbird` as the product in comm-central beetmover tasks; r=aki

Comment 54

[Tom Prince [:tomprince]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/232312/diff/1-2/

Comment 55

[Tom Prince [:tomprince]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/232312/diff/2-3/

Comment 56

[Tom Prince [:tomprince]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/232312/diff/3-4/

Comment 57

[Tom Prince [:tomprince]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/232312/diff/4-5/

Comment 58

[Tom Prince [:tomprince]]

Attachment: Bug 1421062: Remove unused password for staing balrog instance;

Review commit: https://reviewboard.mozilla.org/r/233938/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/233938/

Comment 59

[Tom Prince [:tomprince]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/232312/diff/5-6/

Comment 60

[Phabricator Automation]

Attachment: Bug 1421062: Allow configuring the balrogworker to use per-project; r?aki

Comment 61

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8965221 [details]
Bug 1421062: Remove unused password for staing balrog instance;

https://reviewboard.mozilla.org/r/233938/#review239542

Nice clean up! Thank you for taking care of it.

Comment 62

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8963701 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker;

https://reviewboard.mozilla.org/r/232312/#review239544

LGTM! That reminds me I haven't landed the beetmover part. I'll take care of it after you landed yours. This should avoid rebases. I probably have to rebase mine anyway.

Could you also change the taskcluster client, whenever you have a chance? This doesn't block the landing and can be done in a follow up.

::: modules/balrog_scriptworker/manifests/settings.pp:56
(Diff revision 6)
> +            balrog_password => '',
> +            balrog_api_root => 'https://balrog-admin.stage.mozaws.net/api',
> +
> +            dummy => false,
> +            tools_repo => 'https://hg.mozilla.org/build/tools',
> +            taskcluster_client_id => 'mozilla-auth0/ad|Mozilla-LDAP|tprince/tb-balrog',

Consitency: you may want to create a client named `project/comm/thunderbird/releng/scriptworker/balrog/dev` so that it matches the existing clients:
https://tools.taskcluster.net/auth/clients/project%2Fcomm%2Fthunderbird%2Freleng%2Fscriptworker%2Fbeetmover%2Fdev

If you don't have the rights to create a client under that namespace, you can reach out to :dustin.

Comment 63

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/bcc822db5775
https://hg.mozilla.org/mozilla-central/rev/cbab4fd64be0
https://hg.mozilla.org/mozilla-central/rev/147d5a5eb3ac

Comment 64

[Phabricator Automation]

Comment on attachment 8965223 [details]
Bug 1421062: Allow configuring the balrogworker to use per-project; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D850

Comment 65

[Phabricator Automation]

Comment on attachment 8965223 [details]
Bug 1421062: Allow configuring the balrogworker to use per-project; r?aki

Aki Sasaki [:aki] has been removed from the revision.

https://phabricator.services.mozilla.com/D850

Comment 66

[Phabricator Automation]

Comment on attachment 8965223 [details]
Bug 1421062: Allow configuring the balrogworker to use per-project; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D850

Comment 67

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9c941daebe9f
Allow configuring the balrogworker to use per-project; r=aki

Comment 68

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/9c941daebe9f

Comment 69

[Phabricator Automation]

Attachment: Bug 1421062: Add production thunderbird signing workers; r?aki

Comment 70

[Phabricator Automation]

Attachment: Bug 1421062: Enable production signing on comm-* branches; r?aki

Comment 71

[Phabricator Automation]

Comment on attachment 8966732 [details]
Bug 1421062: Enable production signing on comm-* branches; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D904

Comment 72

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/8e414afd217b
Enable production signing on comm-* branches; r=aki a=Aryx

Comment 73

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/c33356fb03ee
Add production signing worker configuration; r=me

Comment 74

[Phabricator Automation]

Comment on attachment 8966646 [details]
Bug 1421062: Add production thunderbird signing workers; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D902

Comment 75

[Phabricator Automation]

Attachment: Bug 1421062: Allow tb-signing-*.srv.releng.us?1.m.c to reach signing servers; r?aki

Comment 76

[Phabricator Automation]

Comment on attachment 8967040 [details]
Bug 1421062: Allow tb-signing-*.srv.releng.us?1.m.c to reach signing servers; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D909

Comment 77

[Phabricator Automation]

Attachment: Bug 1421062: Enable production beetmoving on comm-central branches; r=aki

Comment 78

[Phabricator Automation]

Comment on attachment 8971712 [details]
Bug 1421062: Enable production beetmoving on comm-central branches; r=aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1060

Comment 79

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/a90bf0a31015
Enable production beetmoving on comm-central branches; r=aki a=me

Comment 80

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/8256eb2caf7b
Add nightly beetmover worker configuration; r=me

Comment 81

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e887e6b063d9
Use `nightly-desktop` as cron task name; r=me

Comment 82

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e2958a1e37ec
Fix nightly beetmover worker configuration; r=me CLOSED TREE DONTBUILD

Comment 83

[Phabricator Automation]

Attachment: Bug 1421062, 1456234: Update beetmover to 7.1.1; r?aki

Comment 84

[Phabricator Automation]

Attachment: Bug 1421062: Prepare beetmover config for thunderbird workers; r=aki

Comment 85

[Phabricator Automation]

Attachment: Bug 1421062: Spin up tb-beetmoverworker-dev; r?aki

Comment 86

[Phabricator Automation]

Attachment: Bug 1421062: Spin up tb-beetmover prod; r=aki

Comment 87

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/2bbe101d3eda
Enable production nightly balrog update on comm-central branches; r=me a=me

Comment 88

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f3cfb4e74573
Add nightly balrog worker configuration; r=me

Comment 89

[Phabricator Automation]

Comment on attachment 8972035 [details]
Bug 1421062, 1456234: Update beetmover to 7.1.1; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1064

Comment 90

[Phabricator Automation]

Comment on attachment 8972037 [details]
Bug 1421062: Spin up tb-beetmoverworker-dev; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1066

Comment 91

[Phabricator Automation]

Comment on attachment 8972036 [details]
Bug 1421062: Prepare beetmover config for thunderbird workers; r=aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1065

Comment 92

[Phabricator Automation]

Comment on attachment 8972038 [details]
Bug 1421062: Spin up tb-beetmover prod; r=aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1067

Comment 93

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/build/puppet/rev/6346dfc55396
1456234: Update beetmover to 7.1.1; r=aki
https://hg.mozilla.org/build/puppet/rev/82e4f479623f
Prepare beetmover config for thunderbird workers; r=aki
https://hg.mozilla.org/build/puppet/rev/6765e2ff927c
Spin up tb-beetmoverworker-dev; r=aki
https://hg.mozilla.org/build/puppet/rev/77fcae37bae6
Spin up tb-beetmover prod; r=aki

Comment 94

[Phabricator Automation]

Attachment: Bug 1421062: Update to balrogscript 3.1.0; r?aki

Comment 95

[Phabricator Automation]

Attachment: Bug 1421062: Spin up tb-balrog-dev scriptworker; r?aki

MozReview-Commit-ID: 2tq83FdrKBY

Comment 96

[Phabricator Automation]

Attachment: Bug 1421062: Spin up production tb-balrog scriptworker; r?aki

Comment 97

[Phabricator Automation]

Comment on attachment 8972654 [details]
Bug 1421062: Update to balrogscript 3.1.0; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1105

Comment 98

[Phabricator Automation]

Comment on attachment 8972656 [details]
Bug 1421062: Spin up production tb-balrog scriptworker; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1107

Comment 99

[Phabricator Automation]

Comment on attachment 8972655 [details]
Bug 1421062: Spin up tb-balrog-dev scriptworker; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1106

Comment 100

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/build/puppet/rev/d3d00b204d94
Update to balrogscript 3.1.0; r=aki
https://hg.mozilla.org/build/puppet/rev/42bb62c79001
Spin up tb-balrog-dev scriptworker; r=aki
https://hg.mozilla.org/build/puppet/rev/4a96bff6f6b4
Spin up production tb-balrog scriptworker; r=aki

Comment 101

[Phabricator Automation]

Attachment: Bug 1421062: [release] Update secondary balrog tasks to get scopes based on project; r?aki

This is a follow up to 9c941daebe9fb3e79066ee4def16ed5ce0eb10a9

Comment 102

[Phabricator Automation]

Comment on attachment 8973299 [details]
Bug 1421062: [release] Update secondary balrog tasks to get scopes based on project; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1140

Comment 103

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6ca277616782
[release] Update secondary balrog tasks to get scopes based on project; r=aki

Comment 104

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/6ca277616782

Comment 105

[Phabricator Automation]

Attachment: Bug 1421062: [release] Use configured scriptworker scope prefix for checksum signing; r?aki

Comment 106

[Phabricator Automation]

Comment on attachment 8975925 [details]
Bug 1421062: [release] Use configured scriptworker scope prefix for checksum signing; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1291

Comment 107

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/61730225e15b
[release] Use configured scriptworker scope prefix for checksum signing; r=aki

Comment 108

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/61730225e15b

Comment 109

[Phabricator Automation]

Attachment: Bug 1421062: Update beetmover to 7.2.3; r?aki

- Fix capitalization of `Thunderbird` in windows installer and dmg files.

Comment 110

[Phabricator Automation]

Comment on attachment 8980630 [details]
Bug 1421062: Update beetmover to 7.2.3; r?aki

Aki Sasaki [:aki] has approved the revision.

https://phabricator.services.mozilla.com/D1385

Comment 111

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/build/puppet/rev/b68369cbad34
Update beetmover to 7.2.3; r=aki

Comment 112

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/d7001cfd3e3c
[release] Add release balrog and beetmover scope configuration; r=me
https://hg.mozilla.org/comm-central/rev/4b3f7cefa2b2
[release] Fix typo'd worker names in release tasks; r=me

Comment 113

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/c26c760692be
[release] Add esr60 balrog scopes; r=me

Comment 114

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e4a6945f1aa0
[release] Enable production scriptworkers on comm-beta and comm-esr60; r=me

Comment 115

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/e4a6945f1aa0