Closed Bug 1307771 Opened 3 years ago Closed 3 years ago

rename opt-* and dbg-* and android-* workerTypes to gecko-L-b-*

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla52

People

(Reporter: dustin, Assigned: panchalhaard, Mentored)

References

Details

(Keywords: good-first-bug)

Attachments

(2 files)

Some of these can probably be combined with gecko-L-b-linux, with a corresponding bump in its capacity, but `gecko-L-b-cross-osx` et al. is OK too.
Hello! I'm new to bugzilla and would like to work on this issue. It would be great if I could get some tips and more detail on the same.
Same. I would like to work on some of these bugs. Need some info on where to start, how to get started, that sort of thing.
Hi!  This might actually be a fun bug to work on together, as there's a fair bit to learn and you might help one another when you get stuck.

You can see the list of workerTypes we have at
  https://tools.taskcluster.net/aws-provisioner/

For a bit of background on what a workerType is, you may want to read through
  https://docs.taskcluster.net/manual
You can probably skip most of the first chapter about API access.

You'll see that some of our workerTypes have nice consistent names starting with gecko-.., while others are basically made up (opt-this, dbg-that, etc.).  The idea is to use consistent names for everything.

The things that *use* these worker types are tasks created from the Firefox source code (Gecko).  You'll find some documentation of that at
  https://gecko.readthedocs.io/en/latest/taskcluster/taskcluster

And in particular you'll find references to the workerTypes under taskcluster/ in the gecko source code.

The process for making these changes is this:

 * create workerTypes with the correct names (I can do this, as it involves handling some passwords)
 * find and change the references in the source code to point to workerTypes with the correct names
   * write a patch
   * get review
   * land the patch
 * wait for those changes to be distributed everywhere so that the old workerTypes are no longer used (could be a few months)
 * remove the old workerTypes

Once you're familiar with this code, there are lots of other things to work on, too :)
Thanks Dustin! I'll start working on it right away.
Haard, how is it going?  Have you had a chance to talk with Matt?
I have gone through the code under taskcluster/ and found references to opt-*, dbg-* and android-*. But in order to change that to gecko-* correspondingly, we would have to create those worker types in https://tools.taskcluster.net/aws-provisioner/#gecko-t-win7-32/ , right?
Yes, that's right.  I can create them by copying the existing definitions.  Can you give me a list of suggested workerType names, and what type to copy them from?
Blocks: 1220686
Here you go, Dustin:
Previous       Final
dbg-linux32 -> gecko-L-b-linux32
dbg-linux64 -> gecko-L-b-linux64
dbg-macosx64 -> gecko-L-b-macosx64
opt-linux32 -> gecko-L-b-linux32
opt-linux64 -> gecko-L-b-linux64
opt-macosx64 -> gecko-L-b-macosx64
android-api-15 -> gecko-L-b-android
Should there be any more?
Sounds good.  Looking in the source, only a few are still used:

dbg-linux32
dbg-linux64
dbg-macosx64
taskcluster/ci/build/macosx.yml:    worker-type: aws-provisioner-v1/dbg-macosx64
opt-linux32
opt-linux64
opt-macosx64
taskcluster/ci/static-analysis/kind.yml:        worker-type: aws-provisioner-v1/opt-macosx64
taskcluster/ci/build/macosx.yml:    worker-type: aws-provisioner-v1/opt-macosx64
android-api-15
taskcluster/ci/build/android.yml:    worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/build/android.yml:    worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/build/android.yml:    worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/build/android.yml:    worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/build/android.yml:    worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/android-stuff/kind.yml:        worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/android-stuff/kind.yml:        worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/android-stuff/kind.yml:        worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/android-stuff/kind.yml:        worker-type: aws-provisioner-v1/android-api-15
taskcluster/ci/l10n/kind.yml:        worker-type: aws-provisioner-v1/android-api-15

As you have done, we can combine the dbg- and opt-macosx64, so we just need gecko-L-b-macosx64 and gecko-L-b-android.  I've created those six (three levels for each).

So all that remains to be done is to change the references that appear above to point to the new names.
I found out the references to the old names, but I not sure which 'level' of the worker-type to use, as I do not about the level concept, and what the difference is between two different levels. Where can I find more information on this?
The levels are outlined here:
  https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/

We handle the levels by substituting `config.params['level']`, handled here
  https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/task.py#484
so you can just write
  worker-type: aws-provisioner/gecko-{level}-b-android
and you should get the correct output.
Attached patch renaming.patchSplinter Review
Please review the patch!
Comment on attachment 8804197 [details] [diff] [review]
renaming.patch

Please set the r? flag when requesting a review :)
Attachment #8804197 - Flags: review?(dustin)
Attachment #8804197 - Flags: feedback?(dustin)
Comment on attachment 8804197 [details] [diff] [review]
renaming.patch

Review of attachment 8804197 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good, except that you've re-introduced the deleted android-partner.yml.  Can you remove that file and attach a new patch?
Attachment #8804197 - Flags: review?(dustin)
Attachment #8804197 - Flags: review+
Attachment #8804197 - Flags: feedback?(dustin)
Sure!
I hope this one is correct :)
Attachment #8804811 - Flags: review?(dustin)
Comment on attachment 8804811 [details] [diff] [review]
Removed android-partner.yml

Review of attachment 8804811 [details] [diff] [review]:
-----------------------------------------------------------------

I'll test this in try..
Attachment #8804811 - Flags: review?(dustin) → review+
Assignee: nobody → panchalhaard
Will the patch be applied in the original source code or I'll have to do it myself?
Looks like the try test was good, so I'll land this now.
https://hg.mozilla.org/integration/mozilla-inbound/rev/c1d12c6ae210912b1e5005ff1cd952cdc605e593
Bug 1307771: rename opt-* and dbg-* and android-* workerTypes to gecko-L-b-*; r=dustin
https://hg.mozilla.org/mozilla-central/rev/c1d12c6ae210
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla52
Depends on: 1315988
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.