1166380 - Add an (empty) "taskcluster" build pool

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

The options for build pools are currently "staging" and "production":
  https://github.com/mozilla/build-mozharness/blob/master/configs/builds/build_pool_specifics.py

Taskcluster is basically a new pool.  Let's add a new clause to that file omitting information that's not useful from taskcluster.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: bz://1166380/dustin

/r/9065 - Bug 1166380: add 'taskcluster' build pool; r=mshal

Pull down this commit:

hg pull -r 6e253a21f3ccc722d668487b5334c0d213ec0e01 https://reviewboard-hg.mozilla.org/build-mozharness

Comment 3

[Michael Shal [:mshal]]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

https://reviewboard.mozilla.org/r/9063/#review7697

Ship It!

Comment 4

[Michael Shal [:mshal]]

Actually, does 'taskcluster' need to be added to buildbase.py:BuildOptionParser where it sets build_pools? Not sure if you need that for your use-case.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Good point - I'll re-draft.

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

/r/9065 - Bug 1166380: add 'taskcluster' build pool; r=mshal
/r/9137 - Bug 1166380: don't hard-code build pool names in buildbase.py

Pull down these commits:

hg pull -r d84645ed8805550daaaef9f40d509f1af0e6b287 https://reviewboard-hg.mozilla.org/build-mozharness

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

https://reviewboard.mozilla.org/r/9063/#review7835

Ship It!

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

remote:   https://hg.mozilla.org/build/mozharness/rev/2e2809fb05a9
remote:   https://hg.mozilla.org/build/mozharness/rev/af1eee401f7f

Comment 10

[Rail Aliiev [:rail]]

mozharness production tag moved to: https://hg.mozilla.org/build/mozharness/rev/10ea0859a820

Comment 11

[Rail Aliiev [:rail]]

mozharness production tag moved to: https://hg.mozilla.org/build/mozharness/rev/10ea0859a820

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

:mshal noted that there are two places that still reference build_pool.

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

testing the fix:
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=07a51a6f4c36

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

/r/9065 - Bug 1166380: fix remaining build_pools occurrences; r=mshal

Pull down this commit:

hg pull -r b3f509ec95ee0a1340f458f340686d8428592b6f https://reviewboard-hg.mozilla.org/build-mozharness

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

Using the right mozharness repo (hopefully): https://treeherder.mozilla.org/#/jobs?repo=try&revision=eb554663cff4

Comment 16

[Michael Shal [:mshal]]

Comment on attachment 8607679 [details]
MozReview Request: bz://1166380/dustin

https://reviewboard.mozilla.org/r/9063/#review7853

Looks good, hopefully try is happy...

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

After several failures to specify the mozharness repo correctly, yes, try is reasonably happy (linux and android so far)
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=eb451f69d5b7

remote:   https://hg.mozilla.org/build/mozharness/rev/0e6e9348c2ed
..and grafted to production..
remote:   https://hg.mozilla.org/build/mozharness/rev/2ae240c4f588

Comment 18

[Dustin J. Mitchell [:dustin] (he/him)]

There are places in buildbase that assume there's a symbol_server_host (and probably other keys).

Traceback (most recent call last):                                                                                      
  File "./mozharness/scripts/fx_desktop_build.py", line 166, in <module>                                                
    fx_desktop_build = FxDesktopBuild()                                                                                 
  File "./mozharness/scripts/fx_desktop_build.py", line 98, in __init__                                                 
    super(FxDesktopBuild, self).__init__(**buildscript_kwargs)                                                          
  File "/home/worker/mozharness/mozharness/mozilla/building/buildbase.py", line 543, in __init__                        
    self.query_build_env()                                                                                              
  File "/home/worker/mozharness/mozharness/mozilla/building/buildbase.py", line 785, in query_build_env                 
    replace_dict.update({"symbol_server_host": c['symbol_server_host']})                                                
KeyError: 'symbol_server_host'

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: dont-fail.patch

I'll test this in try at the same time.

Comment 20

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8610753 [details] [diff] [review]
dont-fail.patch

Never mind, that very much did not work.  I'll just add the values to build_pool_specifics.py

Comment 21

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: symbol-and-stage-for-tc.patch

This has worked in try

Comment 22

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8610825 [details] [diff] [review]
symbol-and-stage-for-tc.patch

remote:   https://hg.mozilla.org/build/mozharness/rev/92af19b51890

Comment 24

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1166380: fix remaining build_pools occurrences; r=mshal

Comment 25

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1166380: don't hard-code build pool names in buildbase.py