Last Comment Bug 803248 - buildbot config changes to support panda_android*
: buildbot config changes to support panda_android*
Status: RESOLVED FIXED
[reit-panda]
:
Product: Release Engineering
Classification: Other
Component: Platform Support (show other bugs)
: other
: x86 Mac OS X
: -- normal (vote)
: ---
Assigned To: Kim Moir [:kmoir]
: Chris Cooper [:coop]
Mentors:
Depends on: 800047 808951
Blocks: android_4.0_testing 799698 820827
  Show dependency treegraph
 
Reported: 2012-10-18 13:02 PDT by Kim Moir [:kmoir]
Modified: 2013-08-12 21:55 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
buildbotcustom changes to support android on pandas (2.68 KB, patch)
2012-10-26 14:01 PDT, Kim Moir [:kmoir]
aki: review+
kmoir: checked‑in+
Details | Diff | Review
buildbot config changes to support android on pandas (3.96 KB, text/plain)
2012-10-26 14:05 PDT, Kim Moir [:kmoir]
no flags Details
buildbot config to enable tests for android + panda in cedar branch (4.83 KB, patch)
2012-11-01 11:18 PDT, Kim Moir [:kmoir]
aki: review+
Details | Diff | Review
buildbot config to enable tests for android + panda in cedar branch (4.82 KB, patch)
2012-11-02 11:26 PDT, Kim Moir [:kmoir]
bugspam.Callek: review+
Details | Diff | Review
fixed to exclude running non-cedar branch builds on android pandas (6.20 KB, patch)
2012-11-02 12:54 PDT, Kim Moir [:kmoir]
aki: review+
kmoir: checked‑in+
Details | Diff | Review
patch to revert addition of esr17 (1.36 KB, patch)
2012-11-05 12:10 PST, Kim Moir [:kmoir]
bugspam.Callek: review+
kmoir: checked‑in+
Details | Diff | Review
fix pandas (3.99 KB, patch)
2012-11-05 19:06 PST, Aki Sasaki [:aki]
bugspam.Callek: review+
Details | Diff | Review
patch to add 66 pandas to production + enable bm44 (11.14 KB, patch)
2012-12-12 13:52 PST, Kim Moir [:kmoir]
bugspam.Callek: review+
kmoir: checked‑in+
Details | Diff | Review
expand range of pandas in prod + enable builds on m-c (2.51 KB, patch)
2012-12-13 11:35 PST, Kim Moir [:kmoir]
bugspam.Callek: review+
kmoir: checked‑in+
Details | Diff | Review
patch to fix how foopies line up (11.99 KB, patch)
2012-12-14 08:27 PST, Kim Moir [:kmoir]
bugspam.Callek: review+
kmoir: checked‑in+
Details | Diff | Review
add new pandas in racks 7 and 8 as requested in bug 824766 (27.79 KB, patch)
2013-01-03 07:47 PST, Kim Moir [:kmoir]
armenzg: review+
kmoir: checked‑in+
Details | Diff | Review

Description Kim Moir [:kmoir] 2012-10-18 13:02:34 PDT

    
Comment 1 Hal Wine [:hwine] (use NI) 2012-10-26 12:57:27 PDT
This bug is most closely associated with putting the first pandas-for-android in production (they are running in staging now). Adding a few interested parties to the cc list for "up to the minute" status.
Comment 2 Kim Moir [:kmoir] 2012-10-26 14:01:24 PDT
Created attachment 675683 [details] [diff] [review]
buildbotcustom changes to support android on pandas
Comment 3 Kim Moir [:kmoir] 2012-10-26 14:05:10 PDT
Created attachment 675685 [details]
buildbot config changes to support android on pandas

Not sure what slaves will be allocated to staging yet.  Also, I'm unsure of how to only enable a tests on the cedar branch.
Comment 4 Armen Zambrano [:armenzg] - Engineering productivity 2012-10-29 06:15:19 PDT
An example that might help is Win7x64 which is only enabled on mozilla-central:
http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/config.py#l741
   747             'enable_opt_unittests': False,
   749             'enable_debug_unittests': False,

http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/config.py#l955
   956 # XXX hacking warning - this code could get out of date easily
   957 BRANCHES['mozilla-central']['platforms']['win64']['enable_opt_unittests'] = True
   958 for suite in SUITES.keys():
   959     options = SUITES[suite]['options']
   960     if options[1] == ALL_PLATFORMS:
   961         options = (options[0], ALL_PLATFORMS + PLATFORMS['win64']['slave_platforms'])
   962     if options[1] == NO_MAC:
   963         options = (options[0], NO_MAC + PLATFORMS['win64']['slave_platforms'])
   964     if not SUITES[suite]['enable_by_default']:
   965         # Suites that are turned off by default
   966         BRANCHES['mozilla-central'][suite + '_tests'] = (0, True) + options
   967     else:
   968         # Suites that are turned on by default
   969         BRANCHES['mozilla-central'][suite + '_tests'] = (1, True) + options

I hope it helps; even though "cedar" additions are best suited inside of project_branches.py:
http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla/project_branches.py#l238

It won't be an easy patch from my experience.
Comment 5 Kim Moir [:kmoir] 2012-11-01 11:18:17 PDT
Created attachment 677493 [details] [diff] [review]
buildbot config to enable tests for android + panda in cedar branch
Comment 6 Aki Sasaki [:aki] 2012-11-01 12:18:48 PDT
Comment on attachment 677493 [details] [diff] [review]
buildbot config to enable tests for android + panda in cedar branch

> -PLATFORMS['android']['slave_platforms'] = ['tegra_android']
> +PLATFORMS['android']['slave_platforms'] = ['tegra_android','panda_android']

Style nit: please put a space after the comma.

> +        [('panda-%04i' % x, {'http_port': '30%03i' % x, 'ssl_port': '31%03i' % x}) for x in range(22,34) + range(46,81)]
> +    ),             

Pet peeve: please remove trailing whitespace.
Comment 7 Justin Wood (:Callek) 2012-11-01 13:45:12 PDT
Comment on attachment 677493 [details] [diff] [review]
buildbot config to enable tests for android + panda in cedar branch

Review of attachment 677493 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +161,4 @@
>  PLATFORMS['android']['env_name'] = 'android-perf'
>  PLATFORMS['android']['is_mobile'] = True
>  PLATFORMS['android']['tegra_android'] = {'name': "Android Tegra 250"}
> +PLATFORMS['android']['panda_android'] = {'name': "Android Panda"}

Bikeshed-NIT: I'm more a fan of "Android ICS Panda" or "Android ICS Opt" or something

But this bikeshed should not block landing!

::: mozilla-tests/production_config.py
@@ +15,5 @@
>          if x not in range(122,129) + [30,31,33,34,43,44,49,65,69,77,131,137,143,147,\
>              153,156,161,175,176,180,184,185,186,193,197,198,202,203,204,205,222,224,\
>              226,239,241,268,275,289,291,292,301]]), # decommissioned tegras
> +     'panda_android': dict(
> +        [('panda-%04i' % x, {'http_port': '30%03i' % x, 'ssl_port': '31%03i' % x}) for x in range(22,34) + range(46,81)]

nit: *when* we get into 4 digit panda numbers here, these port specs will fail to work properly, we should plan to find a good solution there. but WFM for now
Comment 8 Ed Morley [:emorley] 2012-11-01 14:22:47 PDT
(In reply to Justin Wood (:Callek) from comment #7)
> >  PLATFORMS['android']['tegra_android'] = {'name': "Android Tegra 250"}
> > +PLATFORMS['android']['panda_android'] = {'name': "Android Panda"}
> 
> Bikeshed-NIT: I'm more a fan of "Android ICS Panda" or "Android ICS Opt" or
> something

The proposal Clint made in bug 800047 was: "Android 4.0 Panda" (we would later also change "Android Tegra 250" to "Android 2.2 Tegra 250"). I tend to get confused with the codenames, so would prefer the version.

This also must have something between the "Android" and "Panda", or we'll have to make the regexes a bit hacky in TBPL to avoid causing false-positives against b2g panda or android 2.2.
Comment 9 Aki Sasaki [:aki] 2012-11-01 14:25:00 PDT
Ok.

I think we're ok to land anyway, as long as we fix up naming while we're still on Cedar.  Correct?
Comment 10 Justin Wood (:Callek) 2012-11-01 14:30:59 PDT
(In reply to Aki Sasaki [:aki] from comment #9)
> Ok.
> 
> I think we're ok to land anyway, as long as we fix up naming while we're
> still on Cedar.  Correct?

correct, naming for a cedar-landing is not blocking, but still a nice-to-have. We'll discuss it amongst ourselves [releng] tomorrow with myself, kim, armen and coop, and if we have an agreement we'll land this with that, otherwise we'll land this as-is and adjust later without worry for TBPL.
Comment 11 Kim Moir [:kmoir] 2012-11-02 11:26:07 PDT
Created attachment 677833 [details] [diff] [review]
buildbot config to enable tests for android + panda in cedar branch

updated with correct name for pandas + excluded devices from chassis 2 (jmaher testing) and 6 (not working) from production
Comment 12 Justin Wood (:Callek) 2012-11-02 11:50:32 PDT
Comment on attachment 677833 [details] [diff] [review]
buildbot config to enable tests for android + panda in cedar branch

r+ per interdiff
Comment 13 Kim Moir [:kmoir] 2012-11-02 12:54:41 PDT
Created attachment 677855 [details] [diff] [review]
fixed to exclude running non-cedar branch builds on android pandas

I forgot to explicitly exclude builds from running in non-cedar branches in the previous patch.  Here's the interdiff

> @@ -1028,16 +1030,22 @@ for branch in ('mozilla-central', 'mozil
>                  BRANCHES[branch]['platforms'][pf][slave_pf]['debug_unittest_suites'] += [('jetpack', ['jetpack'])]
>                  continue
>              if pf == "macosx64" and slave_pf == "leopard":
>                  BRANCHES[branch]['platforms'][pf][slave_pf]['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
>                  continue
>              BRANCHES[branch]['platforms'][pf][slave_pf]['opt_unittest_suites'] += [('jetpack', ['jetpack'])]
>              BRANCHES[branch]['platforms'][pf][slave_pf]['debug_unittest_suites'] += [('jetpack', ['jetpack'])]
>  
> +#exclude android builds from running on non-cedar branches on pandas
> +for branch in ['mozilla-aurora', 'mozilla-beta', 'mozilla-release', 'mozilla-esr10'] + ACTIVE_PROJECT_BRANCHES:
> +    if 'android' in BRANCHES[branch]['platforms'] and branch != "cedar" :
> +        del BRANCHES[branch]['platforms']['android']['panda_android']
> +        BRANCHES[branch]['platforms']['android']['slave_platforms'] = ['tegra_android']
> +
>  # Let's load Marionette for the following branches:
>  for branch in ('mozilla-central', 'mozilla-inbound', 'try', 'fx-team', 'services-central', ):
>      for pf in PLATFORMS:
>          if 'android' in pf:
>              # this is just for desktop Firefox
>              continue
>          config_file = "marionette/prod_config.py"
>          if pf.startswith('win'):
Comment 14 Aki Sasaki [:aki] 2012-11-02 13:04:50 PDT
Comment on attachment 677855 [details] [diff] [review]
fixed to exclude running non-cedar branch builds on android pandas

That's not pretty, but it'll go away when we're ready.
Comment 15 Kim Moir [:kmoir] 2012-11-05 12:10:13 PST
Created attachment 678417 [details] [diff] [review]
patch to revert addition of esr17
Comment 16 Justin Wood (:Callek) 2012-11-05 12:11:41 PST
Comment on attachment 678417 [details] [diff] [review]
patch to revert addition of esr17

Review of attachment 678417 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the think-o when reading this part of the patch and making that suggestion
Comment 17 Kim Moir [:kmoir] 2012-11-05 17:50:03 PST
In production

Two things 
-Android 2.2 builds are runing in cedar builds, we don't know why, will investigate tomorrow
-Android builders appear on the bm29, this should be disabled, only panda ones should appear
Comment 18 Aki Sasaki [:aki] 2012-11-05 19:06:04 PST
Created attachment 678604 [details] [diff] [review]
fix pandas
Comment 19 Kim Moir [:kmoir] 2012-11-05 20:35:06 PST
Last patch is in production
Comment 20 Kim Moir [:kmoir] 2012-12-12 13:52:07 PST
Created attachment 691511 [details] [diff] [review]
patch to add 66 pandas to production + enable bm44

As imaged in bug 820825 (pandas-[0808-0873] which are panda-chassis-[073-078])
Comment 21 Justin Wood (:Callek) 2012-12-12 14:18:16 PST
Comment on attachment 691511 [details] [diff] [review]
patch to add 66 pandas to production + enable bm44

Review of attachment 691511 [details] [diff] [review]:
-----------------------------------------------------------------

Beyond skimming I did not do a close sanity check (e.g. are the panda-relays/id's correct, is there a typo in there, etc. I leave as a 'I trust you did it right')

I did however verify that all relay's used are identified as "Configured" in Bug 809932 properly.

::: buildfarm/mobile/devices.json
@@ +2306,5 @@
> +    "panda-0840": {
> +        "foopy": "foopy99",
> +        "relayhost": "panda-relay-075.p9.releng.scl1.mozilla.com",
> +        "relayid": "2:7"
> +    },    

nit whitespace EOL
Comment 22 Kim Moir [:kmoir] 2012-12-13 11:35:39 PST
Created attachment 691920 [details] [diff] [review]
expand range of pandas in prod + enable builds on m-c
Comment 23 Justin Wood (:Callek) 2012-12-13 17:34:23 PST
(In reply to Justin Wood (:Callek) from comment #21)
> Comment on attachment 691511 [details] [diff] [review]
> patch to add 66 pandas to production + enable bm44

So, nick just hit probs while reconfig'ing this host, since its auth_keys was out of date. And I also noticed the master was not even started.

While I was there I also noticed it didn't have any keys on the host, which is needed for log uploads. for histroy. I added the keys (recursive scp from bm22), but did not start the master.
Comment 24 Kim Moir [:kmoir] 2012-12-14 08:27:38 PST
Created attachment 692318 [details] [diff] [review]
patch to fix how foopies line up
Comment 25 Justin Wood (:Callek) 2012-12-14 08:29:19 PST
Comment on attachment 692318 [details] [diff] [review]
patch to fix how foopies line up

Review of attachment 692318 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildfarm/mobile/devices.json
@@ +2308,3 @@
>          "relayhost": "panda-relay-075.p9.releng.scl1.mozilla.com",
>          "relayid": "2:7"
>      },    

nit whitespace EOL
Comment 26 Kim Moir [:kmoir] 2013-01-03 07:47:13 PST
Created attachment 697467 [details] [diff] [review]
add new pandas in racks 7 and 8 as requested in bug 824766

Note:   This patch is just to assign the pandas to foopies in racks 7 and 8.  There are still some more to be assigned in rack 9.  

I didn't realize that we should be assigning 12 or 13 boards to each foopy when I set up the 66 pandas (rack 9) earlier. Previously, I assigned 11 boards so each chassis would map to a foopy. But this doesn't scale because we don't have enough foopies in the matching network to do this.

I'll reorganize the board assignments for the foopies in rack 9 once these new boards are online.
Comment 27 Armen Zambrano [:armenzg] - Engineering productivity 2013-01-03 08:14:43 PST
Comment on attachment 697467 [details] [diff] [review]
add new pandas in racks 7 and 8 as requested in bug 824766

I attached in some bug a script that helps sorting out the assignment of boards to foopies taking into consideration the 88 to 7 foopies ratio.

Tangential to the patch:
Is rack #6 going to be fully used? I only see some of its devices listed on devices.json.
Comment 28 Kim Moir [:kmoir] 2013-01-03 08:26:15 PST
Thanks for the review Armen.

Right now Joel is just using some of the devices in rack 6, see bug 825876 for details.  I assume we'll use the rest if them too.
Comment 29 Armen Zambrano [:armenzg] - Engineering productivity 2013-01-09 11:07:54 PST
What is left on this bug? (trying to do new year cleanup)
Comment 30 Kim Moir [:kmoir] 2013-01-09 11:25:51 PST
The only thing that's left is to do is to update the devices.json as I put more panda boards into production so I can attach those patches to other bugs.  Closing.

Note You need to log in before you can comment on or make changes to this bug.