696161 - create simplified talos buildbot for a-team usage

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

A few of us got together to talk about improving the process by which Talos improvements happen. One of the things that came out of that is that it would be helpful to the ateam if they had a simpler Buildbot that could run arbitrary talos tests. I did some hacking on this, and I think I have something that works. It requires a little bit of modification to existing code, and a subclassed TalosFactory. Further explanation to be provided with the patches.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: create RuntimeTalos factory

I originally tried to make this idea work by just modifying TalosFactory. The thing that makes that hard is that we need to allow for a variable amount of pagesets, plugins, and talos addons -- at runtime. AFAICT this is fundamentally incompatible with our existing usage making those decisions at config-time. The doStepIf I'm using to make that decision (doStepIf step: self._propertyIsSet(step, 'plugin')), will cause our existing talos factories not to install any plugins.

With that said, here's an overview of the changes:
- Split up plugin/pageset/addon downloads, to avoid overriding all of addSetupSteps()
- Wrap anything that uses supportUrlBase in a WithProperties
- Make add-on downloading (that is, *Firefox* add-on downloading), conditional on the addonUrl property being set.
- Create RuntimeTalosFactory that sets up all of: plugins, pageset, talosAddOns, and supportUrlBase to be settable with properties
- Support WithProperties in URLs in DownloadFile
- Allow extra parameters to be passed to PerfConfigurator through the 'configurationOptions' property.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: simple talos buildbot config

This is an extremely simple consumer of RuntimeTalosFactory. It's got one builder per platform, which is capable of running any type of Talos job given the correct properties. I plan to document it better, but wanted some feedback first. Here's some sample sendchanges:
firefox addon test: buildbot sendchange -m localhost:9118 -u bhearsum -p 'configurationOptions:--activeTests,ts,--noShutdown,--sampleConfig,addon.config' -p 'supportUrlBase:http://build.mozilla.org/talos' --branch=win7 -p 'addonUrl:/pub/mozilla.org/addons/722/noscript-2.1.5-fn+sm+fx.xpi' -p'extraTalosArgs:--amo' http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-10.0a1.en-US.win32.zip

tp: buildbot sendchange -m localhost:9118 -u bhearsum -p 'configurationOptions:--activeTests,tp5,--mozAfterPaint' -p 'supportUrlBase:http://build.mozilla.org/talos' --branch=snowleopard --revision=ec7577dec4fc --property buildid:20111017080143 --property pgo_build:False --property builduid:6594214c3546463c8a56f853a9d32415 -p 'pageset1:zips/tp5.zip' http://stage.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/mac/en-US/Firefox%207.0.1.dmg

There's a couple minor caveats here:
- releaseTester has to be uncommented to test EXEs, which is only done as part of firefox addon testing. Because TalosFactory will only unpack an exe OR a different file, we can't keep this on all the time. I tried integrating the exe unpacking logic into DownloadFile, and failed.
- We need to pull in this Buildbot patch to make passing URLs as sendchange properties works: https://github.com/buildbot/buildbot/commit/7d0ff3269bc8332159cda19eebf18e6da5fbce62

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

cc-ing Aki, as I know he was working with ctalbert, jmaher on this same problem also, but I dont know state of that work, or what bug# is tracking aki's work.

Comment 4

[[e-mail] (:bhearsum)]

(In reply to John O'Duinn [:joduinn] from comment #3)
> cc-ing Aki, as I know he was working with ctalbert, jmaher on this same
> problem also, but I dont know state of that work, or what bug# is tracking
> aki's work.

Yeah, this isn't meant to replace or undermine Aki's work -- merely a quick and dirty thing for the meantime.

Comment 5

[Rail Aliiev [:rail]]

Comment on attachment 568479 [details] [diff] [review]
simple talos buildbot config

Looks good as well.

Comment 6

[[e-mail] (:bhearsum)]

Attachment: rename addon to talosAddon, for clarity

Sorry, I should've posted these updates sooner. This is a tiny change to the buildbotcustom patch, renaming 'addonN' to 'talosAddonN', for clarity.

Comment 7

[[e-mail] (:bhearsum)]

Attachment: add documentation

This patch addons documentation to the simple talos master.

Comment 8

[[e-mail] (:bhearsum)]

This made it to production today.

Comment 9

[cmtalbert]

(In reply to Ben Hearsum [:bhearsum] from comment #8)
> This made it to production today.

Awesome, thanks Ben!!

Joel, should we open a bug to track landing this in our staging environment?  How do you want to handle that?