448047 - integrate the talos perfrunner into buildbotcustom

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[alice nodelman [:alice] [:anode]]

Too many perfrunners, too much repeated code.

Comment 2

[Chris AtLee [:catlee]]

Attachment: Add talos steps and factory to buildbotcustom

Comment 3

[Chris AtLee [:catlee]]

The pool of slaves for talos is using this code, hence the dependency.

Comment 4

[alice nodelman [:alice] [:anode]]

Comment on attachment 373337 [details] [diff] [review]
Add talos steps and factory to buildbotcustom

Are you only incorporating nomerge schedulers?  We still use merging schedulers for a lot of the talos setup.

If we want this to be shared with stage could there be an option to download talos from a local zip?  Then we could have the same factory running everywhere.

Also, should the tinderboxmailnotifier go in with this?

Comment 5

[Chris AtLee [:catlee]]

Attachment: Add talos steps and factory to buildbotcustom

This adds the merging multi-scheduler, as well as our tinderboxmailnotifier.

I also renamed customPageset to customTalos for downloading talos from a .zip file instead of checking it out from CVS.

Comment 6

[alice nodelman [:alice] [:anode]]

Comment on attachment 373650 [details] [diff] [review]
Add talos steps and factory to buildbotcustom

+        if OS in ('leopard', 'tiger'):
+            self.addStep(FileDownload(
+             mastersrc="%s/buildfarm/utils/installdmg.ex" % toolsDir,
+             slavedest="installdmg.ex",


At the moment, we install on tiger with install.sh not install.ex.  We'd first have to prove that install.ex works on both tiger and leopard before we could go with this step.

Comment 7

[Chris AtLee [:catlee]]

(In reply to comment #6)
> (From update of attachment 373650 [details] [diff] [review])
> +        if OS in ('leopard', 'tiger'):
> +            self.addStep(FileDownload(
> +             mastersrc="%s/buildfarm/utils/installdmg.ex" % toolsDir,
> +             slavedest="installdmg.ex",
> 
> 
> At the moment, we install on tiger with install.sh not install.ex.  We'd first
> have to prove that install.ex works on both tiger and leopard before we could
> go with this step.

It's been running well in the staging talos pool:
http://qm-buildbot01.mozilla.org:2009/builders/MacOSX%20Darwin%208.8.1%20talos%20mozilla-1.9.1%20pool/builds/135/steps/Unpack%20build/logs/stdio

Comment 8

[Chris AtLee [:catlee]]

Attachment: Add talos utillities into tools repository

These are fetched from the tools repository using a regular FileDownload step.  This means you need the tools repository checked out on the talos master.

Comment 9

[alice nodelman [:alice] [:anode]]

Comment on attachment 373650 [details] [diff] [review]
Add talos steps and factory to buildbotcustom

Ship it.

Comment 10

[Chris AtLee [:catlee]]

Comment on attachment 374277 [details] [diff] [review]
Add talos utillities into tools repository

changeset:   272:dcc3a168e807

this doesn't require a reconfig anywhere

Comment 11

[Chris AtLee [:catlee]]

Comment on attachment 373650 [details] [diff] [review]
Add talos steps and factory to buildbotcustom

changeset:   261:9f3e6517d946

Comment 12

[Chris AtLee [:catlee]]

Attachment: Talos Factory cleanup

It looks like we can use the same clean command on all platforms, so this change uses 'rm -vrf *' for all platforms.

Until we have something to replace tinderbox, we still need to lie about our start time (sob).

This patch also calls firefox-bin instead of firefox on mac and linux.

Comment 13

[Chris AtLee [:catlee]]

Attachment: Extract information from application.ini into build properties

This patch will extract out the buildid, revision, and repository path (if available) and set the appropriate build properties.

This should simplify regression hunting a bit, and will also be of benefit once the build database is collecting this information.

Comment 14

[alice nodelman [:alice] [:anode]]

Comment on attachment 376288 [details] [diff] [review]
Talos Factory cleanup

             exepath = WithProperties('%(exepath)s')
-        else:
+        elif OS in ('xp', 'vista'):
             exepath = '../firefox/firefox'
+	else:
+            exepath = '../firefox/firefox-bin'


Should this change be matched up with updates to all the master.cfgs - since they currently specify all windows boxes as being OS='win'?

Comment 15

[alice nodelman [:alice] [:anode]]

Comment on attachment 376497 [details] [diff] [review]
Extract information from application.ini into build properties

+            retval = {}
+            stdout = "\n".join([stdout, stderr])
+            m = re.search("^BuildID=(\w+)", stdout, re.M)
+            if m:
+                retval['buildid'] = m.group(1)
+            m = re.search("^SourceStamp=(.*)", stdout, re.M)
+            if m:
+                retval['revision'] = m.group(1)
+            m = re.search("^SourceRepository=(\S+)", stdout, re.M)
+            if m:
+                retval['repo_path'] = m.group(1)


We've had problems in the past with not being flexible enough with our regexes when searching for buildid/sourcestamp/sourcerepo.  Should be able to absorb spaces before/after the '=', just in case.

Also, would it be valuable to put in default values?  Firefox3.0 does not have sourcestamp/repo.  Otherwise you force checking for existence before using these properties throughout the rest of the code.

Comment 16

[Chris AtLee [:catlee]]

Attachment: Update talos-staging to use buildbotcustom's TalosFactory

Comment 17

[Chris AtLee [:catlee]]

Attachment: Update production talos to use buildbotcustom's TalosFactory

Comment 18

[Chris AtLee [:catlee]]

Attachment: Update talos-try to use buildbotcustom's TalosFactory

Comment 19

[alice nodelman [:alice] [:anode]]

Comment on attachment 376288 [details] [diff] [review]
Talos Factory cleanup

With the addition of the patches for talos production/try/stage I'm happy with this now.

Comment 20

[Chris AtLee [:catlee]]

Attachment: Extract information from application.ini into build properties

Comment 21

[Chris AtLee [:catlee]]

Comment on attachment 376288 [details] [diff] [review]
Talos Factory cleanup

changeset:   303:e74a99683ce4

Comment 22

[Chris AtLee [:catlee]]

Comment on attachment 378385 [details] [diff] [review]
Extract information from application.ini into build properties

changeset:   306:339bc2eba1de

Comment 23

[Chris AtLee [:catlee]]

Attachment: Minor fixes for perfrunner

A few minor fixes for perfrunner that I discovered after the pool has been active for a while.

We should output which slave the build is running on, just like we do for builds.

We should also go back to using installdmg.sh.  installdmg.ex is too unreliable.  This also affects unitests on packaged builds, but since they checkout the tools repository directly, their factories don't need to be updated to look for installdmg.sh.

Comment 24

[Chris AtLee [:catlee]]

Attachment: Minor fixes for perfrunner

Same as before, except unpack symbols in the right place

Comment 25

[alice nodelman [:alice] [:anode]]

Comment on attachment 380458 [details] [diff] [review]
Minor fixes for perfrunner

         self.addStep(ShellCommand(
+          command=['echo', 'TinderboxPrint:', WithProperties('s: %(slavename)s')],
+          workdir=workdirBase,
+        ))
+        self.addStep(ShellCommand(


I can't decide if this should be part of the talos code instead of the master - currently talos is the one that sets up all the things that are Tinderboxprinted, so I don't know if it is a good idea to split reporting between master and slave.  Any opinions?

Comment 26

[[e-mail] (:bhearsum)]

(In reply to comment #25)
> (From update of attachment 380458 [details] [diff] [review])
>          self.addStep(ShellCommand(
> +          command=['echo', 'TinderboxPrint:', WithProperties('s:
> %(slavename)s')],
> +          workdir=workdirBase,
> +        ))
> +        self.addStep(ShellCommand(
> 
> 
> I can't decide if this should be part of the talos code instead of the master -
> currently talos is the one that sets up all the things that are
> Tinderboxprinted, so I don't know if it is a good idea to split reporting
> between master and slave.  Any opinions?

It all ends up in the same log that's sent to Tinderbox so this is totally fine IMHO. More importantly, I think it's proper that Buildbot prints out the slavename rather than Talos.

Comment 27

[Chris AtLee [:catlee]]

(In reply to comment #25)
> (From update of attachment 380458 [details] [diff] [review])
>          self.addStep(ShellCommand(
> +          command=['echo', 'TinderboxPrint:', WithProperties('s:
> %(slavename)s')],
> +          workdir=workdirBase,
> +        ))
> +        self.addStep(ShellCommand(
> 
> 
> I can't decide if this should be part of the talos code instead of the master -
> currently talos is the one that sets up all the things that are
> Tinderboxprinted, so I don't know if it is a good idea to split reporting
> between master and slave.  Any opinions?

This matches what is currently done in the build/unittest factories.

But if the talos code could be modified to print out the equivalent code, and the rest of the Tinderbox print's are there, then you have a good point.

Comment 28

[Chris AtLee [:catlee]]

Attachment: Minor fixes for perfrunner

Comment 29

[Chris AtLee [:catlee]]

Attachment: Print out slave name

Comment 30

[Chris AtLee [:catlee]]

Comment on attachment 382596 [details] [diff] [review]
Minor fixes for perfrunner

changeset:   332:142f03acddd4

Comment 31

[Chris AtLee [:catlee]]

Comment on attachment 382599 [details] [diff] [review]
Print out slave name

>   utils.debug("actual date: %d" % int(time.time()))
>+  print 'RETURN:s: %s' % browser_config['title']

Checked in this instead after some tree burnage:

print 'RETURN:s: %s' % title

Comment 32

[Chris AtLee [:catlee]]

The only thing left to do here is to convert talos-try over to use buildbotcustom's TalosFactory (or a slightly modified version of it).

We don't need to land any of the above changes to talos staging or talos production because those slaves have been moved over to the pool in the meanwhile, which is using the new code.

Comment 33

[Chris AtLee [:catlee]]

Attachment: TryTalosFactory

We need slightly different handling of the start time processing, as well as printing out the build's identifier, hence the requirement for a separate factory.

Comment 34

[Chris AtLee [:catlee]]

Attachment: Using TryTalosFactory in new talos staging try master

Comment 35

[Chris AtLee [:catlee]]

Attachment: TryTalosFactory

Same as before, adjusted for new 'plugins' and 'pageset' parameters

Comment 36

[alice nodelman [:alice] [:anode]]

Comment on attachment 389695 [details] [diff] [review]
TryTalosFactory

Even with this:
+                # Lies!!!
+                # Once we don't need to lie about our start time,
+                # the following code can go away


I'll let it pass. :)

Comment 37

[Chris AtLee [:catlee]]

Comment on attachment 389695 [details] [diff] [review]
TryTalosFactory

changeset:   379:0091ced82f6d

Comment 38

[Chris Cooper [:coop] (he/him)]

Moving closed Future bugs into Release Engineering in preparation for removing the Future component.