560191 - testsUrl not set for some SendChangeSteps

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Rail Aliiev [:rail]]

See sendchange_4 and others (first 3 are OK)
http://production-master02.build.mozilla.org:8010/builders/Linux%20mozilla-1.9.1%20nightly/builds/255

[Failure instance: Traceback: <type 'exceptions.KeyError'>: 'testsUrl'
/tools/twisted-8.0.1/lib/python2.5/site-packages/twisted/internet/defer.py:194:addCallback
/tools/twisted-8.0.1/lib/python2.5/site-packages/twisted/internet/defer.py:185:addCallbacks
/tools/twisted-8.0.1/lib/python2.5/site-packages/twisted/internet/defer.py:323:_runCallbacks
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/buildstep.py:712:_startStep_2
--- <exception caught here> ---
/tools/buildbotcustom/buildbotcustom/steps/misc.py:298:start
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/properties.py:88:render
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/properties.py:86:render
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/properties.py:156:render
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/properties.py:131:__getitem__
/tools/buildbot-production/lib/python2.5/site-packages/buildbot-0.7.10p1-py2.5.egg/buildbot/process/properties.py:42:__getitem__
]


I suspect the following changeset: 
http://hg.mozilla.org/build/buildbotcustom/rev/e02144446979

Comment 1

[Chris AtLee [:catlee]]

Probably because we don't do packaged opt tests on 1.9.1.

Comment 2

[Rail Aliiev [:rail]]

(In reply to comment #1)
> Probably because we don't do packaged opt tests on 1.9.1.

In this case this bug looks like a good candidate for WONTFIX.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

I was expecting to find the exception in the twistd.log emails but didn't, is that expected ?

Comment 4

[Chris AtLee [:catlee]]

(In reply to comment #3)
> I was expecting to find the exception in the twistd.log emails but didn't, is
> that expected ?

I was expecting to as well.

(In reply to comment #2)
> (In reply to comment #1)
> > Probably because we don't do packaged opt tests on 1.9.1.
> 
> In this case this bug looks like a good candidate for WONTFIX.

I think the steps are still throwing an error though...is this causing redness?

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

No redness, but the sendchange fails for non-talos cases. Not that we want unit sendchanges from build/nightly jobs, just unit test for 1.9.1.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

a couple things we can do:

1. leave as is.  the sendchange sent would be ignored anyway as we do unittest builds currently.
2. default testsURL to ''.  This would mean that if make upload doesn't work, the test running side (if listening for the sendchange) would fail trying to download an empty string as the tests package.
3. only do unit test sendchanges if we know that there are packaged unit tests being created for that build and they will be picked up in make upload.

While I think option 3 is the most correct, I don't know if there is time to fix it.  I'd vote for option 1 and agree that this bug can be marked RESOLVED->WONTFIX

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Kinda sucks to have every opt and nightly build on 1.9.1 orange because testsUrl isn't set. Could we change the unittest_masters definition there to not warn on failure, and accept the risk that we break the sendchanges for the unit test builds at some point ? Or just set testUrl = None if the packageUrl matches 1.9.1 and testUrl is undefined in buildbotcustom.process.factory:parse_make_upload() ?

Comment 10

[Serge Gautherie (:sgautherie)]

Linux mozilla-1.9.1 build
OS X 10.5.2 mozilla-1.9.1 build

WontFixing perma-orange :-<

Comment 12

[:Gavin Sharp [email: [e-mail]]]

Perma orange on 1.9.1 isn't acceptable. We should at the very least find a workaround along the lines of comment 8.

Comment 13

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #8)
> Kinda sucks to have every opt and nightly build on 1.9.1 orange because
> testsUrl isn't set. Could we change the unittest_masters definition there to
> not warn on failure, and accept the risk that we break the sendchanges for the
> unit test builds at some point ? Or just set testUrl = None if the packageUrl
> matches 1.9.1 and testUrl is undefined in
> buildbotcustom.process.factory:parse_make_upload() ?

If we are ok with ugly hardcodes, we could do something like

         if '1.9.1' in self.branchName:
             files = [WithProperties('%(packageUrl)s')]
         else:
             files = [WithProperties('%(packageUrl)s'),
                      WithProperties('%(testsUrl)s')]
         for master, warn, retries in self.unittestMasters:
            self.addStep(SendChangeStep(
             name='sendchange_%s' % master,
             warnOnFailure=warn,
             master=master,
             retries=retries,
             branch=self.unittestBranch,
             revision=WithProperties('%(got_revision)s'),
             files=files,
             user=WithProperties('%(who)s'))
           )

Comment 14

[[e-mail] (:bhearsum)]

I think we can override by setting unittest_masters in production_config.py's  BRANCHES['mozilla-1.9.1'] dict.

Comment 15

[Chris AtLee [:catlee]]

I'll get this done

Comment 16

[Chris AtLee [:catlee]]

Attachment: Don't send testsUrl property for 1.9.1 branch

hackity hack

Comment 17

[Chris AtLee [:catlee]]

Comment on attachment 492538 [details] [diff] [review]
Don't send testsUrl property for 1.9.1 branch

changeset:   1129:77a344a6c885

Comment 18

[Serge Gautherie (:sgautherie)]

verified1.9.1, per
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox3.5/1290616310.1290618968.23652.gz
Linux mozilla-1.9.1 build on 2010/11/24 08:31:50