722026 - Thunderbird Try hardcodes 'thunderbird' in some places: packaged tests fail to run on SeaMonkey

Status: NEW

(SeaMonkey :: Release Engineering, enhancement, P5)

Description

[Serge Gautherie (:sgautherie)]

(Noticed while checking around bug 721357 comment 6.)

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327638697.1327638973.19378.gz&fulltext=1
WINNT 5.2 try-comm-central opt test xpcshell on 2012/01/26 20:31:37
{
...
======== BuildStep started ========
set props: exepath
=== Output ===
exepath: thunderbird/thunderbird.exe
=== Output ended ===
======== BuildStep ended ========
======== BuildStep started ========
set props: exedir
=== Output ===
exedir: thunderbird
=== Output ended ===
======== BuildStep ended ========
...

...
======== BuildStep started ========
no change
=== Output ===
python e:/buildbot/test/tools/buildfarm/utils/printbuildrev.py thunderbird
...
Traceback (most recent call last):
  File "e:/buildbot/test/tools/buildfarm/utils/printbuildrev.py", line 14, in <module>
    buildid = appini.get('App', 'BuildID')
  File "d:\mozilla-build\python25\lib\ConfigParser.py", line 511, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'App'
program finished with exit code 1
elapsedTime=1.282000
=== Output ended ===
======== BuildStep ended ========
...

...
======== BuildStep started ========
'bash -c ...' warnings
=== Output ===
bash -c if [ ! -d thunderbird/plugins ]; then mkdir thunderbird/plugins; fi && cp bin/xpcshell.exe thunderbird && cp -R bin/components/* thunderbird/components/ && cp -R bin/plugins/* thunderbird/plugins/ && python -u xpcshell/runxpcshelltests.py --symbols-path=../symbols --manifest=xpcshell/tests/all-test-dirs.list thunderbird/xpcshell.exe
...
mkdir: cannot create directory `thunderbird/plugins': No such file or directory
program finished with exit code 1
elapsedTime=0.375000
TinderboxPrint: xpcshell<br/><em class="testfail">T-FAIL</em>
Unknown Error: command finished with exit code: 1
=== Output ended ===
======== BuildStep ended ========
...
}

1)
I think this used to work a long time ago, but I don't remember for sure.

2)
I know TB Try is not supposed to detect which (other) application is build,
but it looks like some simple changes could make (xpcshell) tests succeed.

3)
In case you actually don't want to (explicitly) support other apps,
running tests should probably be disable for any non-TB apps,
wrt to xpcshell (and maybe wrt mailbloat once bug 721357 is fixed).

Comment 1

[Mark Banner (:standard8)]

(In reply to Serge Gautherie (:sgautherie) from comment #0)
> 1)
> I think this used to work a long time ago, but I don't remember for sure.

It did not.

> 2)
> I know TB Try is not supposed to detect which (other) application is build,
> but it looks like some simple changes could make (xpcshell) tests succeed.

iirc we were (and still are) happy for other apps to push to it, but there was agreement that the Thunderbird team wouldn't put any effort into making this work for other apps (In theory, if there was detection then you could almost have one try server for all apps).

> 3)
> In case you actually don't want to (explicitly) support other apps,
> running tests should probably be disable for any non-TB apps,
> wrt to xpcshell (and maybe wrt mailbloat once bug 721357 is fixed).

This isn't possible because of item 2.

Comment 2

[Serge Gautherie (:sgautherie)]

(In reply to Serge Gautherie (:sgautherie) from comment #0)
> wrt to xpcshell (and maybe wrt mailbloat once bug 721357 is fixed).

'mailbloat' is unaffected, per bug 721357 comment 12.

Comment 3

[Serge Gautherie (:sgautherie)]

Ftr, 'mozmill' tests have the same issues:

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327846880.1327846972.20880.gz
Linux Fedora 12 x86-64 comm-central test mozmill on 2012/01/29 06:21:20

Comment 4

[Serge Gautherie (:sgautherie)]

Fwiw, it is worse on OS X 10.5.2:

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327850239.1327850307.30727.gz
OS X 10.5.2 comm-central test xpcshell on 2012/01/29 07:17:19
http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327850399.1327850474.31214.gz
OS X 10.5.2 comm-central test mozmill on 2012/01/29 07:19:59
{
set build property_2 exception
...
Traceback (most recent call last):
...
exceptions.KeyError: 'exepath'
}

Comment 5

[Mark Banner (:standard8)]

MozMill tests are hugely app-specific Ui, so I wouldn't even bother caring about those.

Comment 6

[Serge Gautherie (:sgautherie)]

(In reply to Serge Gautherie (:sgautherie) from comment #4)
> Fwiw, it is worse on OS X 10.5.2:

On OS X 10.6.2 too:

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327857090.1327857141.17564.gz
OS X 10.6.2 try-comm-central opt test crashtest on 2012/01/29 09:11:30
http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327857175.1327857231.17693.gz&fulltext=1
OS X 10.6.2 try-comm-central opt test mozmill on 2012/01/29 09:12:55
http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327856919.1327856972.17188.gz&fulltext=1
OS X 10.6.2 try-comm-central opt test reftest on 2012/01/29 09:08:39
http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327857006.1327857056.17474.gz&fulltext=1
OS X 10.6.2 try-comm-central opt test xpcshell on 2012/01/29 09:10:06


(In reply to Mark Banner (:standard8) from comment #5)
> MozMill tests are hugely app-specific Ui, so I wouldn't even bother caring
> about those.

I know, I am just reporting it for the record.

Comment 7

[Serge Gautherie (:sgautherie)]

(In reply to Serge Gautherie (:sgautherie) from comment #0)
> http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327638697.
> 1327638973.19378.gz&fulltext=1
> WINNT 5.2 try-comm-central opt test xpcshell on 2012/01/26 20:31:37

And the other tests on Windows:

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327856070.1327856442.16343.gz&fulltext=1
WINNT 5.2 try-comm-central opt test crashtest on 2012/01/29 08:54:30
http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327855587.1327855981.14726.gz&fulltext=1
WINNT 5.2 try-comm-central opt test reftest on 2012/01/29 08:46:27
{
Error: Path e:\buildbot\test\build\thunderbird\thunderbird.exe doesn't exist.
}

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTry/1327856548.1327857140.17561.gz&fulltext=1
WINNT 5.2 try-comm-central opt test mozmill on 2012/01/29 09:02:28
{
python: can't open file 'mozmill/resources/installmozmill.py': [Errno 2] No such file or directory
}

Comment 8

[Serge Gautherie (:sgautherie)]

(In reply to Serge Gautherie (:sgautherie) from comment #0)

Having a look at this:


> mkdir: cannot create directory `thunderbird/plugins': No such file or

Depends on 'exedir':
http://mxr.mozilla.org/build/source/buildbotcustom/steps/unittest.py#736
{
737 class MozillaPackagedXPCShellTests(ShellCommandReportTimeout):
...
751         script = " && ".join(["if [ ! -d %(exedir)s/plugins ]; then mkdir %(exedir)s/plugins; fi",
752                   "cp bin/xpcshell" + bin_extension + " %(exedir)s",
753                   "cp -R bin/components/* %(exedir)s/components/",
754                   "cp -R bin/plugins/* %(exedir)s/plugins/",
755                   "python -u xpcshell/runxpcshelltests.py"])
...
759         script += " --manifest=xpcshell/tests/all-test-dirs.list %(exedir)s/xpcshell" + bin_extension
}


> python e:/buildbot/test/tools/buildfarm/utils/printbuildrev.py thunderbird

Depends on '%(exedir)s':
http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#6723
{
6531 class MozillaTestFactory(MozillaBuildFactory):
...
6724     def addIdentifySteps(self):
...
6736          command=['python', WithProperties('%(toolsdir)s/buildfarm/utils/printbuildrev.py'),
6737                   WithProperties('%(exedir)s')],
}


> exedir: thunderbird

http://mxr.mozilla.org/build/search?string=get_exedir&case=1

Depends on 'exepath':
http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#6650
{
6531 class MozillaTestFactory(MozillaBuildFactory):
...
6651         def get_exedir(build):
6652             return os.path.dirname(build.getProperty('exepath'))
...
7243 class TalosFactory(RequestSortingBuildFactory):
...
7528     def addGetBuildInfoStep(self):
7529         def get_exedir(build):
7530             return os.path.dirname(build.getProperty('exepath'))
}


> exepath: thunderbird/thunderbird.exe

http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#6629
{
6647              value="%s/%s%s" % (self.productName, self.productName,
6648                                 self.posixBinarySuffix),
}
There are linux/mac/win cases and code for Talos too, but that's the idea.

http://mxr.mozilla.org/build/search?string=productName&case=1&find=thunderbird
{
/buildbot-configs/thunderbird/release_master.py
    * line 81 -- 'productName' : 'thunderbird',
    * line 177 -- 'productName' : 'thunderbird',
    * line 278 -- 'productName' : 'thunderbird',
    * line 379 -- 'productName' : 'thunderbird',
}
+
{
/buildbot-configs/thunderbird/master.cfg
    * line 480 -- productName=appname,
    * line 531 -- productName=appname,
    * line 659 -- productName=appname,
    * line 715 -- productName=appname,
}
with
{
247     appname=branch.get('appname', MOZ_APP_NAME)
}

http://mxr.mozilla.org/build/search?string=MOZ_APP_NAME&case=1&find=thunderbird
{
/buildbot-configs/thunderbird/config.py
    * line 25 -- MOZ_APP_NAME = 'thunderbird'
}
Would there be any way to override this value (from a pushed .mozconfig or otherwise)?
And would it break anything?


(In reply to Mark Banner (:standard8) from comment #1)
> iirc we were (and still are) happy for other apps to push to it, but there
> was agreement that the Thunderbird team wouldn't put any effort into making
> this work for other apps

Noted :-|

Comment 9

[Justin Wood (:Callek)]

Me and gozer had a BRIEF discussion where the best solution would be to allow push-to-try for suite which builds/runs-tests on suite systems... Such that we could be sure that the buildbot factories treat it as suite, where the actual code treats it as suite without hacky needs, etc.

That is something I don't have time for yet, and I don't expect doing suite builds/tests to be much easier than at present until I do.

Comment 10

[Serge Gautherie (:sgautherie)]

Ftr, with 1+ simple workarounds, B + X + Ru builds of SeaMonkey can work on new try-comm-central :-)

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=bf50639e22fd&noignore=1

*****

I wonder whether it could be possible to have a "SeaMonkey Try on Firefox infrastructure", similar to what Thunderbird just did.

Comment 11

[Mark Banner (:standard8)]

This bug is now invalid for MoMo try server. I'll move it to SeaMonkey as I don't know what you guys want to do with it, but I don't see releng supporting it just for the same reasons as MoMo didn't previously.