785675 - Port |Bug 748490 - Provide common location for testing modules| to fix multiple perma-oranges.

Status: RESOLVED FIXED

(SeaMonkey :: Testing Infrastructure, defect)

Description

[Philip Chee]

One example is Bug 783519 - [SeaMonkey] "browser_NetUtil.js | Exception thrown - [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIXPCComponents_Utils.import]""

Many of our tests are failing at trying to import:
Cu.import("resource://testing-common/httpd.js");

According to Serge we need to port:
Bug 748490 - Provide common location for testing modules

Comment 1

[Edmund Wong (:ewong)]

Attachment: Port |Bug 748490| to SeaMonkey

Comment 2

[Philip Chee]

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1347044604.1347046278.17635.gz#err7

NEXT ERROR TEST-UNEXPECTED-FAIL | e:\builds\slave\test\build\xpcshell\tests\image\test\unit\test_async_notification.js | test failed (with xpcshell return code: 3), see following log:
>>>>>>>
### XPCOM_MEM_LEAK_LOG defined -- logging leaks to c:\docume~1\seabld\locals~1\temp\tmp23pcxi\runxpcshelltests_leaks.log
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040111: file e:/builds/slave/comm-cen-trunk-w32-dbg/build/mozilla/netwerk/base/src/nsIOService.cpp, line 648
WARNING: NS_ENSURE_SUCCESS(rv, 0x80070057) failed with result 0x80040111: file e:/builds/slave/comm-cen-trunk-w32-dbg/build/mozilla/js/xpconnect/loader/mozJSComponentLoader.cpp, line 1091
async_load_tests.js:14: NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIXPCComponents_Utils.import]

Comment 3

[Frank Wein [:mcsmurf]]

ewong: The argument needs to be --testing-modules-dir (instead of --tests-modules-dir).
BTW: I wonder why TB does not have this problem. Do they directly run the test infrastructure in mozilla/ and include their tests in some way?

Comment 4

[Edmund Wong (:ewong)]

(In reply to Frank Wein [:mcsmurf] from comment #3)
> ewong: The argument needs to be --testing-modules-dir (instead of
> --tests-modules-dir).
> BTW: I wonder why TB does not have this problem. Do they directly run the
> test infrastructure in mozilla/ and include their tests in some way?

Thanks!

Comment 5

[Edmund Wong (:ewong)]

Attachment: Port |Bug 748490 - Provide common location for testing modules| to fix multiple perma-oranges. (v2)

Comment 6

[Frank Wein [:mcsmurf]]

I now looked at this locally and in the buildbot log, I'm not sure if this patch is the correct fix for the problem. When running "make xpcshell-tests" in the objdir, it enters the mozilla/ directory and runs the tests there (see http://hg.mozilla.org/comm-central/annotate/861e8385f731//Makefile.in#l48). So there the --testing-modules-dir argument already gets appended to the command line. But buildbot testing works a bit different as the test run does not necessarily happen on the same box as the build run. buildbot downloads a special zip file (for example http://ftp.mozilla.org/pub/mozilla.org/seamonkey/tinderbox-builds/comm-central-trunk-win32-debug/1349392410/seamonkey-2.15a1.en-US.win32.tests.zip), extracts it and runs those tests. I'll investigate this further.

Comment 7

[Frank Wein [:mcsmurf]]

Found the solution! :)
Callek: We need to adjust this buildbot step for the xpcshell tests, not sure how this works, you probably know this better than I do:
======== BuildStep started ========
unpack tests
=== Output ===
'unzip' '-o' 'seamonkey-2.15a1.en-US.win32.tests.zip' 'bin*' 'certs*' 'xpcshell*'

For comparison, here's the Thunderbird buildbot step:

========= Started unpack tests (results: 0, elapsed: 37 secs) (at 2012-10-04 15:52:18.219199) =========
'unzip' '-oq' u'thunderbird-18.0a1.en-US.win32.tests.zip' 'bin*' 'certs*' 'modules*' 'xpcshell*'
 in dir c:\talos-slave\test\build (timeout 1200 secs)
 watching logfiles {}
 argv: ['unzip', '-oq', u'thunderbird-18.0a1.en-US.win32.tests.zip', 'bin*', 'certs*', 'modules*', 'xpcshell*']

The difference is the modules* part, in the modules directory the file httpd.js can be found. The runxpcshelltests.py tries to guess the location of the testing modules by looking for the modules/ directory in the current working directory. So we also need to unzip the modules directory from the tests.zip file.

Comment 8

[Frank Wein [:mcsmurf]]

Also see Bug 755339 on that solution from Comment 7 (checkin in buildbot config: http://hg.mozilla.org/build/buildbotcustom/rev/7d0cd546fb77)

Comment 9

[Edmund Wong (:ewong)]

Attachment: Modified buildbotcustom configs to fix mulple perma-oranges. (v1)

Comment 10

[Justin Wood (:Callek)]

Comment on attachment 667296 [details] [diff] [review]
Port |Bug 748490 - Provide common location for testing modules| to fix multiple perma-oranges. (v2)

Review of attachment 667296 [details] [diff] [review]:
-----------------------------------------------------------------

::: config/rules.mk
@@ +1366,5 @@
> +# For each file defined in TESTING_JS_MODULES, copy it to
> +# objdir/_tests/modules/. If TESTING_JS_MODULE_DIR is defined, that path
> +# wlll be appended to the output directory.
> +
> +ifdef TESTING_JS_MODULES

nit please wrap this in ifdef ENABLE_TESTS

Comment 11

[Justin Wood (:Callek)]

Sorry for the delay guys, but thanks for the investigation/work!

Comment 12

[Edmund Wong (:ewong)]

Attachment: Port |Bug 748490 - Provide common location for testing modules| to fix multiple perma-oranges. (v3)

Comment 13

[Edmund Wong (:ewong)]

Attachment: Port |Bug 748490 - Provide common location for testing modules| to fix multiple perma-oranges. (v4)

Comment 14

[Edmund Wong (:ewong)]

Pushed to comm-central:
http://hg.mozilla.org/comm-central/rev/8007e33a86ee

Comment 15

[Edmund Wong (:ewong)]

Pushed　ｔｏ buildbotcustom:
http://hg.mozilla.org/build/buildbotcustom/rev/fc2ad1ffc38c

Comment 16

[Frank Wein [:mcsmurf]]

There's been a typo in the buildbotcustom patch, I pushed a typo fix: http://hg.mozilla.org/build/buildbotcustom/rev/f2950891859c

Comment 17

[Frank Wein [:mcsmurf]]

This looks fixed, failing xpcshell tests are down from ~420 to ~3 :-)