817122 - Figure out which xpcshell tests to initially run on b2g and create an xpcshell_b2g.ini

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Andrew Halberstadt [:ahal]]

Due to the same reasons we can't enable all reftests and mochitests on the emulator (linux slave load), we don't have the capacity to turn on all xpcshell tests either.

I think the best strategy is to initially have a very small subset of tests running. These would definitely include the gonk tests (bug 810547) and the update tests (bug 787578).

Are there any other tests that are important to enable asap?

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Patch 1.0 - initial patch to add xpcshell_b2g.ini

This patch will add an xpcshell_b2g.ini manifest.

Notes:
1) I populated it with a handful of manifests that seemed relevant to B2G based on their file paths
2) This is by no means the final set of tests we are running, just a baseline that I arbitrarily chose. Tests can still be added/removed on demand
3) Once we are running a larger set of tests, I plan to remove the xpcshell_b2g.ini file and switch back to using the canonical location.
4) If you have other tests you want to see enabled immediately, please comment in this bug.

See also: https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.b2g/0ryaHNSMNaQ

Comment 2

[Jonathan Griffin (:jgriffin)]

Comment on attachment 687820 [details] [diff] [review]
Patch 1.0 - initial patch to add xpcshell_b2g.ini

Review of attachment 687820 [details] [diff] [review]:
-----------------------------------------------------------------

Awesome!

Comment 3

[Andrew Halberstadt [:ahal]]

https://hg.mozilla.org/mozilla-central/rev/740982d30458
https://hg.mozilla.org/releases/mozilla-aurora/rev/44107c5507f9
https://hg.mozilla.org/releases/mozilla-beta/rev/a44fedf55a6c

Comment 4

[Andrew Halberstadt [:ahal]]

Xpcshell tests could possibly be live on cedar as early as tomorrow with this manifest. I'll leave this bug open for triaging and getting feedback on other sets of tests to enable.

Comment 5

[Panos Astithas (he/him) [:past] (please ni?)]

I want to have the debugger unit tests running against b2g (toolkit/devtools/debugger/tests/unit/). Can we add them please?

Comment 6

[Andrew Halberstadt [:ahal]]

(In reply to Panos Astithas [:past] from comment #5)
> I want to have the debugger unit tests running against b2g
> (toolkit/devtools/debugger/tests/unit/). Can we add them please?

I enabled it on cedar. Once everything's ready to land on the core branches I'll enabled it there too.

Comment 7

[Andrew Halberstadt [:ahal]]

Panos, just a heads up, it looks like all of the breakpoint tests fail: https://tbpl.mozilla.org/php/getParsedLog.php?id=17688432&tree=Cedar

It may or may not be a harness error, it's hard for me to tell. For now I'll be disabling them.

Comment 8

[Andrew Halberstadt [:ahal]]

Attachment: Patch 2.0 - Disable tests which failed on cedar, add debugger tests

Comment 9

[Andrew Halberstadt [:ahal]]

https://hg.mozilla.org/mozilla-central/rev/352bd17710c8
https://hg.mozilla.org/releases/mozilla-aurora/rev/0a2521581d24
https://hg.mozilla.org/releases/mozilla-b2g18/rev/6f346e897dde

Comment 10

[Andrew Halberstadt [:ahal]]

The tests got enabled much quicker than I was expecting (before a merge from m-c -> inbound could happen).

I had to check this patch into inbound as well to get rid of the red:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b2b5567fe14a

Hope it doesn't cause merge problems :/

Comment 11

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/b2b5567fe14a