1116187 - Schedule mochitest-chrome on B2G emulators on cedar

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

Now that gbrown has added support for B2G to mochitest-chrome, we can schedule these on try to allow us to green up the job.

Comment 1

[Jonathan Griffin (:jgriffin)]

Armen, what's the method of scheduling something on try that won't get executed by default?

Comment 2

[Geoff Brown [:gbrown]]

The runtestsb2g.py command needed here is the same as used currently for B2G mochitest-plain, with the addition of the --chrome option.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

is there anything to green up?  My understanding is that we wanted the ability to write new tests, not necessarily run the existing tests.

Comment 4

[Bobby Holley (:bholley)]

(In reply to Joel Maher (:jmaher) from comment #3)
> is there anything to green up?  My understanding is that we wanted the
> ability to write new tests, not necessarily run the existing tests.

I think we should at least try to green up existing non-xul mochitest-chrome tests, even if it just means disabling the ones that are orange. It seems silly to just mark them all as failing without even running them.

Comment 5

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #1)
> Armen, what's the method of scheduling something on try that won't get
> executed by default?

I looked into this recently, and there's an easy config option for making builds non-default:
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679

But there doesn't seem to be any equivalent for test jobs :/.. I have a feeling we'll either need to block on releng to implement it, or try to fix it ourselves. Maybe Armen or someone in releng would know for sure.

Comment 6

[Jonathan Griffin (:jgriffin)]

In that case we could use cedar for this instead.

Comment 7

[Armen [:armenzg]]

(In reply to Andrew Halberstadt [:ahal] from comment #5)
> (In reply to Jonathan Griffin (:jgriffin) from comment #1)
> > Armen, what's the method of scheduling something on try that won't get
> > executed by default?
> 
> I looked into this recently, and there's an easy config option for making
> builds non-default:
> http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679
> 
> But there doesn't seem to be any equivalent for test jobs :/.. I have a
> feeling we'll either need to block on releng to implement it, or try to fix
> it ourselves. Maybe Armen or someone in releng would know for sure.

What ahal says is as far as I discovered.
It needs to implemented (filed as bug 1116275).

Comment 8

[Mark Côté [:mcote]]

jgriffin said he'd be doing this.

Comment 9

[Mark Côté [:mcote]]

It sounds like this is blocking bug 797164, not the other way around.

Comment 10

[Jonathan Griffin (:jgriffin)]

Attachment: Add in-tree config for B2G mochitest-chrome,

Comment 11

[Jonathan Griffin (:jgriffin)]

Attachment: Add mochitest-chrome support to B2G unittest mozharness script,

Comment 12

[Jonathan Griffin (:jgriffin)]

Attachment: Schedule B2G mochitest-chrome on cedar,

Comment 13

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> Created attachment 8548495 [details] [diff] [review]
> Add in-tree config for B2G mochitest-chrome,

https://hg.mozilla.org/integration/mozilla-inbound/rev/111218ebc70b

Comment 14

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/111218ebc70b

Comment 15

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #11)
> Created attachment 8548496 [details] [diff] [review]
> Add mochitest-chrome support to B2G unittest mozharness script,

https://hg.mozilla.org/build/mozharness/rev/b08988b59a3a

Comment 16

[Pete Moore [:pmoore][:pete]]

Need to merge another mozharness change into production, so merging this one too. Should be live shortly...

Comment 17

[Pete Moore [:pmoore][:pete]]

In production: https://hg.mozilla.org/build/mozharness/rev/6415346ee811

Comment 18

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #12)
> Created attachment 8548497 [details] [diff] [review]
> Schedule B2G mochitest-chrome on cedar,

https://hg.mozilla.org/build/buildbot-configs/rev/dfe3e4198247

Comment 19

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/dfe3e4198247

Comment 20

[Jonathan Griffin (:jgriffin)]

Missed a minor change: https://hg.mozilla.org/build/mozharness/rev/1e74fd989cd5

Comment 21

[Jonathan Griffin (:jgriffin)]

The tests are now launching on cedar, but failing with a bunch of script errors:

15:17:55     INFO -  JavaScript strict error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 61: ReferenceError: assignment to undeclared variable p
15:17:57     INFO -  JavaScript error: chrome://global/content/BrowserElementChildPreload.js, line 912: TypeError: content.document.body is undefined
15:17:59     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16  WARNING -  TEST-UNEXPECTED-FAIL: setup.js | error parsing http://mochi.test:8888/tests.json (TypeError: RunSet is undefined)
15:18:16     INFO -  JavaScript error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 253: TypeError: RunSet is undefined
15:23:49     INFO -  DeviceRunner TEST-UNEXPECTED-FAIL | mozrunner-startup | application timed out after 330.0 seconds with no output

The first happens because we're running in strict mode, and is easy to fix.

Comment 22

[Jonathan Griffin (:jgriffin)]

Looks like this was already fixed in https://hg.mozilla.org/integration/mozilla-inbound/rev/8b6b57a1d6eb; just need another merge to cedar.

Comment 23

[Armen [:armenzg]]

https://hg.mozilla.org/build/ash-mozharness/rev/dfcafb29161d

Comment 24

[Jordan Lund (:jlund)]

a mozharness patch has from this bug is in production

Comment 25

[Jonathan Griffin (:jgriffin)]

It looks like the framework is working; disabling most chrome tests on B2G to try to get a green run:

https://hg.mozilla.org/projects/cedar/rev/dd51b56ae609

Comment 26

[Jonathan Griffin (:jgriffin)]

Attachment: Disable failing mochitest-chrome tests for B2G,

Disabling everything that lets achieve a green run on cedar.

Comment 27

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/83781ada01c3

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/83781ada01c3