Closed Bug 1116187 Opened 5 years ago Closed 5 years ago

Schedule mochitest-chrome on B2G emulators on cedar

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jgriffin, Assigned: jgriffin)

References

Details

(Whiteboard: [ateam_harness_work])

Attachments

(4 files)

Now that gbrown has added support for B2G to mochitest-chrome, we can schedule these on try to allow us to green up the job.
Armen, what's the method of scheduling something on try that won't get executed by default?
Flags: needinfo?(armenzg)
The runtestsb2g.py command needed here is the same as used currently for B2G mochitest-plain, with the addition of the --chrome option.
is there anything to green up?  My understanding is that we wanted the ability to write new tests, not necessarily run the existing tests.
(In reply to Joel Maher (:jmaher) from comment #3)
> is there anything to green up?  My understanding is that we wanted the
> ability to write new tests, not necessarily run the existing tests.

I think we should at least try to green up existing non-xul mochitest-chrome tests, even if it just means disabling the ones that are orange. It seems silly to just mark them all as failing without even running them.
(In reply to Jonathan Griffin (:jgriffin) from comment #1)
> Armen, what's the method of scheduling something on try that won't get
> executed by default?

I looked into this recently, and there's an easy config option for making builds non-default:
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679

But there doesn't seem to be any equivalent for test jobs :/.. I have a feeling we'll either need to block on releng to implement it, or try to fix it ourselves. Maybe Armen or someone in releng would know for sure.
In that case we could use cedar for this instead.
(In reply to Andrew Halberstadt [:ahal] from comment #5)
> (In reply to Jonathan Griffin (:jgriffin) from comment #1)
> > Armen, what's the method of scheduling something on try that won't get
> > executed by default?
> 
> I looked into this recently, and there's an easy config option for making
> builds non-default:
> http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679
> 
> But there doesn't seem to be any equivalent for test jobs :/.. I have a
> feeling we'll either need to block on releng to implement it, or try to fix
> it ourselves. Maybe Armen or someone in releng would know for sure.

What ahal says is as far as I discovered.
It needs to implemented (filed as bug 1116275).
Flags: needinfo?(armenzg)
Summary: Schedule mochitest-chrome on B2G emulators on try (non-default) → Schedule mochitest-chrome on B2G emulators on cedar
jgriffin said he'd be doing this.
Assignee: nobody → jgriffin
Status: NEW → ASSIGNED
It sounds like this is blocking bug 797164, not the other way around.
Blocks: 797164
No longer depends on: 797164
Attachment #8548497 - Flags: review?(gbrown) → review+
Attachment #8548495 - Flags: review?(gbrown) → review+
Attachment #8548496 - Flags: review?(gbrown) → review+
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> Created attachment 8548495 [details] [diff] [review]
> Add in-tree config for B2G mochitest-chrome,

https://hg.mozilla.org/integration/mozilla-inbound/rev/111218ebc70b
Keywords: leave-open
(In reply to Jonathan Griffin (:jgriffin) from comment #11)
> Created attachment 8548496 [details] [diff] [review]
> Add mochitest-chrome support to B2G unittest mozharness script,

https://hg.mozilla.org/build/mozharness/rev/b08988b59a3a
Need to merge another mozharness change into production, so merging this one too. Should be live shortly...
(In reply to Jonathan Griffin (:jgriffin) from comment #12)
> Created attachment 8548497 [details] [diff] [review]
> Schedule B2G mochitest-chrome on cedar,

https://hg.mozilla.org/build/buildbot-configs/rev/dfe3e4198247
The tests are now launching on cedar, but failing with a bunch of script errors:

15:17:55     INFO -  JavaScript strict error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 61: ReferenceError: assignment to undeclared variable p
15:17:57     INFO -  JavaScript error: chrome://global/content/BrowserElementChildPreload.js, line 912: TypeError: content.document.body is undefined
15:17:59     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16  WARNING -  TEST-UNEXPECTED-FAIL: setup.js | error parsing http://mochi.test:8888/tests.json (TypeError: RunSet is undefined)
15:18:16     INFO -  JavaScript error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 253: TypeError: RunSet is undefined
15:23:49     INFO -  DeviceRunner TEST-UNEXPECTED-FAIL | mozrunner-startup | application timed out after 330.0 seconds with no output

The first happens because we're running in strict mode, and is easy to fix.
Looks like this was already fixed in https://hg.mozilla.org/integration/mozilla-inbound/rev/8b6b57a1d6eb; just need another merge to cedar.
a mozharness patch has from this bug is in production
It looks like the framework is working; disabling most chrome tests on B2G to try to get a green run:

https://hg.mozilla.org/projects/cedar/rev/dd51b56ae609
Whiteboard: [ateam_harness_work]
Blocks: 1142315
Disabling everything that lets achieve a green run on cedar.
Attachment #8576319 - Flags: review?(gbrown)
Attachment #8576319 - Flags: review?(gbrown) → review+
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Keywords: leave-open
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.