Schedule mochitest-chrome on B2G emulators on cedar

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jgriffin, Assigned: jgriffin)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [ateam_harness_work])

Attachments

(4 attachments)

(Assignee)

Description

3 years ago
Now that gbrown has added support for B2G to mochitest-chrome, we can schedule these on try to allow us to green up the job.
(Assignee)

Comment 1

3 years ago
Armen, what's the method of scheduling something on try that won't get executed by default?
Flags: needinfo?(armenzg)
The runtestsb2g.py command needed here is the same as used currently for B2G mochitest-plain, with the addition of the --chrome option.
is there anything to green up?  My understanding is that we wanted the ability to write new tests, not necessarily run the existing tests.
(In reply to Joel Maher (:jmaher) from comment #3)
> is there anything to green up?  My understanding is that we wanted the
> ability to write new tests, not necessarily run the existing tests.

I think we should at least try to green up existing non-xul mochitest-chrome tests, even if it just means disabling the ones that are orange. It seems silly to just mark them all as failing without even running them.
(In reply to Jonathan Griffin (:jgriffin) from comment #1)
> Armen, what's the method of scheduling something on try that won't get
> executed by default?

I looked into this recently, and there's an easy config option for making builds non-default:
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679

But there doesn't seem to be any equivalent for test jobs :/.. I have a feeling we'll either need to block on releng to implement it, or try to fix it ourselves. Maybe Armen or someone in releng would know for sure.
(Assignee)

Comment 6

3 years ago
In that case we could use cedar for this instead.

Comment 7

3 years ago
(In reply to Andrew Halberstadt [:ahal] from comment #5)
> (In reply to Jonathan Griffin (:jgriffin) from comment #1)
> > Armen, what's the method of scheduling something on try that won't get
> > executed by default?
> 
> I looked into this recently, and there's an easy config option for making
> builds non-default:
> http://mxr.mozilla.org/build/source/buildbot-configs/mozilla/config.py#679
> 
> But there doesn't seem to be any equivalent for test jobs :/.. I have a
> feeling we'll either need to block on releng to implement it, or try to fix
> it ourselves. Maybe Armen or someone in releng would know for sure.

What ahal says is as far as I discovered.
It needs to implemented (filed as bug 1116275).
Flags: needinfo?(armenzg)
(Assignee)

Updated

3 years ago
Summary: Schedule mochitest-chrome on B2G emulators on try (non-default) → Schedule mochitest-chrome on B2G emulators on cedar

Comment 8

3 years ago
jgriffin said he'd be doing this.
Assignee: nobody → jgriffin
Status: NEW → ASSIGNED

Comment 9

3 years ago
It sounds like this is blocking bug 797164, not the other way around.
Blocks: 797164
No longer depends on: 797164
(Assignee)

Comment 10

3 years ago
Created attachment 8548495 [details] [diff] [review]
Add in-tree config for B2G mochitest-chrome,
Attachment #8548495 - Flags: review?(gbrown)
(Assignee)

Comment 11

3 years ago
Created attachment 8548496 [details] [diff] [review]
Add mochitest-chrome support to B2G unittest mozharness script,
Attachment #8548496 - Flags: review?(gbrown)
(Assignee)

Comment 12

3 years ago
Created attachment 8548497 [details] [diff] [review]
Schedule B2G mochitest-chrome on cedar,
Attachment #8548497 - Flags: review?(gbrown)

Updated

3 years ago
Attachment #8548497 - Flags: review?(gbrown) → review+

Updated

3 years ago
Attachment #8548495 - Flags: review?(gbrown) → review+

Updated

3 years ago
Attachment #8548496 - Flags: review?(gbrown) → review+
(Assignee)

Comment 13

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> Created attachment 8548495 [details] [diff] [review]
> Add in-tree config for B2G mochitest-chrome,

https://hg.mozilla.org/integration/mozilla-inbound/rev/111218ebc70b
(Assignee)

Updated

3 years ago
Keywords: leave-open
https://hg.mozilla.org/mozilla-central/rev/111218ebc70b
(Assignee)

Comment 15

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #11)
> Created attachment 8548496 [details] [diff] [review]
> Add mochitest-chrome support to B2G unittest mozharness script,

https://hg.mozilla.org/build/mozharness/rev/b08988b59a3a
Need to merge another mozharness change into production, so merging this one too. Should be live shortly...
In production: https://hg.mozilla.org/build/mozharness/rev/6415346ee811
(Assignee)

Comment 18

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #12)
> Created attachment 8548497 [details] [diff] [review]
> Schedule B2G mochitest-chrome on cedar,

https://hg.mozilla.org/build/buildbot-configs/rev/dfe3e4198247
In production: https://hg.mozilla.org/build/buildbot-configs/rev/dfe3e4198247
(Assignee)

Comment 20

3 years ago
Missed a minor change: https://hg.mozilla.org/build/mozharness/rev/1e74fd989cd5
(Assignee)

Comment 21

3 years ago
The tests are now launching on cedar, but failing with a bunch of script errors:

15:17:55     INFO -  JavaScript strict error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 61: ReferenceError: assignment to undeclared variable p
15:17:57     INFO -  JavaScript error: chrome://global/content/BrowserElementChildPreload.js, line 912: TypeError: content.document.body is undefined
15:17:59     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16     INFO -  JavaScript error: chrome://specialpowers/content/SpecialPowersObserver.js, line 103: NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]
15:18:16  WARNING -  TEST-UNEXPECTED-FAIL: setup.js | error parsing http://mochi.test:8888/tests.json (TypeError: RunSet is undefined)
15:18:16     INFO -  JavaScript error: chrome://mochikit/content/tests/SimpleTest/setup.js, line 253: TypeError: RunSet is undefined
15:23:49     INFO -  DeviceRunner TEST-UNEXPECTED-FAIL | mozrunner-startup | application timed out after 330.0 seconds with no output

The first happens because we're running in strict mode, and is easy to fix.
(Assignee)

Comment 22

3 years ago
Looks like this was already fixed in https://hg.mozilla.org/integration/mozilla-inbound/rev/8b6b57a1d6eb; just need another merge to cedar.

Comment 23

3 years ago
https://hg.mozilla.org/build/ash-mozharness/rev/dfcafb29161d
a mozharness patch has from this bug is in production
(Assignee)

Comment 25

3 years ago
It looks like the framework is working; disabling most chrome tests on B2G to try to get a green run:

https://hg.mozilla.org/projects/cedar/rev/dd51b56ae609
Whiteboard: [ateam_harness_work]
(Assignee)

Updated

3 years ago
Blocks: 1142315
(Assignee)

Comment 26

3 years ago
Created attachment 8576319 [details] [diff] [review]
Disable failing mochitest-chrome tests for B2G,

Disabling everything that lets achieve a green run on cedar.
Attachment #8576319 - Flags: review?(gbrown)

Updated

3 years ago
Attachment #8576319 - Flags: review?(gbrown) → review+
(Assignee)

Comment 27

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/83781ada01c3
https://hg.mozilla.org/mozilla-central/rev/83781ada01c3
(Assignee)

Updated

3 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.