Move emulator gaia-ui-tests on cedar to faster AWS nodes

RESOLVED WONTFIX

Status

Release Engineering
General Automation
RESOLVED WONTFIX
3 years ago
2 years ago

People

(Reporter: jgriffin, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2597] )

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

3 years ago
The emulator gaia-ui-tests currently running on cedar run too slowly to be useful; they are all timing out after 5 minutes of activity per test, since the VM they're run on isn't powerful enough to run them faster.

Let's experiment and move the cedar runs to IX hardware and see if they work well in that environment.
(Reporter)

Updated

3 years ago
Assignee: nobody → jgriffin
(Reporter)

Comment 1

3 years ago
Created attachment 8435077 [details] [diff] [review]
Schedule gaia-ui-test on emulator on ubuntu64 hw slaves on cedar,

I'm not sure we've ever run emlator tests on this config, so we'll see what happens.
Attachment #8435077 - Flags: review?(jlund)
Comment on attachment 8435077 [details] [diff] [review]
Schedule gaia-ui-test on emulator on ubuntu64 hw slaves on cedar,

Review of attachment 8435077 [details] [diff] [review]:
-----------------------------------------------------------------

oh boy, TIL it might be possible that mozilla-tests/b2g_config.py is harder to comprehend than mozilla-tests/config.py :)

so I think I'm up to speed. It looks like there is a small typo and I just have some follow up questions (see in-line)

::: mozilla-tests/b2g_config.py
@@ +1021,5 @@
>          'unittest-env': {'DISPLAY': ':0'},
>          'enable_opt_unittests': True,
>          'enable_debug_unittests': True,
> +        'ubuntu64_hw-b2g-emulator': {
> +            'opt_unittest_suites': []],

extra closing square bracket typo :)

@@ +1611,4 @@
>  BRANCHES['cedar']['platforms']['emulator']['ubuntu64_vm-b2g-emulator']['opt_unittest_suites'] = \
>      MOCHITEST + CRASHTEST + XPCSHELL + MARIONETTE + JSREFTEST + GAIA_UI
>  BRANCHES['cedar']['platforms']['emulator']['ubuntu64_vm-b2g-emulator']['debug_unittest_suites'] = MOCHITEST_EMULATOR_DEBUG[:] + REFTEST + CRASHTEST + MARIONETTE + XPCSHELL
> +BRANCHES['cedar']['platforms']['emulator']['ubuntu64_hw-b2g-emulator']['opt_unittest_suites'] = GAIA_UI

so if this is just for GAIA_UI, do we need to add the 'marionette-webapi' in the 'suite_config' above?

also, If these suites are timing out on the vm slaves, do we want to do disable them or are you wanting to run them on both for testing?
Attachment #8435077 - Flags: review?(jlund) → review-
(Reporter)

Comment 3

3 years ago
Created attachment 8435814 [details] [diff] [review]
Schedule gaia-ui-test on emulator on ubuntu64 hw slaves on cedar,

Addressed review comments; removed marionette-webapi from suite_config.  Since this is only running on cedar, I'd like to them side-by-side very temporarily so I can compare results, and assuming they run better on IX slaves, I'll turn off the vm runs.
Attachment #8435814 - Flags: review?(jlund)
(Reporter)

Updated

3 years ago
Attachment #8435077 - Attachment is obsolete: true
Comment on attachment 8435814 [details] [diff] [review]
Schedule gaia-ui-test on emulator on ubuntu64 hw slaves on cedar,

Review of attachment 8435814 [details] [diff] [review]:
-----------------------------------------------------------------

sgtm! :)
Attachment #8435814 - Flags: review?(jlund) → review+
(Reporter)

Comment 5

3 years ago
Comment on attachment 8435814 [details] [diff] [review]
Schedule gaia-ui-test on emulator on ubuntu64 hw slaves on cedar,

https://hg.mozilla.org/build/buildbot-configs/rev/38b85099ad1e
Attachment #8435814 - Flags: checked-in+
Live with reconfig on 2014-06-11 08:47 PT
(Reporter)

Comment 7

3 years ago
The runs on cedar are failing because they're being launched without a mozharness config:

========= Started '/tools/buildbot/bin/python scripts/scripts/marionette.py ...' failed (results: 2, elapsed: 0 secs) (at 2014-06-11 23:35:05.750262) =========
/tools/buildbot/bin/python scripts/scripts/marionette.py --blob-upload-branch cedar --download-symbols ondemand
 in dir /builds/slave/talos-slave/test/. (timeout 1800 secs) (maxTime 7200 secs)
 watching logfiles {}
 argv: ['/tools/buildbot/bin/python', 'scripts/scripts/marionette.py', '--blob-upload-branch', 'cedar', '--download-symbols', 'ondemand']
23:35:05     INFO - MultiFileLogger online at 20140611 23:35:05 in /builds/slave/talos-slave/test
23:35:05    FATAL - You need to specify a --marionette-address for non-emulator tests! (Try --marionette-address localhost:2828 )
23:35:05    FATAL - Running post_fatal callback...
23:35:05    FATAL - Exiting -1

https://tbpl.mozilla.org/php/getParsedLog.php?id=41582225&tree=Cedar&full=1

Jordan, any idea what's wrong here?
Flags: needinfo?(jlund)
(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> The runs on cedar are failing because they're being launched without a
> mozharness config:
> 
> ========= Started '/tools/buildbot/bin/python scripts/scripts/marionette.py
> ...' failed (results: 2, elapsed: 0 secs) (at 2014-06-11 23:35:05.750262)
> =========
> /tools/buildbot/bin/python scripts/scripts/marionette.py
> --blob-upload-branch cedar --download-symbols ondemand
>  in dir /builds/slave/talos-slave/test/. (timeout 1800 secs) (maxTime 7200
> secs)
>  watching logfiles {}
>  argv: ['/tools/buildbot/bin/python', 'scripts/scripts/marionette.py',
> '--blob-upload-branch', 'cedar', '--download-symbols', 'ondemand']
> 23:35:05     INFO - MultiFileLogger online at 20140611 23:35:05 in
> /builds/slave/talos-slave/test
> 23:35:05    FATAL - You need to specify a --marionette-address for
> non-emulator tests! (Try --marionette-address localhost:2828 )
> 23:35:05    FATAL - Running post_fatal callback...
> 23:35:05    FATAL - Exiting -1
> 
> https://tbpl.mozilla.org/php/getParsedLog.php?id=41582225&tree=Cedar&full=1
> 
> Jordan, any idea what's wrong here?

oops. we missed a typo in your patch:
    http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/b2g_config.py#1053
    s/suite_confg/suite_config/
Flags: needinfo?(jlund)
Created attachment 8439321 [details] [diff] [review]
140612_Bug_1019824_gaia_ui_hw-confg-typo-fix.patch
Attachment #8439321 - Flags: review?(jgriffin)
(Reporter)

Comment 10

3 years ago
Comment on attachment 8439321 [details] [diff] [review]
140612_Bug_1019824_gaia_ui_hw-confg-typo-fix.patch

Review of attachment 8439321 [details] [diff] [review]:
-----------------------------------------------------------------

Doh!  Thanks for spotting that.  I completely missed that typo even though I was staring at that code trying to figure out what was wrong.
(Reporter)

Updated

3 years ago
Attachment #8439321 - Flags: review?(jgriffin) → review+
Comment on attachment 8439321 [details] [diff] [review]
140612_Bug_1019824_gaia_ui_hw-confg-typo-fix.patch

np!

https://hg.mozilla.org/build/buildbot-configs/rev/2ee0d5c205fb
Attachment #8439321 - Flags: checked-in+
buildbot-config patch live in production :)
(Reporter)

Updated

3 years ago
Depends on: 1026802
(Reporter)

Updated

3 years ago
Depends on: 1031083
(Reporter)

Updated

3 years ago
Summary: Move emulator gaia-ui-tests on cedar from AWS to IX slaves → Move emulator gaia-ui-tests on cedar to faster AWS nodes
(Reporter)

Updated

3 years ago
Assignee: jgriffin → nobody

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2597]
TC should solve this.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.