create webgl mochitest run for android

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: jmaher, Assigned: kmoir)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

(Reporter)

Description

4 years ago
this is a test suite that will apply to both an tegra and panda platforms.  It has been shown that removing the webgl conformance tests from the mochitest chunk 1 yields great success for mochitest chunk1.  But we don't want to lose test coverage, so lets create a M(GL).

This will be done like so:
python mochitest/runtestsremote.py --deviceIP 10.250.50.50 --xre-path ../hostutils/xre --utility-path ../hostutils/bin --certificate-path certs --app org.mozilla.fennec --console-level INFO --http-port 30140 --ssl-port 31140 --pidfile /builds/tegra-140/test/../runtestsremote.pid  --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android-armv6/1366828850/fennec-23.0a1.en-US.android-arm-armv6.crashreporter-symbols.zip --test-path=content/canvas/test/webgl/test_webgl_conformance_test_suite.html

we added the --test-path to the cli, but removed:
--total-chunks
--this-chunk
--run-only-tests=android.json

I added :ahal on the CC line to keep the b2g guys aware that we will be doing this and changing android.json.  

I added the 3 headed sheriff team to the CC line so they are aware of this and can help make the M(gl) option.

A side benefit is we will probably be able to unhide M(1) on panda after doing this.
(Reporter)

Comment 1

4 years ago
this should be able to run on all branches, although we would have to land android.json patches on all the branches.  So for now this would need to be selective based on branches.

Comment 2

4 years ago
What will the jobname be? (So I can add TBPL support)
mochitest-webgl ? Symbol 'gl' ?
(Assignee)

Comment 3

4 years ago
Joel, what specific branches should it run on now?
Assignee: nobody → kmoir
(Reporter)

Comment 4

4 years ago
all trunk based branches and aurora, basically not beta, release, esr :)
(Assignee)

Comment 5

4 years ago
I tried to implement this several ways but none of them worked.  How do you specify  --test-path in our configs?
(Assignee)

Comment 6

4 years ago
I tried 

 ('mochitest-gl', (
            {'suite': 'mochitest-plain',
             'testPath': 'content/canvas/test/webgl/test_webgl_conformance_test_suite.html',
             },
        )),

(my previous patch had 'mochitest-webgl' which also didn't work. )


but this didn't work either, testPath wasn't appended to the command, will do some more investigation

argv: ['python', 'mochitest/runtestsremote.py', '--deviceIP', '10.12.130.147', '--xre-path', '../hostutils/xre', '--utility-path', '../hostutils/bin', '--certificate-path', 'certs', '--app', 'org.mozilla.fennec', '--console-level', 'INFO', '--http-port', '30340', '--ssl-port', '31340', '--pidfile', '/builds/panda-0340/test/../runtestsremote.pid', '--symbols-path=http://stage.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-android/1366912171/fennec-23.0a1.en-US.android-arm.crashreporter-symbols.zip', "['--run-slower']"]
(Assignee)

Comment 7

4 years ago
Created attachment 745169 [details] [diff] [review]
patch for buildbot-configs
Attachment #745169 - Flags: review?(armenzg)
(Assignee)

Comment 8

4 years ago
Created attachment 745170 [details] [diff] [review]
patch for buildbotcustom

buildbotcustom wasn't parsing testPath properly and was looking for testPaths instead, this is why the custom path wasn't working
Attachment #745170 - Flags: review?(armenzg)
(Assignee)

Comment 9

4 years ago
Forgot to mention that I tested this on my dev-master, here's an example of the mochitest-gl test

http://dev-master01.build.scl1.mozilla.com:8036/builders/Android%204.0%20Panda%20mozilla-central%20opt%20test%20mochitest-gl/builds/4
Attachment #745169 - Flags: review?(armenzg) → review+
Comment on attachment 745170 [details] [diff] [review]
patch for buildbotcustom

Review of attachment 745170 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +5406,5 @@
>                      stepProc = unittest_steps.RemoteMochitestBrowserChromeStep
>                  else:
>                      stepProc = unittest_steps.RemoteMochitestStep
> +                if suite.get('testPath', None):
> +                    tp = suite.get('testPath', [])                   

Nit: some white spaces at the end of this line.
Attachment #745170 - Flags: review?(armenzg) → review+
(Assignee)

Updated

4 years ago
Attachment #745169 - Flags: checked-in+
(Assignee)

Comment 11

4 years ago
Comment on attachment 745170 [details] [diff] [review]
patch for buildbotcustom

and fixed whitespace
Attachment #745170 - Flags: checked-in+
In production.
(Reporter)

Comment 13

4 years ago
we need to make an adjustment here.  

we have:
--test-path=content/canvas/test/webgl/test_webgl_conformance_test_suite.html

we need:
--test-path=content/canvas/test/webgl

this will allow us to close the browser when we are done.
(Assignee)

Comment 14

4 years ago
Created attachment 745306 [details] [diff] [review]
patch to fix test-path

jmaher: I ran this test with the new path on my dev-master, is this what you expected. Not sure how to tell if the browser could be closed.
http://dev-master01.build.scl1.mozilla.com:8036/builders/Android%204.0%20Panda%20mozilla-central%20opt%20test%20mochitest-gl/builds/10/steps/mochitest-plain/logs/stdio
Attachment #745306 - Flags: review?(jmaher)
(Reporter)

Comment 15

4 years ago
Comment on attachment 745306 [details] [diff] [review]
patch to fix test-path

Review of attachment 745306 [details] [diff] [review]:
-----------------------------------------------------------------

thanks!  this will allow the test to run as a *suite* instead of a testcase.
Attachment #745306 - Flags: review?(jmaher) → review+
(Assignee)

Updated

4 years ago
Attachment #745306 - Flags: checked-in+
(Reporter)

Comment 16

4 years ago
Created attachment 745312 [details] [diff] [review]
disable webgl in the m1 test suite (1.0)
Attachment #745312 - Flags: review?(kmoir)
(Assignee)

Updated

4 years ago
Attachment #745312 - Flags: review?(kmoir) → review+
https://hg.mozilla.org/mozilla-central/rev/99b086e10c8d
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
I remember thinking that I needed to put a [leave open] in the whiteboard after I saw jmaher's push land, so m-cMerge wouldn't close the bug from it, but apparently thinking it doesn't actually cause it to happen.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
And btw, these jobs are now hidden all over everywhere that has triggered an Android build since yesterday, so good luck to us all finding and unhiding them once they're no longer permaorange.
(Reporter)

Comment 20

4 years ago
we just need a reconfig to get the tegra suite up and green, as for pandas...hmm.
in production

Updated

4 years ago
Depends on: 869365
(Assignee)

Comment 22

4 years ago
Since bug 869365 has been fixed and I can see it in tbpl, I think this bug can be closed again.
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Comment on attachment 745312 [details] [diff] [review]
disable webgl in the m1 test suite (1.0)

Pushed to beta in https://hg.mozilla.org/releases/mozilla-beta/rev/c29337e88494 since we're enabling Fennec-on-Panda one branch ahead of where this makes it possible for us to gaze upon mochitest-1 without being turned to stone.
Product: mozilla.org → Release Engineering
Blocks: 1060183
See Also: → bug 1051886
You need to log in before you can comment on or make changes to this bug.