Add {mochitest-gl,robocop,jsreftest,crashtest} suite_definitions to in-tree configurations

RESOLVED FIXED

Status

Release Engineering
Mozharness
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: nalexander, Assigned: armenzg)

Tracking

({leave-open})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [easier-mozharness])

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(18 attachments, 6 obsolete attachments)

761 bytes, patch
ahal
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
9.26 KB, patch
Details | Diff | Splinter Review
12.38 KB, patch
ahal
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
964 bytes, patch
chmanchester
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
5.15 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1.16 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
38 bytes, text/x-review-board-request
armenzg
: review+
Details | Review
(Reporter)

Description

3 years ago
In mozharness, we have robocop definitions [1] like:

289         "robocop-1": {
290             "category": "mochitest",
291             "extra_args": ["--total-chunks", "4", "--this-chunk", "1", "--robocop-path=../..",
292                 "--robocop-ids=fennec_ids.txt", "--robocop=robocop.ini"],
293         },

I'd like a "category": "robocop" that includes the "--robocop" arguments.  I think said categories come from the in tree "suite_definitions" defined at [2]; can somebody confirm that?  (ahal?)

This would make it easier to purge fennec_ids.txt from the tree and from automation.  It might also make it easier to configure Robocop.

It seems like the jsreftest definitions could use the same love.

In my opinion, the goal should be for the mozharness configurations that are *not* in tree to include only the chunking information, but perhaps this is not a reasonable state of affairs.

[1] http://mxr.mozilla.org/build/source/mozharness/configs/android/androidarm.py#289

[2] http://mxr.mozilla.org/mozilla-central/source/testing/config/mozharness/android_arm_config.py#7
(Reporter)

Updated

3 years ago
Flags: needinfo?(ahalberstadt)
Yeah, that looks like the file you want. The mozharness script uses the "category" value to find the proper directory, so we'd likely have to restructure how this works a bit (or else define 'abs_robocop_dir' = 'abs_mochitest_dir'):
http://mxr.mozilla.org/build/source/mozharness/scripts/android_emulator_unittest.py#387

Ideally I'd like everything (including chunking) defined in-tree. We have a vague notion that this is important and should be done, but haven't exactly nailed down where it belongs in Q4 goals.
Flags: needinfo?(ahalberstadt)
(Assignee)

Updated

3 years ago
Assignee: nobody → armenzg
Whiteboard: [easier-mozharness]
Blocks: 1067535
(Assignee)

Comment 2

3 years ago
The only difference between mochitest and robocop jobs are the ones defined in the mh config.
They grab common values from the "mochitest" category.

I think the best approach here is to create a "robocop" category in the in-tree config which defines:
--robocop-path=../.. --robocop-ids=fennec_ids.txt --robocop=robocop.ini
while the mochitest category defines:
--run-only-tests=android23.json

We would need to uplift the changes to every branch before landing the mozharness changes.

From mochitest-1:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-android/1415892129/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=16 --this-chunk=1 --run-only-tests=android23.json

From robocop-1:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-android/1415892129/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=4 --this-chunk=1 --robocop-path=../.. --robocop-ids=fennec_ids.txt --robocop=robocop.ini
(Assignee)

Comment 3

3 years ago
For xcpshell, we will have to land a different patch for mozilla-release's in-tree config.
We will have to add this:
--manifest=tests/xpcshell_android.ini

mc: http://hg.mozilla.org/mozilla-central/file/default/testing/config/mozharness/android_arm_config.py#l35
ma: http://hg.mozilla.org/releases/mozilla-aurora/file/9c775c8888ef/testing/config/mozharness/android_arm_config.py#l34
mb: http://hg.mozilla.org/releases/mozilla-beta/file/117eb4e49c72/testing/config/mozharness/android_arm_config.py#l35
mr: http://hg.mozilla.org/releases/mozilla-release/file/f2bdbc945da2/testing/config/mozharness/android_arm_config.py#l34

In m-c:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py --deviceIP=127.0.0.1 --devicePort=20701 --xre-path=/builds/slave/test/build/hostutils/xre --testing-modules-dir=/builds/slave/test/build/tests/modules --apk=/builds/slave/test/build/fennec-36.0a1.en-US.android-arm.apk --no-logfiles --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-android/1415892129/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --manifest=tests/xpcshell.ini --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=3 --this-chunk=1

In m-b:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py --deviceIP=127.0.0.1 --devicePort=20701 --xre-path=/builds/slave/test/build/hostutils/xre --testing-modules-dir=/builds/slave/test/build/tests/modules --apk=/builds/slave/test/build/fennec-34.0.en-US.android-arm.apk --no-logfiles --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-beta-android/1415891288/fennec-34.0.en-US.android-arm.crashreporter-symbols.zip --manifest=tests/xpcshell.ini --total-chunks=3 --this-chunk=1

In m-r:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py --deviceIP=127.0.0.1 --devicePort=20701 --xre-path=/builds/slave/test/build/hostutils/xre --testing-modules-dir=/builds/slave/test/build/tests/modules --apk=/builds/slave/test/build/fennec-33.1.en-US.android-arm.apk --no-logfiles --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-release-android/1415656612/fennec-33.1.en-US.android-arm.crashreporter-symbols.zip --total-chunks=3 --this-chunk=1 --manifest=tests/xpcshell_android.ini
(Assignee)

Comment 4

3 years ago
Created attachment 8522461 [details] [diff] [review]
[m-r] add android manifest
Attachment #8522461 - Flags: review?(ahalberstadt)
(Assignee)

Comment 5

3 years ago
Created attachment 8522475 [details]
MozReview Request: bz://1064002/mynick
Attachment #8522475 - Flags: review?(ahalberstadt)
(Assignee)

Comment 6

3 years ago
/r/601 - Bug 1064002 - Modify Android in-tree configs to allow further experimentation

Pull down this commit:

hg pull review -r 0ec9371ab0c8cf52c2f7bb47015570752bcf38f1
(Assignee)

Comment 7

3 years ago
Created attachment 8522477 [details]
MozReview Request: bz://1064002/mynick
(Assignee)

Comment 8

3 years ago
/r/605 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch

Pull down this commit:

hg pull review -r 2437947ec53662eef1969b32421761c991724532
(Assignee)

Comment 9

3 years ago
https://hg.mozilla.org/projects/ash/rev/2028827f51f1
(Assignee)

Comment 10

3 years ago
https://hg.mozilla.org/build/ash-mozharness/rev/7c397fe95f01
(Assignee)

Comment 11

3 years ago
Testing both patches in:
https://tbpl.mozilla.org/?tree=Ash&rev=2028827f51f1

Testing only the mc patch in:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=a0920d310649
Attachment #8522475 - Flags: review?(ahalberstadt) → review+
https://reviewboard.mozilla.org/r/599/#review287

This is awesome! I'm kind of surprised that adding '--total-chunks' works, but I guess you've tested on try. Is it possible to add '--total-chunks' to desktop unittests as well? If so is there a bug on file?
Comment on attachment 8522461 [details] [diff] [review]
[m-r] add android manifest

Review of attachment 8522461 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/config/mozharness/android_arm_config.py
@@ +31,5 @@
>              "options": ["--deviceIP=%(device_ip)s", "--devicePort=%(device_port)s",
>                  "--xre-path=%(xre_path)s", "--testing-modules-dir=%(modules_dir)s",
>                  "--apk=%(installer_path)s", "--no-logfiles",
>                  "--symbols-path=%(symbols_path)s",
> +                "--manifest=tests/xpcshell_android.ini",

So I removed xpcshell_android.ini awhile ago, but I think this makes sense because this is what the default is if it isn't defined in-tree. Just clarifying. Though, I'm surprised this wasn't defined already.
Attachment #8522461 - Flags: review?(ahalberstadt) → review+
(Assignee)

Comment 14

3 years ago
https://hg.mozilla.org/projects/ash/rev/37b99bf4a92e
(Assignee)

Comment 15

3 years ago
https://hg.mozilla.org/projects/ash/rev/3b9d91bee0e1
(Assignee)

Comment 16

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=a4f942c0fe7c
(Assignee)

Comment 17

3 years ago
I wonder what I enabled that it post my pushes to the bugs.

I found a syntax typo in my previous patch which I'm retesting on Ash and Try.

If all goes well, I will land the trunk changes and start uplifting.

Without the mozharness change:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=104e5feaa4ac

With the mozharness change:
https://tbpl.mozilla.org/?tree=Ash&rev=3b9d91bee0e1
(Assignee)

Comment 18

3 years ago
Comment on attachment 8522461 [details] [diff] [review]
[m-r] add android manifest

https://hg.mozilla.org/releases/mozilla-release/rev/85d4fdc6d572
Attachment #8522461 - Flags: checked-in+
(Assignee)

Comment 19

3 years ago
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #18)
> Comment on attachment 8522461 [details] [diff] [review]
> [m-r] add android manifest
> 
> https://hg.mozilla.org/releases/mozilla-release/rev/85d4fdc6d572

https://hg.mozilla.org/releases/mozilla-esr31/rev/f01ae373216d
(Assignee)

Comment 20

3 years ago
Created attachment 8523180 [details]
MozReview Request: bz://1064002/armenzg
(Assignee)

Comment 21

3 years ago
/r/637 - Bug 1064002 - Modify Android in-tree configs to allow further experimentation
/r/639 - Fix syntax error
/r/647 - Specify testsdir for each Android suite

Pull down these commits:

hg pull review -r 93d21fa78e0c785bdd9141c7b9dd8088cecc6826
(Assignee)

Comment 22

3 years ago
Created attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg
(Assignee)

Comment 23

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined

Pull down these commits:

hg pull review -r d9e78846c07c14d1b25ba6d9cb4c830b2903c7d1
(Assignee)

Comment 24

3 years ago
This is now working on Ash.
(Assignee)

Comment 25

3 years ago
/r/637 - Bug 1064002 - Modify Android in-tree configs to allow further experimentation
/r/639 - Fix syntax error
/r/647 - Specify testsdir for each Android suite

Pull down these commits:

hg pull review -r 93d21fa78e0c785bdd9141c7b9dd8088cecc6826
(Assignee)

Comment 26

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined

Pull down these commits:

hg pull review -r d9e78846c07c14d1b25ba6d9cb4c830b2903c7d1
(Assignee)

Comment 27

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined

Pull down these commits:

hg pull review -r d9e78846c07c14d1b25ba6d9cb4c830b2903c7d1
(Assignee)

Updated

3 years ago
Attachment #8523185 - Flags: review?(ahalberstadt)
(Assignee)

Comment 28

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined

Pull down these commits:

hg pull review -r d9e78846c07c14d1b25ba6d9cb4c830b2903c7d1
(Assignee)

Updated

3 years ago
Attachment #8523180 - Flags: review?(ahalberstadt)
(Assignee)

Comment 29

3 years ago
/r/637 - Bug 1064002 - Modify Android in-tree configs to allow further experimentation
/r/639 - Fix syntax error
/r/647 - Specify testsdir for each Android suite

Pull down these commits:

hg pull review -r 93d21fa78e0c785bdd9141c7b9dd8088cecc6826
(Assignee)

Comment 30

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined

Pull down these commits:

hg pull review -r d9e78846c07c14d1b25ba6d9cb4c830b2903c7d1
Attachment #8523185 - Flags: review?(jlund)
(Assignee)

Updated

3 years ago
Attachment #8522477 - Attachment is obsolete: true
(Assignee)

Comment 31

3 years ago
/r/637 - Bug 1064002 - Modify Android in-tree configs to allow further experimentation
/r/639 - Fix syntax error
/r/647 - Specify testsdir for each Android suite
/r/707 - Add testsdir to crashtests for Android configs

Pull down these commits:

hg pull review -r a2c5f56ec27599c063834fb5bada3a6a04f95afd
(Assignee)

Comment 32

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined
/r/709 - Fix typo

Pull down these commits:

hg pull review -r 204353fcab3cacfd17bc86b61252d99a3b14cdf8
https://reviewboard.mozilla.org/r/635/#review319

::: testing/config/mozharness/android_arm_config.py
(Diff revision 2)
> +                "--run-only-tests=android23.json"

nit: would be nice to add trailing commas to all the last list items just for consistency
Attachment #8523180 - Flags: review?(ahalberstadt) → review+
https://reviewboard.mozilla.org/r/635/#review321

Ship It!
Comment on attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg

This looks good to me, but I'd rather have Jordan do the final sign off.
Attachment #8523185 - Flags: review?(ahalberstadt) → feedback+
https://reviewboard.mozilla.org/r/649/#review371

looks good!
Attachment #8523185 - Flags: review?(jlund) → review+
https://reviewboard.mozilla.org/r/649/#review373
(Assignee)

Comment 38

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c4360f398ff
(Assignee)

Updated

3 years ago
Keywords: leave-open
(Assignee)

Updated

3 years ago
Attachment #8523185 - Flags: review?(jlund)
Attachment #8523185 - Flags: review?(ahalberstadt)
Attachment #8523185 - Flags: review+
Attachment #8523185 - Flags: feedback+
(Assignee)

Comment 39

3 years ago
/r/651 - Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
/r/653 - Read testsdir if it exists
/r/655 - Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined
/r/709 - Fix typo
/r/813 - Remove --manifest for xcpshell

Pull down these commits:

hg pull review -r b8daccc949472300d61bf33ec3cec39f6e99dc5f
(Assignee)

Comment 40

3 years ago
https://reviewboard.mozilla.org/r/813/#review375

Since I already landed:
https://hg.mozilla.org/releases/mozilla-release/rev/85d4fdc6d572
https://hg.mozilla.org/releases/mozilla-esr31/rev/f01ae373216d
https://hg.mozilla.org/mozilla-central/rev/7c4360f398ff
sorry had to back this out in https://hg.mozilla.org/mozilla-central/rev/7d17b594834f for causing Bug 1102115

Seems the problem is 

05:26 < vaibhav1994> Tomcat|sheriffduty: basically --test-manifest and --run-only-tests cannot be used together
Flags: needinfo?(armenzg)

Updated

3 years ago
Blocks: 1102115
(Assignee)

Comment 43

3 years ago
My apologies about that! I had tested it on try. I will review the issues.
(Assignee)

Updated

3 years ago
Flags: needinfo?(armenzg)
Attachment #8523185 - Flags: review?(ahalberstadt)
(Assignee)

Comment 44

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=efe17a8f2581
(Assignee)

Comment 45

3 years ago
It was on my try push as well. Unfortunately I though that the 1 and 2 that are green after they were confirming that it was an intermittent while they actually were different jobs.

Pushing a new patch to try:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=cec4080ae299

And to Ash for post-landing mozharness work:
https://tbpl.mozilla.org/?tree=Ash&rev=207a65587960
(Assignee)

Updated

3 years ago
Summary: Add {robocop,jsreftest} suite_definitions to in-tree configurations → Add {mochitest-gl,robocop,jsreftest,crashtest} suite_definitions to in-tree configurations
(Assignee)

Comment 46

3 years ago
The tests on try all passed, including the mochitest-gl ones running with the right arguments.
(Assignee)

Updated

3 years ago
Attachment #8523180 - Flags: review+ → review?(ahalberstadt)
(Assignee)

Comment 47

3 years ago
/r/637 - Bug 1064002 - We need to remove the parameters that are specific to the original suites until we land the mozharness changes
/r/639 - Add note with regards to not yet in use sections

Pull down these commits:

hg pull review -r bac5fac8730fdeb9495f13343e1a5e8303a43357
Attachment #8523180 - Flags: review?(ahalberstadt) → review+
https://reviewboard.mozilla.org/r/635/#review383

Thanks, looks good!

::: testing/config/mozharness/android_arm_config.py
(Diff revision 3)
> +        # Bug 1064002 - Not yet in use 

nit: these comments all have an extra whitespace
(Assignee)

Comment 49

3 years ago
/r/637 - Bug 1064002 - We need to remove the parameters that are specific to the original suites until we land the mozharness changes
/r/639 - Add note with regards to not yet in use sections
/r/829 - Fix extra white spaces

Pull down these commits:

hg pull review -r 1650517126a3729b378e09361dc584f21c69b39b
Attachment #8523180 - Flags: review+ → review?(ahalberstadt)
(Assignee)

Comment 50

3 years ago
Comment on attachment 8523180 [details]
MozReview Request: bz://1064002/armenzg

Recuperating missing r+
Attachment #8523180 - Flags: review?(ahalberstadt) → review+
(Assignee)

Updated

3 years ago
Attachment #8523180 - Flags: review+ → review?(ahalberstadt)
(Assignee)

Comment 51

3 years ago
/r/637 - Bug 1064002 - We need to remove the parameters that are specific to the original suites until we land the mozharness changes
/r/639 - Add note with regards to not yet in use sections
/r/829 - Fix extra white spaces
/r/831 - Fix extra white spaces

Pull down these commits:

hg pull review -r 5ac226783cfa830a8db44f00fbc99a5e86f4c6c7
(Assignee)

Comment 52

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/6a381fc27af8
https://hg.mozilla.org/mozilla-central/rev/6a381fc27af8
(Assignee)

Comment 54

3 years ago
Uplifted:
https://hg.mozilla.org/releases/mozilla-aurora/rev/e0d8c2ae3c4e
https://hg.mozilla.org/releases/mozilla-beta/rev/cb637f8504a8
https://hg.mozilla.org/releases/mozilla-release/rev/a47b1b720b67
https://hg.mozilla.org/releases/mozilla-esr31/rev/0be79c17ef59
(Assignee)

Comment 55

3 years ago
Comment on attachment 8523180 [details]
MozReview Request: bz://1064002/armenzg

Silly reviewboard.
Attachment #8523180 - Flags: review?(ahalberstadt) → review+
(Assignee)

Comment 56

3 years ago
Comment on attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg

I'm going to push a different mozharness review request.
Attachment #8523185 - Flags: review?(jlund)
(Assignee)

Comment 57

3 years ago
https://hg.mozilla.org/build/ash-mozharness/rev/b5a469957a88
(Assignee)

Comment 58

3 years ago
/r/651 - Remove --manifest for xcpshell
/r/653 - Bug 1064002 - Put back the original suites to its original values

Pull down these commits:

hg pull review -r e8c3575bce60231969152c9cc20e6e4607e059ba
Attachment #8523185 - Flags: review?(jlund)
(Assignee)

Comment 59

3 years ago
Created attachment 8526893 [details] [diff] [review]
Start using the proper categories for each suite (cannot land before attachment 8545428 [details] [diff] [review] and needs to be uplifted)

Once the mozharness bit lands this will follow.
I will need to test on try once the mozharness bits land.

Oh! how I long for pinning mozharness from the tree! :)
(Assignee)

Comment 60

3 years ago
Testing on Ash the mozharness changes and I can see it working.
https://tbpl.mozilla.org/?tree=Ash&jobname=Android&rev=1f6c9176557a

Crashtest job using the right parameters:
https://tbpl.mozilla.org/php/getParsedLog.php?id=53176580&tree=Ash&full=1
10:51:39     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/reftest/remotereftest.py --app=org.mozilla.fennec --ignore-window-size --bootstrap --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --httpd-path reftest/components --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ash-android/1416592334/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --total-chunks=2 tests/testing/crashtest/crashtests.list --this-chunk=1

From mozharness:
10:47:55     INFO -  'test_suite_definitions': {'crashtest-1': {'category': 'crashtest',
10:47:55     INFO -                                             'extra_args': ('--this-chunk=1',)},

From in-tree:
    80                 "--total-chunks=2",
    81                 "tests/testing/crashtest/crashtests.list",
    82             ],
    83         },


Reftest job using the right parameters:
https://tbpl.mozilla.org/php/getParsedLog.php?id=53176993&tree=Ash&full=1
10:51:40     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/reftest/remotereftest.py --app=org.mozilla.fennec --ignore-window-size --bootstrap --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --httpd-path reftest/components --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ash-android/1416592334/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --total-chunks=16 --this-chunk=1 tests/layout/reftests/reftest.list

From mozharness:
10:48:00     INFO -                             'reftest-1': {'category': 'reftest',
10:48:00     INFO -                                           'extra_args': ('--total-chunks=16',
10:48:00     INFO -                                                          '--this-chunk=1',
10:48:00     INFO -                                                          'tests/layout/reftests/reftest.list')},

From in-tree:
    63                 "--symbols-path=%(symbols_path)s",
    64                 # Bug 1064002 - Land once mozharness changes land
    65                 #"--total-chunks=16",
    66                 #"tests/layout/reftests/reftest.list",
Attachment #8523185 - Flags: review?(jlund) → review+
https://reviewboard.mozilla.org/r/649/#review499

Ship It!
(Assignee)

Comment 62

3 years ago
https://hg.mozilla.org/build/mozharness/rev/96810ab51f0d
(Assignee)

Comment 63

3 years ago
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #62)
> https://hg.mozilla.org/build/mozharness/rev/96810ab51f0d

This change once merge into production will do the following.
- Every suite that is *not* mochitest, reftest and xpcshell will start reading their info from the tree instead of from mozharness for *every* branch

We now have to land another change that will switch mochitest, reftest and xpcshell.
It will require uplifts and then a mozharness change.
(Assignee)

Updated

3 years ago
Attachment #8522475 - Attachment is obsolete: true
(Assignee)

Comment 64

3 years ago
Comment on attachment 8523180 [details]
MozReview Request: bz://1064002/armenzg

This was reviewed by ahal in reviewboard. I don't know why it says that I reviewed it.
Attachment #8523180 - Flags: checked-in+
(Assignee)

Comment 65

3 years ago
Comment on attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg

Checked-in. Not yet in production.
Attachment #8523185 - Flags: checked-in+
In production: https://hg.mozilla.org/build/mozharness/rev/96810ab51f0d
Comment on attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg

and backed out :(

default: https://hg.mozilla.org/build/mozharness/rev/0049e8c8e2e9
production: https://hg.mozilla.org/build/mozharness/rev/cdbe77f88f8e

reason: robocop failure: http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-aurora-android/1416872493/mozilla-aurora_ubuntu64_vm_mobile_test-robocop-3-bm118-tests1-linux64-build48.txt.gz

looks like we still somewhere depend on abs_%s_dir being: mochitest || reftest || xpcshell

not sure why this test didn't show up in your tests.
Attachment #8523185 - Flags: checked-in+ → checked-in-
(Assignee)

Comment 68

3 years ago
https://hg.mozilla.org/build/ash-mozharness/rev/5130c2f01f06
(Assignee)

Comment 69

3 years ago
Well, it seems that with all the micro-commits and hg handling I did locally I ended up landing missing the changes to the script.

I'm re-testing on Ash (so far so good):
https://tbpl.mozilla.org/?tree=Ash&rev=6a32858f85ff
here's the missing part of the patch:
http://hg.mozilla.org/users/armenzg_mozilla.com/mozharness/rev/fc37f42c4b0d
(Assignee)

Comment 70

3 years ago
https://hg.mozilla.org/build/mozharness/rev/6c0f0bca4f54
(Assignee)

Comment 71

3 years ago
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #70)
> https://hg.mozilla.org/build/mozharness/rev/6c0f0bca4f54

I triaged all issues that could be related in https://tbpl.mozilla.org/?tree=Ash&rev=6a32858f85ff

Landing now the patch as originally intended.
(Assignee)

Comment 72

3 years ago
Comment on attachment 8526893 [details] [diff] [review]
Start using the proper categories for each suite (cannot land before attachment 8545428 [details] [diff] [review] and needs to be uplifted)

Once the mozharness change makes it into the production branch we can land this change.
Attachment #8526893 - Flags: review?(ahalberstadt)
(Assignee)

Comment 73

3 years ago
Created attachment 8529108 [details] [diff] [review]
Make reftest, mochitest and xpchsell read the in-tree configs (cannot land before attachment 8526893 [details] [diff] [review] is landed)

Once attachment 8226893 is landed and uplifted everywhere.
Attachment #8529108 - Flags: review?(jlund)
(Assignee)

Updated

3 years ago
No longer blocks: 1040383
(Assignee)

Comment 74

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=1c840a6a08d3
In production: https://hg.mozilla.org/build/mozharness/rev/6c0f0bca4f54
(Assignee)

Comment 76

3 years ago
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #74)
> https://tbpl.mozilla.org/?tree=Try&rev=1c840a6a08d3

The try push went well. Waiting for the reviews.
In production: https://hg.mozilla.org/build/mozharness/rev/6c0f0bca4f54
(Assignee)

Comment 78

3 years ago
<philor> uh oh, did anything that might have involved android mochitest-gl land in this morning's reconfig?
<philor> armenzg: could a side-effect of bug 1064002 have been "make android 2.3 mochitest-gl1 and -gl2 fail every single run on every tree"?
<armenzg> philor, let me check
<bhearsum> yes, it did
<philor> armenzg: https://treeherder.mozilla.org/logviewer.html#?job_id=4289120&repo=mozilla-inbound - successful run marked as failure

I'm re-triggering on trunk and Aurora to determine if it is due to the mozharness change:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=Android.*gl
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=Android.*gl

The only difference so far is the order of the arguments:
--test-manifest=gl.json --this-chunk=1
--this-chunk=1 --test-manifest=gl.json

gl-1 failing job:
http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1417182580/mozilla-inbound_ubuntu64_vm_mobile_test-mochitest-gl-1-bm118-tests1-linux64-build128.txt.gz
Command:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1417182580/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=2 --test-manifest=gl.json --this-chunk=1

gl-1 successful job:
https://tbpl.mozilla.org/php/getParsedLog.php?id=53753300&tree=Mozilla-Inbound&full=1

/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1417181261/fennec-36.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=2 --this-chunk=1 --test-manifest=gl.json
(Assignee)

Comment 79

3 years ago
Backed out:
http://hg.mozilla.org/build/mozharness/rev/b59011a6f670

Related with parsing issues:
http://dxr.mozilla.org/build:mozharness/source/mozharness/mozilla/testing/unittest.py#159
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #73)
> Created attachment 8529108 [details] [diff] [review]
> Make reftest, mochitest and xpchsell read the in-tree configs
> 
> Once attachment 8226893 is landed and uplifted everywhere.

sanity check. My interpretation is that previous dependent work on this had a regression and was backed out in comment 78. Are you still wanting a review on 8529108 or do you need to fix something first?
(Assignee)

Comment 81

3 years ago
Comment on attachment 8529108 [details] [diff] [review]
Make reftest, mochitest and xpchsell read the in-tree configs (cannot land before attachment 8526893 [details] [diff] [review] is landed)

I don't need the review right now. It is likely not to change but I will first land and test everything blocking this patch.
Attachment #8529108 - Flags: review?(jlund)
Comment on attachment 8526893 [details] [diff] [review]
Start using the proper categories for each suite (cannot land before attachment 8545428 [details] [diff] [review] and needs to be uplifted)

Review of attachment 8526893 [details] [diff] [review]:
-----------------------------------------------------------------

This is so awesome. Can't wait for all jobs to have in-tree defined chunks.
Attachment #8526893 - Flags: review?(ahalberstadt) → review+
(Assignee)

Comment 83

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=9cbe966d2b5f
(Assignee)

Comment 84

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=1af8b0766860
(Assignee)

Updated

3 years ago
Depends on: 1112200
(Assignee)

Comment 85

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=e017d8290036
(Assignee)

Comment 86

3 years ago
It seems that for some harnesses the order of parameters can be troublesome. [1]

This fails:
--total-chunks=2 --test-manifest=gl.json --this-chunk=1

This succeeds:
--total-chunks=2 --this-chunk=1 --test-manifest=gl.json

[1]
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=bd4f7fbc017b
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-bd4f7fbc017b/try-android-api-9/fennec-37.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/raw_structured_logs.log --total-chunks=2 --test-manifest=gl.json --this-chunk=1
(Assignee)

Comment 87

3 years ago
Created attachment 8538684 [details] [diff] [review]
add mochitest-gl information so we can parse

No parsing? No passing.
Attachment #8538684 - Flags: review?(jlund)
(Assignee)

Comment 88

3 years ago
https://tbpl.mozilla.org/?tree=Try&rev=472d3afc9c6b
Attachment #8538684 - Flags: review?(jlund) → review+
(Assignee)

Updated

3 years ago
Blocks: 1083347
(Assignee)

Comment 89

3 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ecdb1c21ae76
(Assignee)

Comment 90

3 years ago
Created attachment 8545428 [details] [diff] [review]
Original patch + add mochitest-gl as a parsing category

This is what my current try push is testing.

If this works, then we will follow up with:
* attachment 8526893 [details] [diff] [review] - uplift everywhere
* attachment 8529108 [details] [diff] [review]
Attachment #8538684 - Attachment is obsolete: true
(Assignee)

Comment 91

3 years ago
I hit a hiccup since I merged code from default and it had issues (bug 1109346).
I'm re-triggering few jobs to see where we stand.
(Assignee)

Comment 92

3 years ago
Comment on attachment 8523185 [details]
MozReview Request: bz://1064002/armenzg

This patch does not show properly on MozReview.
I will re-attach.
Attachment #8523185 - Attachment is obsolete: true
(Assignee)

Comment 93

3 years ago
Comment on attachment 8545428 [details] [diff] [review]
Original patch + add mochitest-gl as a parsing category

This has worked on Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ecdb1c21ae76&filter-searchStr=gl1

ahal, this patch is the combination of https://reviewboard.mozilla.org/r/649 and attachment 8538684 [details] [diff] [review] which already have been r+, however, MozReview does not show it clearly (probably a bug) and would like to get your approval one more time before landing it.

I will not land this before I see all remaining jobs running green, however, I'm confident of it after seeing that gl1 job run properly.
Attachment #8545428 - Attachment description: wip - android configs → Original patch + add mochitest-gl as a parsing category
Attachment #8545428 - Flags: review?(ahalberstadt)
(Assignee)

Updated

3 years ago
Attachment #8526893 - Attachment description: android_mc.diff.final → Start using the proper categories for each suite (cannot land before attachment 8545428 and needs to be uplifted)
(Assignee)

Updated

3 years ago
Attachment #8529108 - Attachment description: Make reftest, mochitest and xpchsell read the in-tree configs → Make reftest, mochitest and xpchsell read the in-tree configs (cannot land before attachment 8526893 is landed)
I just noticed this removed the raw log option from the in-tree android_arm_config.py for xpcshell, so we aren't getting logs from those jobs. Is there a patch pending here to put that back in?
Flags: needinfo?(armenzg)
Comment on attachment 8545428 [details] [diff] [review]
Original patch + add mochitest-gl as a parsing category

Review of attachment 8545428 [details] [diff] [review]:
-----------------------------------------------------------------

Lgtm!
Attachment #8545428 - Flags: review?(ahalberstadt) → review+
(Assignee)

Comment 96

3 years ago
Created attachment 8546692 [details] [diff] [review]
xpcshell.diff

Not intentional.
It needs to be uplifted.
I can take care of it on Monday.
Flags: needinfo?(armenzg)
Attachment #8546692 - Flags: review?(cmanchester)
Comment on attachment 8546692 [details] [diff] [review]
xpcshell.diff

Review of attachment 8546692 [details] [diff] [review]:
-----------------------------------------------------------------

Great, thanks!
Attachment #8546692 - Flags: review?(cmanchester) → review+
(Assignee)

Comment 98

3 years ago
https://hg.mozilla.org/build/mozharness/rev/8d9a05daf751
(Assignee)

Updated

3 years ago
Attachment #8545428 - Flags: checked-in+
(Assignee)

Comment 99

3 years ago
https://hg.mozilla.org/integration/fx-team/rev/a13aa1af5d75
(Assignee)

Updated

3 years ago
Attachment #8546692 - Flags: checked-in+
In production: https://hg.mozilla.org/build/mozharness/rev/8d9a05daf751
https://hg.mozilla.org/mozilla-central/rev/a13aa1af5d75
(Assignee)

Comment 102

3 years ago
Already on mozilla-aurora from the merge day:
https://hg.mozilla.org/releases/mozilla-beta/rev/93587eeda731
https://hg.mozilla.org/releases/mozilla-release/rev/d6308b922ec7
https://hg.mozilla.org/releases/mozilla-esr31/rev/97610709b8af
(Assignee)

Comment 103

3 years ago
Created attachment 8547812 [details] [diff] [review]
Start using the proper categories for each suite (cannot land before attachment 8545428 [details] [diff] [review] and needs to be uplifted)

Carrying forward the r+.
Attachment #8547812 - Flags: review+
(Assignee)

Comment 104

3 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=987985e5c936
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #102)
> https://hg.mozilla.org/releases/mozilla-esr31/rev/97610709b8af

This appears to have broken xpcshell and robocop tests on esr31: https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr31&revision=97610709b8af&filter-searchStr=android%202.3%20armv6

Updated

3 years ago
Flags: needinfo?(armenzg)
(In reply to Wes Kocher (:KWierso) from comment #105)
> (In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from
> comment #102)
> > https://hg.mozilla.org/releases/mozilla-esr31/rev/97610709b8af
> 
> This appears to have broken xpcshell and robocop tests on esr31:
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> esr31&revision=97610709b8af&filter-searchStr=android%202.3%20armv6

Support for "--log-raw" merged to central in October, this should be backed out of the esr31.
Backed out from esr31 in https://hg.mozilla.org/releases/mozilla-esr31/rev/3590ef725e1d
(Assignee)

Comment 108

3 years ago
Thanks for fixing it. My apologies!
Flags: needinfo?(armenzg)
(Assignee)

Comment 109

3 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f970daecdc34
(Assignee)

Comment 110

3 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/03d2076110a3
(Assignee)

Comment 111

3 years ago
Comment on attachment 8547812 [details] [diff] [review]
Start using the proper categories for each suite (cannot land before attachment 8545428 [details] [diff] [review] and needs to be uplifted)

We can uplift it after it merges to central.

All jobs are running properly:
Mochitest [1]
Mochitest-gl [2]
Robocop-1 [3]
Xpcshell [4]
Crashtest [5]
Jsreftest [6]
Reftest [7]

[1]
08:44:42     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/mochitest-14_raw.log --total-chunks=16 --run-only-tests=android23.json --this-chunk=14

[2]
10:44:54     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/mochitest-gl-1_raw.log --total-chunks=2 --test-manifest=gl.json --this-chunk=1

[3]
08:44:55     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtestsremote.py --autorun --close-when-done --dm_trans=sut --console-level=INFO --app=org.mozilla.fennec --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --certificate-path=/builds/slave/test/build/tests/certs --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/robocop-1_raw.log --total-chunks=4 --robocop-path=../.. --robocop-ids=fennec_ids.txt --robocop=robocop.ini --this-chunk=1

[4]
08:45:07     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py --deviceIP=127.0.0.1 --devicePort=20701 --xre-path=/builds/slave/test/build/hostutils/xre --testing-modules-dir=/builds/slave/test/build/tests/modules --apk=/builds/slave/test/build/fennec-38.0a1.en-US.android-arm.apk --no-logfiles --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --manifest=tests/xpcshell.ini --log-raw=/builds/slave/test/build/blobber_upload_dir/xpcshell-3_raw.log --total-chunks=3 --this-chunk=3

[5]
07:50:39     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/reftest/remotereftest.py --app=org.mozilla.fennec --ignore-window-size --bootstrap --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --httpd-path /builds/slave/test/build/tests/modules --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --total-chunks=2 tests/testing/crashtest/crashtests.list --this-chunk=1

[6]
07:50:16     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/reftest/remotereftest.py --app=org.mozilla.fennec --ignore-window-size --bootstrap --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --httpd-path /builds/slave/test/build/tests/modules --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip ../jsreftest/tests/jstests.list --total-chunks=6 --extra-profile-file=jsreftest/tests/user.js --this-chunk=1

[7]
07:50:10     INFO - Running on test-1 the command /builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/reftest/remotereftest.py --app=org.mozilla.fennec --ignore-window-size --bootstrap --remote-webserver=10.0.2.2 --xre-path=/builds/slave/test/build/hostutils/xre --utility-path=/builds/slave/test/build/hostutils/bin --deviceIP=127.0.0.1 --devicePort=20701 --http-port=8854 --ssl-port=4454 --httpd-path /builds/slave/test/build/tests/modules --symbols-path=https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/armenzg@mozilla.com-f970daecdc34/try-android-api-9/fennec-38.0a1.en-US.android-arm.crashreporter-symbols.zip --total-chunks=16 tests/layout/reftests/reftest.list --this-chunk=1
Attachment #8547812 - Flags: checked-in+
(Assignee)

Updated

3 years ago
Attachment #8526893 - Attachment is obsolete: true
(Assignee)

Comment 112

3 years ago
Created attachment 8548422 [details] [diff] [review]
robocop.diff

The last of the release jobs that is trying to use --log-raw.
Attachment #8548422 - Flags: review?(cmanchester)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Updated

3 years ago
Attachment #8548422 - Flags: review?(cmanchester)
Attachment #8548422 - Flags: review+
Attachment #8548422 - Flags: checked-in+
https://hg.mozilla.org/mozilla-central/rev/03d2076110a3
(Assignee)

Comment 122

2 years ago
Mass comment: With mozharness moving into the tree [1] and pinning of mozharness we can assume that anything left (if any) in this bug will be easily taken care of.

[1] http://jordan-lund.ghost.io/mozharness-steps-into-the-forest
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
(Assignee)

Comment 123

2 years ago
Created attachment 8618301 [details]
MozReview Request: Remove --manifest for xcpshell
Attachment #8523180 - Attachment is obsolete: true
Attachment #8618307 - Flags: review+
Attachment #8618308 - Flags: review+
Attachment #8618309 - Flags: review+
Attachment #8618310 - Flags: review+
Attachment #8618311 - Flags: review+
Attachment #8618312 - Flags: review+
(Assignee)

Comment 124

2 years ago
Created attachment 8618302 [details]
MozReview Request: Remove --manifest for xcpshell
(Assignee)

Comment 125

2 years ago
Created attachment 8618303 [details]
MozReview Request: Use suite_name instead of suite_category and fallback to suite_category if testsdir is not defined
(Assignee)

Comment 126

2 years ago
Created attachment 8618304 [details]
MozReview Request: Bug 1064002 - Put back the original suites to its original values
(Assignee)

Comment 127

2 years ago
Created attachment 8618305 [details]
MozReview Request: Fix typo
(Assignee)

Comment 128

2 years ago
Created attachment 8618306 [details]
MozReview Request: Bug 1064002 - Remove information of androidarm.py which exists in the tree. We can't land this w/o landing previous patch
(Assignee)

Comment 129

2 years ago
Comment on attachment 8523180 [details]
MozReview Request: bz://1064002/armenzg
(Assignee)

Comment 130

2 years ago
Created attachment 8618307 [details]
MozReview Request: Add testsdir to crashtests for Android configs
(Assignee)

Comment 131

2 years ago
Created attachment 8618308 [details]
MozReview Request: Specify testsdir for each Android suite
(Assignee)

Comment 132

2 years ago
Created attachment 8618309 [details]
MozReview Request: Bug 1064002 - We need to remove the parameters that are specific to the original suites until we land the mozharness changes
(Assignee)

Comment 133

2 years ago
Created attachment 8618310 [details]
MozReview Request: Add note with regards to not yet in use sections
(Assignee)

Comment 134

2 years ago
Created attachment 8618311 [details]
MozReview Request: Fix extra white spaces
(Assignee)

Comment 135

2 years ago
Created attachment 8618312 [details]
MozReview Request: Fix extra white spaces
You need to log in before you can comment on or make changes to this bug.