Enable chunking for B2G ICS Emulator Debug XPCShell tests

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: Chris Manchester (Offline Jun 23-26), Assigned: ahal)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 3 obsolete attachments)

https://bugzilla.mozilla.org/show_bug.cgi?id=1033126#c13 explains the situation. I'm investigating the performance issue with structured logging, but it seems like we're going to need to chunk these sooner or later.
(Reporter)

Updated

3 years ago
Blocks: 1050827
(Reporter)

Comment 1

3 years ago
Created attachment 8472439 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

It looks like this needs something in mozharness as well, and I guess we should distinguish between debug and opt.

Who needs to sign off on this sort of thing?
Attachment #8472439 - Flags: feedback?(ahalberstadt)
(Reporter)

Updated

3 years ago
Assignee: nobody → cmanchester
Status: NEW → ASSIGNED
(Assignee)

Comment 2

3 years ago
Comment on attachment 8472439 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

Review of attachment 8472439 [details] [diff] [review]:
-----------------------------------------------------------------

Releng needs to review the buildbot-config patches, otherwise the sheriffs should be consulted too. I think this looks right, but it's been awhile since I've modified this file. Maybe rail would be a good person to ask for review?
Attachment #8472439 - Flags: feedback?(ahalberstadt) → feedback+
TBPL should Just Work (the chunking support is generic), so this shouldn't cause any issues on our end. I assume these'll be enabled on Cedar first, so we can always fix up any issues that do arise then.
(Reporter)

Comment 4

3 years ago
Created attachment 8472491 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

Hi rail, I'm trying to add chunking to debug xpcshell tests on b2g ics emulators. Please feel free to redirect review to someone more appropriate. My main question right now is whether there's a mozharness change to be made as well, and if so where to make it. Thank you.
Attachment #8472491 - Flags: review?(rail)
(Reporter)

Updated

3 years ago
Attachment #8472439 - Attachment is obsolete: true
Comment on attachment 8472491 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

Review of attachment 8472491 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, except a nit:

::: mozilla-tests/b2g_config.py
@@ +732,5 @@
>          'brand_name': 'Gecko',
>          'builds_before_reboot': 1,
>          'unittest-env': {'DISPLAY': ':0'},
>          'enable_opt_unittests': True,
> +        'enable_debug_unittests': True,       

This hunk is redundant, it adds trailing spaces only.
Attachment #8472491 - Flags: review?(rail) → review+
(Reporter)

Comment 6

3 years ago
Created attachment 8472502 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks on cedar.

This gets rid of whitespace changes and enables the chunking on cedar only.

r=rail
(Reporter)

Updated

3 years ago
Attachment #8472502 - Flags: review+
(Reporter)

Comment 7

3 years ago
Cedar only version:

https://hg.mozilla.org/build/buildbot-configs/rev/ad66388f3636
(Reporter)

Updated

3 years ago
Depends on: 1055014
(Assignee)

Comment 8

3 years ago
Created attachment 8479189 [details] [diff] [review]
Enable 2 chunk debug xpcshell across all branches

Four chunks adds too much setup overhead (an additional hour across all four chunks combined). But ~100 minute chunks is still too long. RyanVM wants us to at least split it into two chunks across all branches.
Attachment #8479189 - Flags: review?(rail)
Attachment #8479189 - Flags: review?(rail) → review+
(Assignee)

Updated

3 years ago
Component: General → General Automation
Product: Testing → Release Engineering
QA Contact: catlee
(Assignee)

Comment 9

3 years ago
Comment on attachment 8479189 [details] [diff] [review]
Enable 2 chunk debug xpcshell across all branches

https://hg.mozilla.org/build/buildbot-configs/rev/61b8ce3541cf
Attachment #8479189 - Flags: checked-in+
(Reporter)

Updated

3 years ago
Attachment #8472491 - Attachment is obsolete: true
(Reporter)

Updated

3 years ago
Attachment #8472502 - Attachment is obsolete: true
(Reporter)

Updated

3 years ago
No longer blocks: 1033126
https://hg.mozilla.org/build/buildbot-configs/rev/61b8ce3541cf is in production
https://tbpl.mozilla.org/php/getParsedLog.php?id=46916851&tree=Mozilla-B2g28-v1.3

21:47:59     INFO - Calling ['/builds/slave/test/build/venv/bin/python', 'runtestsb2g.py', '--adbpath=/builds/slave/test/build/emulator/b2g-distro/out/host/linux-x86/bin/adb', '--b2gpath=/builds/slave/test/build/emulator/b2g-distro', '--emulator=arm', '--logcat-dir=/builds/slave/test/build', '--manifest=tests/xpcshell_b2g.ini', '--use-device-libs', '--testing-modules-dir=/builds/slave/test/build/tests/modules', '--symbols-path=/builds/slave/test/build/symbols', '--busybox=/builds/slave/test/build/busybox', '--total-chunks=2', '--this-chunk=1'] with output_timeout 1000
21:47:59     INFO -  /builds/slave/test/build/venv/local/lib/python2.7/site-packages/mozrunner/utils.py:20: UserWarning: Module mozinfo was already imported from /builds/slave/test/build/tests/xpcshell/mozinfo.py, but /builds/slave/test/build/venv/lib/python2.7/site-packages is being added to sys.path
21:47:59     INFO -    import pkg_resources
21:52:52     INFO -  Cleaning files from previous run..
21:53:54     INFO -  Included file '/builds/slave/test/build/tests/xpcshell/tests/toolkit/devtools/debugger/tests/unit/xpcshell.ini' does not exist
21:53:54     INFO -  Traceback (most recent call last):
21:53:54     INFO -    File "runtestsb2g.py", line 205, in main
21:53:54     INFO -      **options.__dict__):
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 1294, in runTests
21:53:54     INFO -      self.buildTestList()
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 426, in buildTestList
21:53:54     INFO -      xpcshell.XPCShellTests.buildTestList(self)
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 777, in buildTestList
21:53:54     INFO -      self.chunkTests()
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 786, in chunkTests
21:53:54     INFO -      totalTests += len(self.alltests[dir])
21:53:54     INFO -  TypeError: list indices must be integers, not dict
Depends on: 1059806
(Reporter)

Updated

3 years ago
Assignee: cmanchester → ahalberstadt
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.