Enable chunking for B2G ICS Emulator Debug XPCShell tests



5 years ago
11 months ago


(Reporter: cmanchester, Assigned: ahal)


Firefox Tracking Flags

(Not tracked)



(1 attachment, 3 obsolete attachments)

https://bugzilla.mozilla.org/show_bug.cgi?id=1033126#c13 explains the situation. I'm investigating the performance issue with structured logging, but it seems like we're going to need to chunk these sooner or later.
It looks like this needs something in mozharness as well, and I guess we should distinguish between debug and opt.

Who needs to sign off on this sort of thing?
Attachment #8472439 - Flags: feedback?(ahalberstadt)
Assignee: nobody → cmanchester

Comment 2

5 years ago
Comment on attachment 8472439 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

Review of attachment 8472439 [details] [diff] [review]:

Releng needs to review the buildbot-config patches, otherwise the sheriffs should be consulted too. I think this looks right, but it's been awhile since I've modified this file. Maybe rail would be a good person to ask for review?
Attachment #8472439 - Flags: feedback?(ahalberstadt) → feedback+
TBPL should Just Work (the chunking support is generic), so this shouldn't cause any issues on our end. I assume these'll be enabled on Cedar first, so we can always fix up any issues that do arise then.
Hi rail, I'm trying to add chunking to debug xpcshell tests on b2g ics emulators. Please feel free to redirect review to someone more appropriate. My main question right now is whether there's a mozharness change to be made as well, and if so where to make it. Thank you.
Attachment #8472491 - Flags: review?(rail)
Attachment #8472439 - Attachment is obsolete: true

Comment 5

5 years ago
Comment on attachment 8472491 [details] [diff] [review]
Enable xpcshell tests on the b2g emulator in 4 chunks

Review of attachment 8472491 [details] [diff] [review]:

LGTM, except a nit:

::: mozilla-tests/b2g_config.py
@@ +732,5 @@
>          'brand_name': 'Gecko',
>          'builds_before_reboot': 1,
>          'unittest-env': {'DISPLAY': ':0'},
>          'enable_opt_unittests': True,
> +        'enable_debug_unittests': True,       

This hunk is redundant, it adds trailing spaces only.
Attachment #8472491 - Flags: review?(rail) → review+
This gets rid of whitespace changes and enables the chunking on cedar only.

Depends on: 1055014

Comment 8

5 years ago
Four chunks adds too much setup overhead (an additional hour across all four chunks combined). But ~100 minute chunks is still too long. RyanVM wants us to at least split it into two chunks across all branches.
Attachment #8479189 - Flags: review?(rail)


5 years ago
Attachment #8479189 - Flags: review?(rail) → review+


5 years ago
Component: General → General Automation
Product: Testing → Release Engineering
QA Contact: catlee

Comment 9

5 years ago
Comment on attachment 8479189 [details] [diff] [review]
Enable 2 chunk debug xpcshell across all branches

Attachment #8479189 - Flags: checked-in+
Attachment #8472491 - Attachment is obsolete: true
Attachment #8472502 - Attachment is obsolete: true
No longer blocks: 1033126

21:47:59     INFO - Calling ['/builds/slave/test/build/venv/bin/python', 'runtestsb2g.py', '--adbpath=/builds/slave/test/build/emulator/b2g-distro/out/host/linux-x86/bin/adb', '--b2gpath=/builds/slave/test/build/emulator/b2g-distro', '--emulator=arm', '--logcat-dir=/builds/slave/test/build', '--manifest=tests/xpcshell_b2g.ini', '--use-device-libs', '--testing-modules-dir=/builds/slave/test/build/tests/modules', '--symbols-path=/builds/slave/test/build/symbols', '--busybox=/builds/slave/test/build/busybox', '--total-chunks=2', '--this-chunk=1'] with output_timeout 1000
21:47:59     INFO -  /builds/slave/test/build/venv/local/lib/python2.7/site-packages/mozrunner/utils.py:20: UserWarning: Module mozinfo was already imported from /builds/slave/test/build/tests/xpcshell/mozinfo.py, but /builds/slave/test/build/venv/lib/python2.7/site-packages is being added to sys.path
21:47:59     INFO -    import pkg_resources
21:52:52     INFO -  Cleaning files from previous run..
21:53:54     INFO -  Included file '/builds/slave/test/build/tests/xpcshell/tests/toolkit/devtools/debugger/tests/unit/xpcshell.ini' does not exist
21:53:54     INFO -  Traceback (most recent call last):
21:53:54     INFO -    File "runtestsb2g.py", line 205, in main
21:53:54     INFO -      **options.__dict__):
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 1294, in runTests
21:53:54     INFO -      self.buildTestList()
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 426, in buildTestList
21:53:54     INFO -      xpcshell.XPCShellTests.buildTestList(self)
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 777, in buildTestList
21:53:54     INFO -      self.chunkTests()
21:53:54     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 786, in chunkTests
21:53:54     INFO -      totalTests += len(self.alltests[dir])
21:53:54     INFO -  TypeError: list indices must be integers, not dict
Depends on: 1059806
Assignee: cmanchester → ahalberstadt
Last Resolved: 4 years ago
Resolution: --- → FIXED
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.