Break reftests into two chunks on linux64 debug

RESOLVED FIXED

Status

RESOLVED FIXED
4 years ago
5 months ago

People

(Reporter: nical, Assigned: nical)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

4 years ago
OMTC on Linux makes reftests run significantly slower, which makes us hit the 7200 seconds limit for debug 64bit builds (32bit being already split we don't hit the limit).
Interestingly, talos performance isn't affected like this, OMTC linux is actually nice for performance in most cases (especially if you watch videos on the web). So far, my investigations have pointed to X11 inefficiencies with reading back the content (which reftests do a lot) probably specific to driver versions since I can't reproduce the issue on my own computers.
We can't get rid of the X11/xrender drawing in the short term because of a long list of dependencies, but we want OMTC on linux soon.

I propose that we bite the bullet and split linux64 debug reftests in two so that we can land OMTC now, until we have finally removed our dependency to xrender.
(Assignee)

Comment 1

4 years ago
Created attachment 8573224 [details] [diff] [review]
Split reftests into two chunks on linux 64 for gecko 39+
Assignee: nobody → nical.bugzilla
Attachment #8573224 - Flags: review?(jlund)
Comment on attachment 8573224 [details] [diff] [review]
Split reftests into two chunks on linux 64 for gecko 39+

Review of attachment 8573224 [details] [diff] [review]:
-----------------------------------------------------------------

wow, you're brave touching config.py !!

This should get you what you want and I'll r+ this so you have the choice to land however I would recommend that you land this slowly like on a single project branch (cedar is our testing playground) first and then all of trunk. If there is bustage, this will require time from sheriffs and releng to backout and do another reconfig of buildbot. Since this is untested, there are a number of places this could go wrong: from the mozharness script to the actual test harness that mozharness calls. Granted we have already successfully split reftests with linux32, there could be hard coded bits specific to arch.

If you have tested this with mozharness already either locally on your machine or by doing a refest try push and messing with the mozharness repo to force a chunk, then we should just land this across all of trunk at once.

I see you found jgriffin's patch to land linux32 reftest split across 36: http://hg.mozilla.org/build/buildbot-configs/rev/847725a71b7e
for context, here was his initial patch testing on cedar: http://hg.mozilla.org/build/buildbot-configs/diff/4cefbd4f12be/mozilla-tests/config.py

::: mozilla-tests/config.py
@@ +2072,5 @@
> +# reftest is chunked on linux64 for gecko >= 39
> +for platform in PLATFORMS.keys():
> +    for name, branch in items_at_least(BRANCHES, 'gecko_version', 39):
> +        for slave_platform in PLATFORMS[platform]['slave_platforms']:
> +            if slave_platform not in ['ubuntu64_vm']:

parsing the top of this file for what platform will have 'ubuntu64_vm' I see:
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/config.py?rev=2bcf41d107df#176
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/config.py?rev=2bcf41d107df#201

but I also see we have a different name associated with asan:
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/config.py?rev=2bcf41d107df#190

It looks like we also do reftests for linux64 asan builds. did you want to split those too?
Attachment #8573224 - Flags: review?(jlund) → review+
(Assignee)

Comment 3

4 years ago
Created attachment 8573896 [details] [diff] [review]
Split reftests into two chunks on linux 64 debug for gecko 39+

Hi Jordan, thanks for the detailed review (very useful as I am discovering this stuff)!
On second thought, I think that it's sufficient for now do this with debug targets only (which leaves out asan build). Here's the updated patch.
Attachment #8573224 - Attachment is obsolete: true
Attachment #8573896 - Flags: review?(jlund)
(Assignee)

Comment 4

4 years ago
Created attachment 8573897 [details] [diff] [review]
Cedar patch

Following jgriffin's example, here's the patch to try it out on cedar first.
Attachment #8573897 - Flags: review?(jlund)
Comment on attachment 8573897 [details] [diff] [review]
Cedar patch

Review of attachment 8573897 [details] [diff] [review]:
-----------------------------------------------------------------

cool :)

I think we almost have it.

r+ if you adjust your patch to support in line comments.

::: mozilla-tests/config.py
@@ +1875,5 @@
>  BRANCHES['try']['platforms']['macosx64']['yosemite']['opt_unittest_suites'] = UNITTEST_SUITES['opt_unittest_suites'][:]
>  BRANCHES['try']['platforms']['macosx64']['yosemite']['debug_unittest_suites'] = UNITTEST_SUITES['debug_unittest_suites'][:]
>  
>  ######## cedar
> +BRANCHES['cedar']['platforms']['linux']['ubuntu64_vm']['debug_unittest_suites'] = \

it's confusing but 'linux' represents linux 32 arch while 'linux64' is for 64 arch.

iow - we want s/linux/linux64 here.

@@ +1876,5 @@
>  BRANCHES['try']['platforms']['macosx64']['yosemite']['debug_unittest_suites'] = UNITTEST_SUITES['debug_unittest_suites'][:]
>  
>  ######## cedar
> +BRANCHES['cedar']['platforms']['linux']['ubuntu64_vm']['debug_unittest_suites'] = \
> +    [x for x in BRANCHES['cedar']['platforms']['linux']['ubuntu64_vm']['debug_unittest_suites'] if x[0] != 'reftest'] + REFTEST_TWO_CHUNKS[:]

s/linux/linux64 here too
Attachment #8573897 - Flags: review?(jlund) → review+
Comment on attachment 8573896 [details] [diff] [review]
Split reftests into two chunks on linux 64 debug for gecko 39+

Review of attachment 8573896 [details] [diff] [review]:
-----------------------------------------------------------------

should work :)

let's land this after we land and try out the cedar patch
Attachment #8573896 - Flags: review?(jlund) → review+
(Assignee)

Comment 7

4 years ago
Updated the cedar patch and landed it: https://hg.mozilla.org/build/buildbot-configs/rev/7b7711ab6689
(Assignee)

Comment 8

4 years ago
Cedar is looking good.
(In reply to Nicolas Silva [:nical] from comment #9)
> Backed out the cedar-specific patch and landed the real one.
> 
> https://hg.mozilla.org/build/buildbot-configs/rev/fc52aa0ed8eb
> https://hg.mozilla.org/build/buildbot-configs/rev/d66f673350a3

\o/
(Assignee)

Updated

4 years ago
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.