Closed Bug 1071310 Opened 5 years ago Closed 5 years ago

Schedule Talos (including e10s variants) on Holly

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(e10s+)

RESOLVED FIXED
Tracking Status
e10s + ---

People

(Reporter: jgriffin, Assigned: jgriffin)

References

Details

Attachments

(4 files, 1 obsolete file)

Will is getting talos to run with e10s in bug 1050706, so we want to schedule Talos tests (with and without e10s support) on Holly, where the rest of the e10s work is happening.
Attached patch Schedule Talos on Holly (obsolete) — Splinter Review
Assignee: nobody → jgriffin
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Builders added:
+ Rev4 MacOSX Snow Leopard 10.6 holly talos chromez
+ Rev4 MacOSX Snow Leopard 10.6 holly talos chromez-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos dromaeojs
+ Rev4 MacOSX Snow Leopard 10.6 holly talos dromaeojs-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos g1
+ Rev4 MacOSX Snow Leopard 10.6 holly talos g1-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos other_nol64
+ Rev4 MacOSX Snow Leopard 10.6 holly talos other_nol64-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos svgr
+ Rev4 MacOSX Snow Leopard 10.6 holly talos svgr-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos tp5o
+ Rev4 MacOSX Snow Leopard 10.6 holly talos tp5o-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos chromez
+ Rev5 MacOSX Mountain Lion 10.8 holly talos chromez-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos dromaeojs
+ Rev5 MacOSX Mountain Lion 10.8 holly talos dromaeojs-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos g1
+ Rev5 MacOSX Mountain Lion 10.8 holly talos g1-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos other_nol64
+ Rev5 MacOSX Mountain Lion 10.8 holly talos other_nol64-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos svgr
+ Rev5 MacOSX Mountain Lion 10.8 holly talos svgr-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos tp5o
+ Rev5 MacOSX Mountain Lion 10.8 holly talos tp5o-e10s
+ Ubuntu HW 12.04 holly talos chromez
+ Ubuntu HW 12.04 holly talos chromez-e10s
+ Ubuntu HW 12.04 holly talos dromaeojs
+ Ubuntu HW 12.04 holly talos dromaeojs-e10s
+ Ubuntu HW 12.04 holly talos g1
+ Ubuntu HW 12.04 holly talos g1-e10s
+ Ubuntu HW 12.04 holly talos other_nol64
+ Ubuntu HW 12.04 holly talos other_nol64-e10s
+ Ubuntu HW 12.04 holly talos svgr
+ Ubuntu HW 12.04 holly talos svgr-e10s
+ Ubuntu HW 12.04 holly talos tp5o
+ Ubuntu HW 12.04 holly talos tp5o-e10s
+ Ubuntu HW 12.04 x64 holly talos chromez
+ Ubuntu HW 12.04 x64 holly talos chromez-e10s
+ Ubuntu HW 12.04 x64 holly talos dromaeojs
+ Ubuntu HW 12.04 x64 holly talos dromaeojs-e10s
+ Ubuntu HW 12.04 x64 holly talos g1
+ Ubuntu HW 12.04 x64 holly talos g1-e10s
+ Ubuntu HW 12.04 x64 holly talos other_l64
+ Ubuntu HW 12.04 x64 holly talos other_l64-e10s
+ Ubuntu HW 12.04 x64 holly talos svgr
+ Ubuntu HW 12.04 x64 holly talos svgr-e10s
+ Ubuntu HW 12.04 x64 holly talos tp5o
+ Ubuntu HW 12.04 x64 holly talos tp5o-e10s
+ WINNT 6.2 holly talos chromez
+ WINNT 6.2 holly talos chromez-e10s
+ WINNT 6.2 holly talos dromaeojs
+ WINNT 6.2 holly talos dromaeojs-e10s
+ WINNT 6.2 holly talos g1
+ WINNT 6.2 holly talos g1-e10s
+ WINNT 6.2 holly talos other_nol64
+ WINNT 6.2 holly talos other_nol64-e10s
+ WINNT 6.2 holly talos svgr
+ WINNT 6.2 holly talos svgr-e10s
+ WINNT 6.2 holly talos tp5o
+ WINNT 6.2 holly talos tp5o-e10s
+ Windows 7 32-bit holly talos chromez
+ Windows 7 32-bit holly talos chromez-e10s
+ Windows 7 32-bit holly talos dromaeojs
+ Windows 7 32-bit holly talos dromaeojs-e10s
+ Windows 7 32-bit holly talos g1
+ Windows 7 32-bit holly talos g1-e10s
+ Windows 7 32-bit holly talos other_nol64
+ Windows 7 32-bit holly talos other_nol64-e10s
+ Windows 7 32-bit holly talos svgr
+ Windows 7 32-bit holly talos svgr-e10s
+ Windows 7 32-bit holly talos tp5o
+ Windows 7 32-bit holly talos tp5o-e10s
+ Windows 7 32-bit holly talos xperf
+ Windows 7 32-bit holly talos xperf-e10s
+ Windows XP 32-bit holly talos chromez
+ Windows XP 32-bit holly talos chromez-e10s
+ Windows XP 32-bit holly talos g1
+ Windows XP 32-bit holly talos g1-e10s
+ Windows XP 32-bit holly talos other_nol64
+ Windows XP 32-bit holly talos other_nol64-e10s
+ Windows XP 32-bit holly talos svgr
+ Windows XP 32-bit holly talos svgr-e10s
+ Windows XP 32-bit holly talos tp5o
+ Windows XP 32-bit holly talos tp5o-e10s
Attachment #8493425 - Flags: feedback?(jmaher)
The "-e10s" variants are exact copies of the non-e10s ones; we'd have to add the --setPref for the e10s variants in talos.json.  Alternately, we could do it here...I'm not sure which is preferred.
cool, this looks normal and good.  Would we want talos to have a --e10s flag instead of just passing a --set-pref.
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +1994,5 @@
> +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> +
> +    if s in branch['suites']:
> +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> +

this just doesn't seem right, but I don't have a better way of doing this.  Maybe we could get Callek to weigh in here.
Attachment #8493425 - Flags: feedback?(jmaher) → feedback?(bugspam.Callek)
(In reply to Joel Maher (:jmaher) from comment #5)
> Comment on attachment 8493425 [details] [diff] [review]
> Schedule Talos on Holly
> 
> Review of attachment 8493425 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: mozilla-tests/config.py
> @@ +1994,5 @@
> > +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> > +
> > +    if s in branch['suites']:
> > +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> > +
> 
> this just doesn't seem right, but I don't have a better way of doing this. 
> Maybe we could get Callek to weigh in here.

I'm basically just cloning all the Talos entries just for Holly, for e10s.  I agree, it's pretty weird, compared to the way that Talos suites are normally specified.

The alternate would be to add these all to the main SUITES definition, but set to disabled, then enable them just on Holly.
if we do this, lets make sure we have comments as to why we are doing this and the scope of how long it should live.
* will we continue to run talos on holly once this is done?
* should we go ahead and define e10s versions of all the jobs now instead of doing the copy dynamically?
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +1994,5 @@
> +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> +
> +    if s in branch['suites']:
> +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> +

This looks horrid, but I don't have the headspace this week for it, deflecting to <roll dice> ben whom I hope has a much better idea.
Attachment #8493425 - Flags: feedback?(bugspam.Callek) → feedback?(bhearsum)
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

I'm O.K. with this if we know it works without breaking anything else. However, I think this idea is safer:
> The alternate would be to add these all to the main SUITES definition, but
> set to disabled, then enable them just on Holly.
Attachment #8493425 - Flags: feedback?(bhearsum) → feedback+
I'll make a new patch which is more explicit.
I've made this more explicit; changed builders list is the same as comment #2.
Attachment #8493969 - Flags: review?(bhearsum)
Attachment #8493425 - Attachment is obsolete: true
Attachment #8493969 - Flags: review?(bhearsum) → review+
Comment on attachment 8493969 [details] [diff] [review]
Schedule Talos on Holly

https://hg.mozilla.org/build/buildbot-configs/rev/c0197e1d9397

If this goes live via a reconfig before the talos.json changes are live, the e10s variants will simply be copies of the regular ones, which I think is OK since these are just on holly.

We'll also need TBPL/Treeherder patches so that these show up correctly.
Attachment #8493969 - Flags: checked-in+
Blocks: 1071875
Blocks: 1071876
Reduced the number of suites we initially enable on Holly, per wlach:  https://hg.mozilla.org/build/buildbot-configs/rev/0cd5639ac300
This patch updates the talos version used by desktop to one that supports e10s. We'll need to add the --e10s parameter to the e10s talos jobs to actually make them use e10s.
Attachment #8496164 - Flags: review?(jmaher)
Comment on attachment 8496164 [details] [diff] [review]
Patch to talos.json

Review of attachment 8496164 [details] [diff] [review]:
-----------------------------------------------------------------

I would like to see this on inbound as well.
Attachment #8496164 - Flags: review?(jmaher) → review+
Attached patch Add e10s suitesSplinter Review
Just copies of the non-e10s suites, with the --e10s variable
Attachment #8496185 - Flags: review?(jmaher)
Comment on attachment 8496185 [details] [diff] [review]
Add e10s suites

Review of attachment 8496185 [details] [diff] [review]:
-----------------------------------------------------------------

not other, chromez, g1, dromaeo?
Attachment #8496185 - Flags: review?(jmaher) → review+
(In reply to Joel Maher (:jmaher) from comment #18)
> Comment on attachment 8496185 [details] [diff] [review]
> Add e10s suites
> 
> Review of attachment 8496185 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> not other, chromez, g1, dromaeo?

As discussed on irc, we'll add versions of those suites even though they won't be running initially (because they're busted w/ e10s enabled, see bug 1050706)

https://hg.mozilla.org/projects/holly/rev/c494f3eb39c3
Looks like we don't want these tests on WinXP, so disabling them.

Builders removed
- Windows XP 32-bit holly talos svgr-e10s
- Windows XP 32-bit holly talos tp5o-e10s
Attachment #8497603 - Flags: review?(jlund)
Are the current talos runs on holly e10s runs?
(In reply to Jim Mathies [:jimm] from comment #21)
> Are the current talos runs on holly e10s runs?

Those in the T-e10s brackets are, those in the regular T bracket are non-e10s.
Comment on attachment 8497603 [details] [diff] [review]
Disable e10s Talos tests on winxp,

Review of attachment 8497603 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm
Attachment #8497603 - Flags: review?(jlund) → review+
Working as intended; can file follow-ups for additional changes.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Merged to production, and deployed.
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.