Schedule Talos (including e10s variants) on Holly

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: jgriffin, Assigned: jgriffin)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(e10s+)

Details

Attachments

(4 attachments, 1 obsolete attachment)

(Assignee)

Description

3 years ago
Will is getting talos to run with e10s in bug 1050706, so we want to schedule Talos tests (with and without e10s support) on Holly, where the rest of the e10s work is happening.
(Assignee)

Comment 1

3 years ago
Created attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly
(Assignee)

Updated

3 years ago
Assignee: nobody → jgriffin
(Assignee)

Comment 2

3 years ago
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Builders added:
+ Rev4 MacOSX Snow Leopard 10.6 holly talos chromez
+ Rev4 MacOSX Snow Leopard 10.6 holly talos chromez-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos dromaeojs
+ Rev4 MacOSX Snow Leopard 10.6 holly talos dromaeojs-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos g1
+ Rev4 MacOSX Snow Leopard 10.6 holly talos g1-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos other_nol64
+ Rev4 MacOSX Snow Leopard 10.6 holly talos other_nol64-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos svgr
+ Rev4 MacOSX Snow Leopard 10.6 holly talos svgr-e10s
+ Rev4 MacOSX Snow Leopard 10.6 holly talos tp5o
+ Rev4 MacOSX Snow Leopard 10.6 holly talos tp5o-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos chromez
+ Rev5 MacOSX Mountain Lion 10.8 holly talos chromez-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos dromaeojs
+ Rev5 MacOSX Mountain Lion 10.8 holly talos dromaeojs-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos g1
+ Rev5 MacOSX Mountain Lion 10.8 holly talos g1-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos other_nol64
+ Rev5 MacOSX Mountain Lion 10.8 holly talos other_nol64-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos svgr
+ Rev5 MacOSX Mountain Lion 10.8 holly talos svgr-e10s
+ Rev5 MacOSX Mountain Lion 10.8 holly talos tp5o
+ Rev5 MacOSX Mountain Lion 10.8 holly talos tp5o-e10s
+ Ubuntu HW 12.04 holly talos chromez
+ Ubuntu HW 12.04 holly talos chromez-e10s
+ Ubuntu HW 12.04 holly talos dromaeojs
+ Ubuntu HW 12.04 holly talos dromaeojs-e10s
+ Ubuntu HW 12.04 holly talos g1
+ Ubuntu HW 12.04 holly talos g1-e10s
+ Ubuntu HW 12.04 holly talos other_nol64
+ Ubuntu HW 12.04 holly talos other_nol64-e10s
+ Ubuntu HW 12.04 holly talos svgr
+ Ubuntu HW 12.04 holly talos svgr-e10s
+ Ubuntu HW 12.04 holly talos tp5o
+ Ubuntu HW 12.04 holly talos tp5o-e10s
+ Ubuntu HW 12.04 x64 holly talos chromez
+ Ubuntu HW 12.04 x64 holly talos chromez-e10s
+ Ubuntu HW 12.04 x64 holly talos dromaeojs
+ Ubuntu HW 12.04 x64 holly talos dromaeojs-e10s
+ Ubuntu HW 12.04 x64 holly talos g1
+ Ubuntu HW 12.04 x64 holly talos g1-e10s
+ Ubuntu HW 12.04 x64 holly talos other_l64
+ Ubuntu HW 12.04 x64 holly talos other_l64-e10s
+ Ubuntu HW 12.04 x64 holly talos svgr
+ Ubuntu HW 12.04 x64 holly talos svgr-e10s
+ Ubuntu HW 12.04 x64 holly talos tp5o
+ Ubuntu HW 12.04 x64 holly talos tp5o-e10s
+ WINNT 6.2 holly talos chromez
+ WINNT 6.2 holly talos chromez-e10s
+ WINNT 6.2 holly talos dromaeojs
+ WINNT 6.2 holly talos dromaeojs-e10s
+ WINNT 6.2 holly talos g1
+ WINNT 6.2 holly talos g1-e10s
+ WINNT 6.2 holly talos other_nol64
+ WINNT 6.2 holly talos other_nol64-e10s
+ WINNT 6.2 holly talos svgr
+ WINNT 6.2 holly talos svgr-e10s
+ WINNT 6.2 holly talos tp5o
+ WINNT 6.2 holly talos tp5o-e10s
+ Windows 7 32-bit holly talos chromez
+ Windows 7 32-bit holly talos chromez-e10s
+ Windows 7 32-bit holly talos dromaeojs
+ Windows 7 32-bit holly talos dromaeojs-e10s
+ Windows 7 32-bit holly talos g1
+ Windows 7 32-bit holly talos g1-e10s
+ Windows 7 32-bit holly talos other_nol64
+ Windows 7 32-bit holly talos other_nol64-e10s
+ Windows 7 32-bit holly talos svgr
+ Windows 7 32-bit holly talos svgr-e10s
+ Windows 7 32-bit holly talos tp5o
+ Windows 7 32-bit holly talos tp5o-e10s
+ Windows 7 32-bit holly talos xperf
+ Windows 7 32-bit holly talos xperf-e10s
+ Windows XP 32-bit holly talos chromez
+ Windows XP 32-bit holly talos chromez-e10s
+ Windows XP 32-bit holly talos g1
+ Windows XP 32-bit holly talos g1-e10s
+ Windows XP 32-bit holly talos other_nol64
+ Windows XP 32-bit holly talos other_nol64-e10s
+ Windows XP 32-bit holly talos svgr
+ Windows XP 32-bit holly talos svgr-e10s
+ Windows XP 32-bit holly talos tp5o
+ Windows XP 32-bit holly talos tp5o-e10s
Attachment #8493425 - Flags: feedback?(jmaher)
(Assignee)

Comment 3

3 years ago
The "-e10s" variants are exact copies of the non-e10s ones; we'd have to add the --setPref for the e10s variants in talos.json.  Alternately, we could do it here...I'm not sure which is preferred.

Updated

3 years ago
tracking-e10s: --- → +
cool, this looks normal and good.  Would we want talos to have a --e10s flag instead of just passing a --set-pref.
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +1994,5 @@
> +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> +
> +    if s in branch['suites']:
> +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> +

this just doesn't seem right, but I don't have a better way of doing this.  Maybe we could get Callek to weigh in here.
Attachment #8493425 - Flags: feedback?(jmaher) → feedback?(bugspam.Callek)
(Assignee)

Comment 6

3 years ago
(In reply to Joel Maher (:jmaher) from comment #5)
> Comment on attachment 8493425 [details] [diff] [review]
> Schedule Talos on Holly
> 
> Review of attachment 8493425 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: mozilla-tests/config.py
> @@ +1994,5 @@
> > +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> > +
> > +    if s in branch['suites']:
> > +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> > +
> 
> this just doesn't seem right, but I don't have a better way of doing this. 
> Maybe we could get Callek to weigh in here.

I'm basically just cloning all the Talos entries just for Holly, for e10s.  I agree, it's pretty weird, compared to the way that Talos suites are normally specified.

The alternate would be to add these all to the main SUITES definition, but set to disabled, then enable them just on Holly.
if we do this, lets make sure we have comments as to why we are doing this and the scope of how long it should live.
* will we continue to run talos on holly once this is done?
* should we go ahead and define e10s versions of all the jobs now instead of doing the copy dynamically?
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +1994,5 @@
> +        branch['%s-e10s_tests' % s] = deepcopy(branch[tests_key])
> +
> +    if s in branch['suites']:
> +        branch['suites']['%s-e10s' % s] = deepcopy(branch['suites'][s])
> +

This looks horrid, but I don't have the headspace this week for it, deflecting to <roll dice> ben whom I hope has a much better idea.
Attachment #8493425 - Flags: feedback?(bugspam.Callek) → feedback?(bhearsum)
Comment on attachment 8493425 [details] [diff] [review]
Schedule Talos on Holly

Review of attachment 8493425 [details] [diff] [review]:
-----------------------------------------------------------------

I'm O.K. with this if we know it works without breaking anything else. However, I think this idea is safer:
> The alternate would be to add these all to the main SUITES definition, but
> set to disabled, then enable them just on Holly.
Attachment #8493425 - Flags: feedback?(bhearsum) → feedback+
(Assignee)

Comment 10

3 years ago
I'll make a new patch which is more explicit.
(Assignee)

Comment 11

3 years ago
Created attachment 8493969 [details] [diff] [review]
Schedule Talos on Holly

I've made this more explicit; changed builders list is the same as comment #2.
Attachment #8493969 - Flags: review?(bhearsum)
(Assignee)

Updated

3 years ago
Attachment #8493425 - Attachment is obsolete: true
Attachment #8493969 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 12

3 years ago
Comment on attachment 8493969 [details] [diff] [review]
Schedule Talos on Holly

https://hg.mozilla.org/build/buildbot-configs/rev/c0197e1d9397

If this goes live via a reconfig before the talos.json changes are live, the e10s variants will simply be copies of the regular ones, which I think is OK since these are just on holly.

We'll also need TBPL/Treeherder patches so that these show up correctly.
Attachment #8493969 - Flags: checked-in+
(Assignee)

Updated

3 years ago
Blocks: 1071875
(Assignee)

Updated

3 years ago
Blocks: 1071876
(Assignee)

Comment 13

3 years ago
Reduced the number of suites we initially enable on Holly, per wlach:  https://hg.mozilla.org/build/buildbot-configs/rev/0cd5639ac300
Something here landed in production today: https://wiki.mozilla.org/ReleaseEngineering/Maintenance#Reconfigs_.2F_Deployments
Created attachment 8496164 [details] [diff] [review]
Patch to talos.json

This patch updates the talos version used by desktop to one that supports e10s. We'll need to add the --e10s parameter to the e10s talos jobs to actually make them use e10s.
Attachment #8496164 - Flags: review?(jmaher)
Comment on attachment 8496164 [details] [diff] [review]
Patch to talos.json

Review of attachment 8496164 [details] [diff] [review]:
-----------------------------------------------------------------

I would like to see this on inbound as well.
Attachment #8496164 - Flags: review?(jmaher) → review+
Created attachment 8496185 [details] [diff] [review]
Add e10s suites

Just copies of the non-e10s suites, with the --e10s variable
Attachment #8496185 - Flags: review?(jmaher)
Comment on attachment 8496185 [details] [diff] [review]
Add e10s suites

Review of attachment 8496185 [details] [diff] [review]:
-----------------------------------------------------------------

not other, chromez, g1, dromaeo?
Attachment #8496185 - Flags: review?(jmaher) → review+
(In reply to Joel Maher (:jmaher) from comment #18)
> Comment on attachment 8496185 [details] [diff] [review]
> Add e10s suites
> 
> Review of attachment 8496185 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> not other, chromez, g1, dromaeo?

As discussed on irc, we'll add versions of those suites even though they won't be running initially (because they're busted w/ e10s enabled, see bug 1050706)

https://hg.mozilla.org/projects/holly/rev/c494f3eb39c3
(Assignee)

Comment 20

3 years ago
Created attachment 8497603 [details] [diff] [review]
Disable e10s Talos tests on winxp,

Looks like we don't want these tests on WinXP, so disabling them.

Builders removed
- Windows XP 32-bit holly talos svgr-e10s
- Windows XP 32-bit holly talos tp5o-e10s
Attachment #8497603 - Flags: review?(jlund)

Comment 21

3 years ago
Are the current talos runs on holly e10s runs?
(Assignee)

Comment 22

3 years ago
(In reply to Jim Mathies [:jimm] from comment #21)
> Are the current talos runs on holly e10s runs?

Those in the T-e10s brackets are, those in the regular T bracket are non-e10s.

Updated

3 years ago
Blocks: 1074992
Comment on attachment 8497603 [details] [diff] [review]
Disable e10s Talos tests on winxp,

Review of attachment 8497603 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm
Attachment #8497603 - Flags: review?(jlund) → review+
(Assignee)

Comment 24

3 years ago
Comment on attachment 8497603 [details] [diff] [review]
Disable e10s Talos tests on winxp,

https://hg.mozilla.org/build/buildbot-configs/rev/429bbbe97c6e
Attachment #8497603 - Flags: checked-in+
(Assignee)

Comment 25

3 years ago
Working as intended; can file follow-ups for additional changes.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Merged to production, and deployed.

Updated

2 years ago
Blocks: 1144120
You need to log in before you can comment on or make changes to this bug.