724561 - cleanup after a side by side staging run for new talos test updates

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

our week+ of side by side staging is over and things are looking good.  Lets cleanup our configs and streamline the talos tests we run.

We need to:
 * disable chrome, nochrome, chrome_mac (default)
 * enable chrome.2, nochrome.2, chrome_mac.2 (default)
 * disable tp (default)
 * enable tp_responsiveness (default no_mac)
 * enable tpr_responsiveness (default mac_only)
 * turn off .2 and tpr tests for m-b/release/1.9.2/esr10

Also we want to:
 * add side by side staging for tp5r on linux and windows for m-c only.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: cleanup the side by side talos staging (1.0)

Comment 2

[Armen [:armenzg]]

Comment on attachment 594718 [details] [diff] [review]
cleanup the side by side talos staging (1.0)

Review of attachment 594718 [details] [diff] [review]:
-----------------------------------------------------------------

I have to r- because of the name collision of "tpr_responsiveness" and the NO_MAC exception for mozilla-central. Clarifying it should be no issue.
Having a description at the beginning listing what are the changes you want to do made the review much easier. Thanks jmaher!

::: mozilla-tests/config.py
@@ +213,4 @@
>          'suites': GRAPH_CONFIG + ['--activeTests', 'tp5', '--mozAfterPaint'],
>          'options': (TALOS_TP_OPTS, ALL_PLATFORMS),
>      },
> +

extra white line.

@@ +251,1 @@
>      'tpr_responsiveness': {

I see "tpr_responsiveness" declared twice in this dictionary. Copy/paste issue?

@@ +1074,5 @@
>  BRANCHES['mozilla-central']['xperf_tests'] = (1, True, {}, WIN7_ONLY)
>  BRANCHES['mozilla-central']['tp_tests'] = (0, True, TALOS_TP_OPTS, ALL_PLATFORMS)
> +#side by side staging for RSS collection on linux and windows
> +BRANCHES['mozilla-central']['tpr_responsiveness_tests'] = (1, True, TALOS_TP_OPTS, NO_MAC)
> +

Should this be "tp_responsiveness" (instead of "tpr")?

You say prior to this:
"tpr_responsiveness" -> Mac only -> Enabled
which you overwrite for mozilla-central with "everything except" mac.

@@ +1098,5 @@
> +BRANCHES['mozilla-beta']['chrome_mac.2_tests'] = (0, True, {}, MAC_ONLY)
> +BRANCHES['mozilla-beta']['nochrome.2_tests'] = (0, True, {}, ALL_PLATFORMS)
> +BRANCHES['mozilla-beta']['chrome_tests'] = (1, True, {}, NO_MAC)
> +BRANCHES['mozilla-beta']['chrome_mac_tests'] = (1, True, {}, MAC_ONLY)
> +BRANCHES['mozilla-beta']['nochrome_tests'] = (1, True, {}, ALL_PLATFORMS)

These beta & release exceptions are better in "delete" sections like in http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/config.py#l1203 for merge days.

Comment 3

[Armen [:armenzg]]

Assigning based on patch work. Let me know if you want me to take this.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: cleanup the side by side talos staging (2.0)

* fixed the tpr_responsiveness_tests duplication, now I have tpr_responsiveness_tests for MAC_ONLY and tpr_responsiveness_other_tests for NO_MAC.
* added comments in the sections for deletion when firefox 12 makes it to the branches in question.
* removed extra newline

Comment 6

[Armen [:armenzg]]

Comment on attachment 595424 [details] [diff] [review]
cleanup the side by side talos staging (2.0)

Review of attachment 595424 [details] [diff] [review]:
-----------------------------------------------------------------

My mind was about to explode but this patch is actually correct to the dot :)

Comment 7

[Armen [:armenzg]]

When are we ready for switching this?

Is this the talos.zip that was landed for the .2 and tpr side by side runs?
http://build.mozilla.org/talos/zips/talos.bug721857.05f01e049452.zip
I will need to know once we mention it on the mailing lists.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

landed:
http://hg.mozilla.org/build/buildbot-configs/rev/6dddc55c434a

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

oops, we still need to deploy/reconfig.  This also requires the new talos.

Comment 10

[Armen [:armenzg]]

which new talos?

Comment 11

[Armen [:armenzg]]

Comment on attachment 595424 [details] [diff] [review]
cleanup the side by side talos staging (2.0)

Merged and live for the last 22 hours.

http://hg.mozilla.org/build/buildbot-configs/rev/7b538d2ef7fa

Comment 12

[Brian R. Bondy [:bbondy]]

Why don't we run ts_paint anymore?  Seems like we can now accidentally let through startup time regressions like in bug 699247.

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

a simple oversight, we had it running twice, and now we don't have it running at all.  This will be turned back on soon as we have a reconfig to do first thing next week for bug 727823.

Comment 14

[Brian R. Bondy [:bbondy]]

oh ok thanks, I thought it was a permanent change.

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

The zombie jobs leftover in the buildbot db from the disabling the tests have been cleaned up.

Comment 16

[Armen [:armenzg]]

*sigh*

Thanks nthomas. I will have it in mind today for another cleanup we're doing.