1243024 - (tracker) Make TC Linux64 *debug* test jobs tier1

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Armen [:armenzg]]

This depends on:
* bug 1171033 - Make the tests run side by side as tier2
   * This should be ready in the next 1/2 weeks
* bug 1242556 - Make the Linux64 debug build tier1
   * A workaround would be to use the Buildbot build with BBB (bug 1242023)
* bug 1218537: It's not possible to submit multiple "Job Info" artifacts
* bug 1241280: Fix web platform tests grouping

These issues are not specific to Linux64 tests, however, needed in general for sheriffing:
* Pending jobs reports for sherrifs [1]
* Integration under Treeherder -> Infra menu [2]

These are a matter to bring parity and cost saving:
* Coalescing
* SETA support

Pending reports are important for sheriffs to know when to report issues.
If there was a service which they can use to adjust thresholds and report the TaskCluster team automatically it would probably be ideal.
We could also fix the releng reporting tool to include taskcluster tasks if preferred.

[1]
http://builddata.pub.build.mozilla.org/reports/pending/pending.html
[2] http://people.mozilla.org/~armenzg/sattap/99d8ad71.png

Comment 1

[Armen [:armenzg]]

By tier1, we mean that we can shut off the Buildbot jobs because we've come to parity.

Some of the dependencies are general TC deficiencies rather than specific to debug test jobs in themselves.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

(In reply to Ryan VanderMeulen [:RyanVM] from bug 1250596 comment #3)
> Has anybody been able to confirm that the mochitest harness is able to take
> and upload screenshots on failure? A cursory look at TH at least gives off
> the impression that the URL isn't being TinderboxPrinted if it is.

Comment 3

[Armen [:armenzg]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #2)
> (In reply to Ryan VanderMeulen [:RyanVM] from bug 1250596 comment #3)
> > Has anybody been able to confirm that the mochitest harness is able to take
> > and upload screenshots on failure? A cursory look at TH at least gives off
> > the impression that the URL isn't being TinderboxPrinted if it is.

You have to click on the job, then on inspect task and then look under artifacts:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=tc&selectedJob=3380929
https://tools.taskcluster.net/task-inspector/#AscoNNTbT_Kwo_AzLOT58Q/0

This is bug 1218537 (in the dependencies)

Comment 4

[Armen [:armenzg]]

bug 992323 is to remove make check from builds.
Until then, end-to-end times for TaskCluster builds + tests is slowed down by make check tests.
It also has the side-effect of not publishing a task to the index (assuming that the task is marked as unsuccesful if the make tests fail).
I think we need to fix that as a separate issue.

Should we have a tracker for optimizations for the TC migration? I find bugs that are not necessarily blockers but need to be tracked.

Comment 5

[Chris AtLee [:catlee]]

How much time is this adding?

Comment 6

[Armen [:armenzg]]

I don't know. It used to be a lot. I don't know these days.

Comment 7

[Armen [:armenzg]]

I moved everything that should not block L64 as tier1 to bug 1080265.

Comment 8

[Armen [:armenzg]]

Attachment: Make Firefox TC Linux64 debug jobs tier1

Comment 9

[Armen [:armenzg]]

Attachment: MozReview Request: Bug 1243024 - Make Firefox TC Linux64 debug jobs tier1. r=jmaher

Review commit: https://reviewboard.mozilla.org/r/46221/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/46221/

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8741130 [details]
MozReview Request: Bug 1243024 - Make Firefox TC Linux64 debug jobs tier1. r=jmaher

https://reviewboard.mozilla.org/r/46221/#review42731

this is great; when we get opt, pgo, asan tests- we will have to have them off of a different base.

Comment 11

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/59c4fa834f30

Comment 12

[Pulsebot]

https://hg.mozilla.org/mozilla-central/rev/8630367f5e3f

Comment 13

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Turned buildbot-based jobs on Linux64 debug as tier-2 for trunk trees (including try, not including project branches), and everything so far seems to be working correctly. Feel free to mention any lingering buildbot jobs that aren't getting tier-2'd and someone can add them to the tier-2 list, but I think I got all of them.

I noticed bug 1264494 while I was creating that tier-2 list, and it will need someone to look at that, but I went ahead and tier-2'd the buildbot version of the static build, even though there's no TC equivalent.


Marking this bug as fixed, since comment 12 tier-1'd all of the TC jobs for linux64 debug.

Comment 14

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/59c4fa834f30

Comment 15

[Armen [:armenzg]]

If anyone believes we need this uplifted to other release branches please speak up.
Otherwise let the trains ride!