Closed Bug 1311224 Opened 3 years ago Closed 3 years ago

Talos jobs don't get scheduled for artifact builds on try

Categories

(Testing :: Talos, defect)

defect
Not set

Tracking

(firefox53 fixed)

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: MattN, Assigned: maja_zf)

References

(Blocks 1 open bug, )

Details

Attachments

(1 file)

On my try push with the following syntax:
> try: -b o -p linux64 -u none -t svgr,tp5o,svgr-e10s,tp5o-e10s --rebuild-talos 5 --artifact

none of the talos jobs were scheduled. I had to manually add the jobs.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=77853207f5c8b5dc76dd7a127dfc2395b7733c24
Yes, |'enable_talos_sendchange': False| in the mozharness configs for artifact builds. Any reason *not* to schedule talos jobs against an artifact build?
Flags: needinfo?(cmanchester)
It guess it's possible we'll find small differences in results between artifact and non-artifact builds, but in general that should be fine.
Flags: needinfo?(cmanchester)
I have seen the same earlier today for Marionette tests as triggered via mach:

mach try -b d -p linux -u marionette-e10s -t none --artifact

Could this be related here and more test suites beside Talos are affected?

Leaving out the --artifact flag the tests are getting triggered automatically as expected.
(In reply to Henrik Skupin (:whimboo) from comment #3)
> I have seen the same earlier today for Marionette tests as triggered via
> mach:
> 
> mach try -b d -p linux -u marionette-e10s -t none --artifact
> 
> Could this be related here and more test suites beside Talos are affected?

No, the behaviour you observe with marionette-e10s is actually a regression. (Talos was never triggered by artifact builds in the first place, whereas other test-suites were set up to be triggered under --artifact with Bug 1302152.) 

I will investigate further and file a new bug if appropriate.
I set 'enable_talos_sendchange' to True for the artifact builds: 
* Talos jobs run fine on linux, linux64
* Talos is busted on win and mac with "unable to find changeset or repository". jmaher says we are missing this line from the mozversion output: "mozversion application_repository: https://hg.mozilla.org/try"

See https://treeherder.mozilla.org/#/jobs?repo=try&revision=d824bbb9de69ea2ac2193bf36c8b25ce332db08e&selectedJob=32201415

I propose to land scheduling of talos jobs under --artifact here, and then we can file a separate bug to fix this mozversion issue as part of greening up tests.
Assignee: nobody → mjzffr
Comment on attachment 8816545 [details]
Bug 1311224 - Talos jobs don't get scheduled for artifact builds on try;

https://reviewboard.mozilla.org/r/97252/#review97582

::: testing/mozharness/configs/builds/releng_sub_linux_configs/32_artifact.py
(Diff revision 1)
>      'tooltool_script': ["/builds/tooltool.py"],
>      'tooltool_bootstrap': "setup.sh",
>      'enable_count_ctors': True,
> -    'enable_talos_sendchange': False,
> -    # allows triggering of test jobs when --artifact try syntax is detected on buildbot
> -    'enable_unittest_sendchange': True,

Do we really want to remove `True` here? Are we inheriting that from another config after all?

::: testing/mozharness/configs/builds/releng_sub_windows_configs/32_debug_artifact.py:6
(Diff revision 1)
>  import os
>  import sys
>  
>  config = {
>      'base_name': 'WINNT_5.2_%(branch)s_Artifact_build',
> +    'enable_talos_sendchange': False,

I'm confused. I know Talos jobs aren't working on these platforms, but it sounds like the intent here is to turn these on everywhere and green them up in bug 1321872. Why are we turning them off here?
Attachment #8816545 - Flags: review?(cmanchester)
Comment on attachment 8816545 [details]
Bug 1311224 - Talos jobs don't get scheduled for artifact builds on try;

https://reviewboard.mozilla.org/r/97252/#review97582

> Do we really want to remove `True` here? Are we inheriting that from another config after all?

Judging by try pushes, we're inheriting. I'm fine with keeping it if you prefer.

> I'm confused. I know Talos jobs aren't working on these platforms, but it sounds like the intent here is to turn these on everywhere and green them up in bug 1321872. Why are we turning them off here?

I just added this to the _debug_ artifact configs. Judging by other build configs, Talos is intended to run on opt only?
Setting n-i for above replies since r? was cleared in MozReview.
Flags: needinfo?(cmanchester)
Comment on attachment 8816545 [details]
Bug 1311224 - Talos jobs don't get scheduled for artifact builds on try;

https://reviewboard.mozilla.org/r/97252/#review97582

> Judging by try pushes, we're inheriting. I'm fine with keeping it if you prefer.

Ok, in that case let's do this cleanup in a separate bug.

> I just added this to the _debug_ artifact configs. Judging by other build configs, Talos is intended to run on opt only?

That makes sense, but in that case aren't we getting the right values from the base config and/or we don't have debug talos builders to trigger here? I have no problem keeping this, I just can't tell why it's necessary.
Flags: needinfo?(cmanchester)
Comment on attachment 8816545 [details]
Bug 1311224 - Talos jobs don't get scheduled for artifact builds on try;

https://reviewboard.mozilla.org/r/97252/#review97582

> That makes sense, but in that case aren't we getting the right values from the base config and/or we don't have debug talos builders to trigger here? I have no problem keeping this, I just can't tell why it's necessary.

I tried this out and you're right: the right value is coming in from the base config, so this is unnecessary.
Comment on attachment 8816545 [details]
Bug 1311224 - Talos jobs don't get scheduled for artifact builds on try;

https://reviewboard.mozilla.org/r/97252/#review97938
Attachment #8816545 - Flags: review?(cmanchester) → review+
Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/79b6b6ff9a1b
Talos jobs don't get scheduled for artifact builds on try; r=chmanchester
https://hg.mozilla.org/mozilla-central/rev/79b6b6ff9a1b
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.