Closed Bug 1356700 Opened 7 years ago Closed 7 years ago

Set MOZ_AUTOMATION on all taskcluster jobs

Categories

(Firefox Build System :: General, enhancement)

Product:
Firefox Build System
Component:
General
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox55 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla55
Tracking Flags:
Tracking Status
firefox55 --- fixed

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(3 files)

Bug 1356700 - Set MOZ_AUTOMATION on all taskcluster jobs.
59 bytes, text/x-review-board-request
dustin
: review+
Details
Bug 1356700 - Don't set MOZ_AUTOMATION from mozharness configurations.
59 bytes, text/x-review-board-request
mshal
: review+
Details
Bug 1356700 - Don't set MOZ_AUTOMATION individually for hazard and l10n builds.
59 bytes, text/x-review-board-request
mshal
: review+
Details 
In bug 1324998 comment 15, I wrote: Note, if it weren't for buildbot, I would push for MOZ_AUTOMATION to be set at the taskcluster level already.

Well, in fact, MOZ_AUTOMATION is set for all buildbot jobs since bug 1011352. I should have remembered that.

Multiple things in the tree assume that MOZ_AUTOMATION is set on all jobs, or most of them. There are a few places that test both MOZ_AUTOMATION and TASK_ID, but it's unconvenient to have to test two variables. Also, AIUI, TASK_ID is not set on Windows taskcluster builds.

So we should set MOZ_AUTOMATION everywhere, all the time.
Depends on: 1356708
Depends on: 1356709
Blocks: 1356683
Comment on attachment 8858438 [details]
Bug 1356700 - Set MOZ_AUTOMATION on all taskcluster jobs.

https://reviewboard.mozilla.org/r/130402/#review133180

::: taskcluster/taskgraph/transforms/task.py:960
(Diff revision 1)
>          attributes = task.get('attributes', {})
>          attributes['run_on_projects'] = task.get('run-on-projects', ['all'])
>  
> +        # Set MOZ_AUTOMATION on all jobs.
> +        payload = task_def.get('payload')
> +        if payload:

I think we need more conditions for this -- for example, scriptworker tasks do not have a `payloda.env` field.  I'm not sure whether scriptworker tasks will fail of such a field exists, but in any case we shouldn't be supplying one.

I suspect the best way to check is to look at worker.implementation -- if that's generic-worker, docker-engine, native-engine, or docker-worker, then set the env var.
Attachment #8858438 - Flags: review?(dustin) → review-
Comment on attachment 8858438 [details]
Bug 1356700 - Set MOZ_AUTOMATION on all taskcluster jobs.

https://reviewboard.mozilla.org/r/130402/#review133180

> I think we need more conditions for this -- for example, scriptworker tasks do not have a `payloda.env` field.  I'm not sure whether scriptworker tasks will fail of such a field exists, but in any case we shouldn't be supplying one.
> 
> I suspect the best way to check is to look at worker.implementation -- if that's generic-worker, docker-engine, native-engine, or docker-worker, then set the env var.

That sounds like something that will come bite us back later. Why don't all taskcluster workers take payload.env?
Comment on attachment 8858438 [details]
Bug 1356700 - Set MOZ_AUTOMATION on all taskcluster jobs.

https://reviewboard.mozilla.org/r/130402/#review133180

> That sounds like something that will come bite us back later. Why don't all taskcluster workers take payload.env?

Also, how does one trigger *everything* on try? Because nothing broke with this patch, but I guess no scriptworkers will run on try...
Depends on: 1356928
Blocks: 1356955
(In reply to Mike Hommey [:glandium] (VAC: Apr 20-May 4) from comment #5)
> That sounds like something that will come bite us back later. Why don't all
> taskcluster workers take payload.env?

What would it do with that value?

To my knowledge most of the nightly stuff (which is where scriptworker is used) can't run on try right now, although that's planned.
Comment on attachment 8858439 [details]
Bug 1356700 - Don't set MOZ_AUTOMATION from mozharness configurations.

https://reviewboard.mozilla.org/r/130404/#review133366

::: testing/mozharness/configs/builds/releng_sub_linux_configs/64_fastermake.py:1
(Diff revision 2)
> +import os

Is this file supposed to be part of a different commit? I don't see how the fastermake variant gets used anywhere.

::: testing/mozharness/configs/builds/releng_sub_linux_configs/64_fastermake.py:39
(Diff revision 2)
> +    'platform': 'linux64',
> +    'stage_platform': 'linux64',
> +    'publish_nightly_en_US_routes': False,
> +    'env': {
> +        'MOZBUILD_STATE_PATH': os.path.join(os.getcwd(), '.mozbuild'),
> +        'MOZ_AUTOMATION': '1',

Shouldn't this MOZ_AUTOMATION flag also go away?
Attachment #8858439 - Flags: review?(mshal) → review+
Comment on attachment 8858440 [details]
Bug 1356700 - Don't set MOZ_AUTOMATION individually for hazard and l10n builds.

https://reviewboard.mozilla.org/r/130406/#review133374
Attachment #8858440 - Flags: review?(mshal) → review+
Comment on attachment 8858438 [details]
Bug 1356700 - Set MOZ_AUTOMATION on all taskcluster jobs.

https://reviewboard.mozilla.org/r/130402/#review133400
Attachment #8858438 - Flags: review?(dustin) → review+
Comment on attachment 8858439 [details]
Bug 1356700 - Don't set MOZ_AUTOMATION from mozharness configurations.

https://reviewboard.mozilla.org/r/130404/#review133366

> Is this file supposed to be part of a different commit? I don't see how the fastermake variant gets used anywhere.

Looks like I had it in my tree from a long time ago and inadvertently git add'ed it.
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/26ed99596031
Set MOZ_AUTOMATION on all taskcluster jobs. r=dustin
https://hg.mozilla.org/integration/autoland/rev/4a30e8a325b8
Don't set MOZ_AUTOMATION from mozharness configurations. r=mshal
https://hg.mozilla.org/integration/autoland/rev/8b43c62ad836
Don't set MOZ_AUTOMATION individually for hazard and l10n builds. r=mshal
Yeah, forgot that this needs bug 1356928 to land first (or bug 1356708 and bug 1356709 to be fixed)
Flags: needinfo?(mh+mozilla)
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/3446cf5d9cce
Set MOZ_AUTOMATION on all taskcluster jobs. r=dustin
https://hg.mozilla.org/integration/autoland/rev/0682b34d2ab9
Don't set MOZ_AUTOMATION from mozharness configurations. r=mshal
https://hg.mozilla.org/integration/autoland/rev/a56275e2228b
Don't set MOZ_AUTOMATION individually for hazard and l10n builds. r=mshal
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: