Closed Bug 1354088 Opened 7 years ago Closed 4 years ago

Clean up env vars generated in mozharness OS X jobs running on generic worker

Categories

(Infrastructure & Operations :: RelOps: Posix OS, task, P3)

Product:
Infrastructure & Operations
Component:
RelOps: Posix OS
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: pmoore, Assigned: egao)

References

Details

Attachments

(1 file)

Bug 1354088 - clean up inherited environment variables
47 bytes, text/x-phabricator-request
Details | Review 
In bug 1349980, support was added for running mozharness test tasks on generic worker on OS X. At the time this was done, it was unknown exactly which env vars were required, so a list of env vars was copied over from a working environment.

We should whittle down this list, to reduce noise in the task definitions.
Pete, any updates here?
Flags: needinfo?(pmoore)
Catlee, I'm afraid not. Is this something I could hand over to releng?
Flags: needinfo?(pmoore)
I'm not sure that we have the context for this any more. But feel free to unassign yourself.
Priority: -- → P3
Assignee: pmoore → nobody
Thanks Chris!

The context is this list here, which can probably be reduced/removed.

https://hg.mozilla.org/mozilla-central/file/58c5151bfd62/taskcluster/taskgraph/transforms/job/mozharness_test.py#l230
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Hello,

I want to give it a shot with this, if anyone has extra info apart from what Pete said, is well welcomed.
Assignee: nobody → bcrisan

Unassigned myself - no update here, tried a few times but did not figure out where and how these env vars are used.

Assignee: bcrisan → nobody

@pmoore, is this something that now falls on relops? Maybe dhouse? Does it still apply?

Component: CIDuty → RelOps: Posix OS
Flags: needinfo?(pmoore)

It still applies. The idea is for someone to look at the macOS env vars set here and remove any which are not needed. Thanks!

Flags: needinfo?(pmoore) → needinfo?(dhouse)

I've started reviewing and testing removing some of these variables.

Assignee: nobody → dhouse
Flags: needinfo?(dhouse)

Note, it isn't particularly critical that env vars get removed, it may just be a little confusing for env vars to be set that are not for any reason needed. They probably don't do any harm, but for the benefit of clarity I created the bug...

reading through the history, vars came from the taskcluster_worker config here https://hg.mozilla.org/build/puppet/file/a5ba4999d9a5/modules/taskcluster_worker/templates/taskcluster-worker.yml.erb#l31

    env:
      extra:
        TMPDIR: {$env: TMPDIR}
        SHELL: '/bin/bash'
        LANG: 'en_US.UTF-8'
        LC_ALL: 'en_US.UTF-8'
        XPC_FLAGS: '0x0'
        XPC_SERVICE_NAME: '0'
        IDLEIZER_DISABLE_SHUTDOWN: 'true'
        PATH: "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

These were added in "Bug 1309197: upgrade taskcluster-worker version to 0.0.7 r=dustin" - https://hg.mozilla.org/build/puppet/rev/4860fb9968afd3d0a25dcfd5e6dd2aa26b195d2a (https://github.com/mozilla-releng/build-puppet/commit/e40b07a8cbd2bb199d2a5b8fa08e511d404ff1d3#diff-e71c10ccd4c2cee85fac20452be5ec72R26)

IDLEIZER_DISABLE_SHUTDOWN: buildbot specific re: https://bugzilla.mozilla.org/show_bug.cgi?id=1309197#c34

buildbot macos runner had explicit sets of IDLEIZER* and a PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin, and used TMPDIR for cleanup (so, expected also for tasks/other).

(In reply to Pete Moore [:pmoore][:pete] from comment #10)

Note, it isn't particularly critical that env vars get removed, it may just be a little confusing for env vars to be set that are not for any reason needed. They probably don't do any harm, but for the benefit of clarity I created the bug...

No problem. I won't make this a high priority.

If we can remove some/all of them from the transforms, then the tasks themselves will have the only environment variables required. ANd I'm guessing that will be less confusing for task writing and debugging.

Wow, Idelizer is a real blast from the past!

I removed the full env block for macos and ran the test-macosx1014 tasks on some staging mac minis: https://treeherder.mozilla.org/#/jobs?repo=try&revision=22324b79907a2445a589a820aa69deb52d9ebc5e (https://hg.mozilla.org/try/rev/f62f91caae6d604bb8d58a378a7c708061dad974)
There were some failures because of timeouts. I'll run those again against try to see the full set of failed tests.

14 test failures with the block removed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d8a3d2c02e2c90a7cc2cc333602465416803ef5b
but it looks intermittent

egao, could you look at this (in a few weeks)? If I'm reading the test results correctly, then removing these environment variables is not a problem.

Flags: needinfo?(egao)

Sure, I can take a look at it.

To be clear, this is a cleanup task, correct?

(In reply to Edwin Takahashi (:egao | UTC+9 from 01/22-01/27) from comment #16)

Sure, I can take a look at it.

To be clear, this is a cleanup task, correct?

that's correct. These variables were added when generic-worker was set up, but were copied from the tascluster-worker and, before that, the build-bot config.
So we can keep some, but from what I understand they are somewhat hidden and unexpected from the task-writer perspective.

Taking a look - I will assign this to myself.

Assignee: dhouse → egao
Flags: needinfo?(egao)

Try push: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&tier=1%2C2%2C3&revision=03b8e721d01b1898e54297e661e105bb5b563b8d

Seems to be clean. Given that macosx1014 machines are in short-ish supply I will hope that nothing else breaks when it lands on autoland instead of making more comprehensive pushes.

Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a52d82375742
clean up inherited environment variables r=dhouse

https://hg.mozilla.org/mozilla-central/rev/a52d82375742

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: