1507488 - Add python3.6 and hg to $PATH on OSX generic-worker

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Puppet, task)

Description

[Andrew Halberstadt [:ahal]]

In bug 1466535 we installed python 3 at /tools/python36/bin/python3.6. It also created a symlink to /usr/bin for the Linux generic-worker hosts (but not OSX). Then bug 1505057 added python3 to the $PATH for Windows generic-worker hosts.

Now OSX is the last platform without python3 on the $PATH. Getting it added will allow us to call `python3` in our tasks without needing to hardcode the full path to it.

Comment 1

[Dragos Crisan [:dragrom]]

Attachment: Add python3.6 to PATH on OSX workers

Add python3.6 to PATH on OSX workers

Comment 2

[Andrew Halberstadt [:ahal]]

Has this change been deployed? In my testing, both `which python3` [0] and `which python3.6` [1] seem to return non-zero.

[0] https://taskcluster-artifacts.net/eR-phY51QlyhpdFvMrLBIw/0/public/logs/live_backing.log
[1] https://taskcluster-artifacts.net/ZsgHoJNsTj-jVvmtstVniA/0/public/logs/live_backing.log

Or was this bug fixing some other worker type?

Comment 3

[Dragos Crisan [:dragrom]]

(In reply to Andrew Halberstadt [:ahal] from comment #2)
> Has this change been deployed? In my testing, both `which python3` [0] and
> `which python3.6` [1] seem to return non-zero.
> 
> [0]
> https://taskcluster-artifacts.net/eR-phY51QlyhpdFvMrLBIw/0/public/logs/
> live_backing.log
> [1]
> https://taskcluster-artifacts.net/ZsgHoJNsTj-jVvmtstVniA/0/public/logs/
> live_backing.log
> 
> Or was this bug fixing some other worker type?

The patch was landed:

[[e-mail] ~]# echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/tools/python36/bin/python3.6

Maybe not all workers make changes until now.

Comment 4

[Justin Wood (:Callek)]

Still failed... https://taskcluster-artifacts.net/eR-phY51QlyhpdFvMrLBIw/1/public/logs/live_backing.log  (I'm running a test to try and see what the $PATH actually is when running a task on the worker)

https://tools.taskcluster.net/groups/AybU237lSE2TKB5JOyztTg/tasks/AybU237lSE2TKB5JOyztTg/runs/0 -- still pending though :(

Comment 5

[Andrew Halberstadt [:ahal]]

Oh, looks like maybe the binary itself got added to the $PATH rather than the dir containing the binary?

Comment 6

[Andrew Halberstadt [:ahal]]

Also, could we make sure that "python3" is also on the $PATH (not just python3.6?). I assume putting the dir there will solve that at the same time though.

Comment 7

[Justin Wood (:Callek)]

(In reply to Andrew Halberstadt [:ahal] from comment #5)
> Oh, looks like maybe the binary itself got added to the $PATH rather than
> the dir containing the binary?

I'm not sure that is the issue -- from the log on my "whats our path":


[taskcluster 2018-12-20T18:22:59.233Z] [mounts] Copying /Users/cltbld/downloads/KygM4sJRQRGrw1jUt92ulw to /Users/cltbld/tasks/task_1545330175/run-task
[taskcluster 2018-12-20T18:22:59.233Z] Executing command 0: bash -cx 'cd / && echo $PATH'
+ cd /
+ echo /usr/bin:/bin:/usr/sbin:/sbin
/usr/bin:/bin:/usr/sbin:/sbin

Comment 8

[Dragos Crisan [:dragrom]]

(In reply to Justin Wood (:Callek) from comment #7)
> (In reply to Andrew Halberstadt [:ahal] from comment #5)
> > Oh, looks like maybe the binary itself got added to the $PATH rather than
> > the dir containing the binary?
> 
> I'm not sure that is the issue -- from the log on my "whats our path":
> 
> 
> [taskcluster 2018-12-20T18:22:59.233Z] [mounts] Copying
> /Users/cltbld/downloads/KygM4sJRQRGrw1jUt92ulw to
> /Users/cltbld/tasks/task_1545330175/run-task
> [taskcluster 2018-12-20T18:22:59.233Z] Executing command 0: bash -cx 'cd /
> && echo $PATH'
> + cd /
> + echo /usr/bin:/bin:/usr/sbin:/sbin
> /usr/bin:/bin:/usr/sbin:/sbin

generic-worker could not access the PTATH environment variable from the wokrer. Need to follow another aporach.

Comment 9

[Dragos Crisan [:dragrom]]

Attachment: Add hg and python3 to PATH on OSX generic-worker

Comment 10

[Dragos Crisan [:dragrom]]

Created another patch and tested over staging environment:

This is the task:
https://tools.taskcluster.net/groups/TVm9HJWEQ8-ile2bokAA6A/tasks/TVm9HJWEQ8-ile2bokAA6A/runs/0/logs/public%2Flogs%2Flive_backing.log

:ahal You can use our staging environment to test your changes against hg and python3. All steps are described in this documentation: https://wiki.mozilla.org/ReleaseEngineering#Run_the_tasks_to_staging_pools_.28OSX.2FLinux.29

The workertype is:gecko-t-osx-1010-beta

I'll resolve booth hg and python3 issue with this patch, so maybe you can rename this bug to reflect these changes and close the bug for hg.

Comment 12

[Dragos Crisan [:dragrom]]

Run tests to linux workers:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f2bbdcc079288403bd7f708507b88aae3711d498

Comment 13

[Dragos Crisan [:dragrom]]

everything looks OK on Linux jobs

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Is bug 1519849 the right bug?

Comment 15

[Dragos Crisan [:dragrom]]

(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #14)

Is bug 1519849 the right bug

Need to test changes against OSX and linux. But staging pool disappeared due to issue described in bug 1519849 :)

Comment 16

[Dragos Crisan [:dragrom]]

Starting tests on OSX staging workers:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c1bffd541147f57b96259725167782f13f08682

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

If that's the case, then it's because everything in the staging pool was quarantined :)