Closed Bug 1175701 Opened 5 years ago Closed 3 years ago

Windows Puppet machines failing on sendchange

Categories

(Infrastructure & Operations :: RelOps: Puppet, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: markco, Assigned: jlund)

References

Details

(Whiteboard: [windows])

Attachments

(5 files, 2 obsolete files)

Similar errors have appeared multiple times in the try test pool:

14:29:19     INFO - Running command: ['c:\\mozilla-build\\python27\\python.exe', 'c:\\mozilla-build\\buildbotve\\scripts\\buildbot', 'sendchange', '--master', 'buildbot-master81.build.mozilla.org:9301', '--username', 'sendchange-unittest', '--branch', 'try-win32-debug-unittest', '-r', u'eacce28c98b6e2be79f5f36e3487d52db0b23f13', '--username', u'msreckovic@mozilla.com', '--comments', 'try: -b d -p win32,win64 -u all -t none', '--property', 'buildid:20150616132629', '--property', 'pgo_build:False', '--property', 'builduid:328ab5a737234a1a9909135c1f903c62', u'https://queue.taskcluster.net/v1/task/B4nnGoSiTe-6FAF6kveL2A/artifacts/public/build/firefox-41.0a1.en-US.win32.zip', u'https://queue.taskcluster.net/v1/task/B4nnGoSiTe-6FAF6kveL2A/artifacts/public/build/test_packages.json']
14:29:19     INFO - Copy/paste: c:\mozilla-build\python27\python.exe c:\mozilla-build\buildbotve\scripts\buildbot sendchange --master buildbot-master81.build.mozilla.org:9301 --username sendchange-unittest --branch try-win32-debug-unittest -r eacce28c98b6e2be79f5f36e3487d52db0b23f13 --username msreckovic@mozilla.com --comments "try: -b d -p win32,win64 -u all -t none" --property buildid:20150616132629 --property pgo_build:False --property builduid:328ab5a737234a1a9909135c1f903c62 https://queue.taskcluster.net/v1/task/B4nnGoSiTe-6FAF6kveL2A/artifacts/public/build/firefox-41.0a1.en-US.win32.zip https://queue.taskcluster.net/v1/task/B4nnGoSiTe-6FAF6kveL2A/artifacts/public/build/test_packages.json
14:29:19     INFO -  c:\mozilla-build\python27\python.exe: can't open file 'c:\mozilla-build\buildbotve\scripts\buildbot': [Errno 2] No such file or directory
14:29:19    ERROR - Return code: 2
14:29:19     INFO - The sendchange failed but we don't want to turn the build orange: 2


Even though send change failed the jobs returned green.
Assignee: relops → mcornmesser
Blocks: 1175246
Depends on: 1175635
Looks like the same issue of mozharness assuming c:\mozilla-build\buildbotve is a virtualenv with the buildbot server code installed in it.  I forgot how we solved this last time, though..
We solved that with a symbolic link in the startbuildbot.bat 

REM A temporary terrible hack to get pass hardcoding in mozharness for a deprecated path 
REM Once that is addressed this should be removed
REM Ref Bug 1162230
REM the hack is here because the link command needs to be rerun on each boot

if not exist C:\mozilla-build\buildbotve mkdir C:\mozilla-build\buildbotve
ln -s C:\mozilla-build\virtualenv.py C:\mozilla-build\buildbotve\virtualenv.py
I've disabled these machines in slavealloc until we can get this fixed
Summary: Windows Puppet try pool machines failing on sendchange → Windows Puppet machines failing on sendchange
Ah, ok -- I wonder if we could modify mozharness to install buildbot in its virtualenv, instead.  Jordan?
Flags: needinfo?(jlund)
Blocks: 1175635
No longer depends on: 1175635
(In reply to Dustin J. Mitchell [:dustin] from comment #4)
> Ah, ok -- I wonder if we could modify mozharness to install buildbot in its
> virtualenv, instead.  Jordan?

could do. Though, we currently don't create a venv for building ff (we create one for uploading TC artifacts and recording influxdb stats). As it is right now, we call the actual mh build script (and the 'mach build') subprocess with the system wide py2.7

I previously was trying to install buildbot into a venv but ran into problems on linux because of mock[1]. So either we would have to address that or make a windows edge case. Both don't sound optimal.

How about we config more than one path for the buildbot exe that supports both pre and post puppet windows and then search for one that exists before calling it. Even as a stopgap until we are 100% on puppet with new file paths.

[1] https://bugzil.la/1087102
Flags: needinfo?(jlund)
That works.  The path has a version # in it, so when we upgraded Buildbot we'd need to make sure that version number changed, too, but that doesn't seem like such a big deal and is unlikely to happen too often in the next year or so.
discussed over irc, I can pick this up next week in a hack session @ whistler :)

/me steals from markco. feel free to steal back if you really want it
Assignee: mcornmesser → jlund
Status: NEW → ASSIGNED
Whiteboard: [windows]
hacking on this today.

so it looks like exes supports a list already for something else. This introduces a dict where key==reason_or_name_of_path, value=path

# negative testing
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9534be641a87

while these tests run, I'll look at addressing the buildbot fix for this. Unfortunetly, since we are not using mh for beta/release, we will need to fix buildbot (MozillaBuildFactory)
this adds the new puppetized path for all windows jobs. I am putting it in front of the old/current buildbotve path.

My assumption here is 'if the windows slave has a desired exe from the new path (buildbot-0.8.4-pre-moz6\scripts\*'), use that. Which would include the buildbot exe
Comment on attachment 8626290 [details] [diff] [review]
150625_win_puppet_sendchange-bbot-cfgs.patch

This looks correct.  Do you think a comment would be helpful, or will hg blame be sufficient?
Attachment #8626290 - Flags: review+
took a couple iterations but https://treeherder.mozilla.org/#/jobs?repo=try&revision=9534be641a87 is finally happy

logic still applies from https://bugzilla.mozilla.org/show_bug.cgi?id=1175701#c8
Attachment #8626228 - Attachment is obsolete: true
Attachment #8627372 - Flags: review?(mshal)
Comment on attachment 8627372 [details] [diff] [review]
150629_windows_sendchange_query_exe_paths_support-mh.patch

This looks fine - the only thing I see that might cause trouble is that we do the '% repl_dict' substitution after the os.path.exists() checks. So if we used %(abs_work_dir) or whatever in the path, I think it would fail. It doesn't look like we actually use that support in any of the exe paths, however.
Attachment #8627372 - Flags: review?(mshal) → review+
(In reply to Michael Shal [:mshal] from comment #12)
> Comment on attachment 8627372 [details] [diff] [review]
> 150629_windows_sendchange_query_exe_paths_support-mh.patch
> 
> This looks fine - the only thing I see that might cause trouble is that we
> do the '% repl_dict' substitution after the os.path.exists() checks

as mentioned on irc, tx for the catch. we should not remove support for this expansion if the list, tuple, and string case can use it already.

I've respun a new patch and will attach for r? once try runs are green
Comment on attachment 8628023 [details] [diff] [review]
150630_windows_sendchange_query_exe.patch

Looks good!
Attachment #8628023 - Flags: review?(mshal) → review+
Comment on attachment 8628023 [details] [diff] [review]
150630_windows_sendchange_query_exe.patch

thanks, on default

remote:   https://hg.mozilla.org/build/mozharness/rev/af321e004210
Attachment #8628023 - Flags: checked-in+
Comment on attachment 8626290 [details] [diff] [review]
150625_win_puppet_sendchange-bbot-cfgs.patch

thanks, I added bug # and comment above each path

remote:   https://hg.mozilla.org/build/buildbot-configs/rev/fdeaeb10ee77
Attachment #8626290 - Flags: checked-in+
mozharness production tag moved to: https://hg.mozilla.org/build/mozharness/rev/production
markco: feel free to resolve this bug once you are able to test it again. Let me know if I can help at all :D
Flags: needinfo?(mcornmesser)
I am going to resolve this. If this proves to be an issue after I get passed a few blockers I will reopen.
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Flags: needinfo?(mcornmesser)
Resolution: --- → FIXED
It seems we are having some failures. 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=103193c1cd13

9:43:16     INFO -  Traceback (most recent call last):
09:43:16     INFO -    File "c:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\buildbot", line 3, in <module>
09:43:16     INFO -      from buildbot.scripts import runner
09:43:16     INFO -  ImportError: No module named buildbot.scripts
09:43:16    ERROR - Return code: 1
09:43:16     INFO - The sendchange failed but we don't want to turn the build orange: 1
09:43:16     INFO - Running command: ['c:\\mozilla-build\\python27\
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Blocks: 1164214
Blocks: 1186586
so the patch seems to be working as expected:

if searches the possible 'buildbot' exe paths and

# if the machine is puppet based, it runs: 
'c:\\mozilla-build\\python27\\python.exe', 'c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\scripts\\buildbot' .. etc

if the machine is gpo based, it runs:
'c:\\mozilla-build\\python27\\python.exe', 'c:\\mozilla-build\\buildbotve\\scripts\\buildbot' etc

however, the puppet one is failing because it doesn't have 'buildbot' installed in the interpreter: 

puppet:
cltbld@B-2008-IX-0175 ~
$ cat /c/mozilla-build/buildbot-0.8.4-pre-moz6/Scripts/buildbot
#!c:\mozilla-build\buildbot-0.8.4-pre-moz6\Scripts\python2.7.exe

from buildbot.scripts import runner
runner.run()

cltbld@B-2008-IX-0175 ~
$ /c/mozilla-build/python27/python.exe
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from buildbot.scripts import runner
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named buildbot.scripts

gpo:
cltbld@B-2008-IX-0033 ~
$ cat /c/mozilla-build/buildbotve/scripts/buildbot
#!c:\mozilla-build\buildbotve\Scripts\python.exe

from buildbot.scripts import runner
runner.run()

cltbld@B-2008-IX-0033 ~
$ /c/mozilla-build/python27/python.exe
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from buildbot.scripts import runner
>>>
The module is available from within the virtualenv directory:

C:\Users\cltbld>C:\mozilla-build\buildbot-0.8.4-pre-moz6\Scripts\python.exe
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> from buildbot.scripts import runner
>>>

Would this be a viable option for the Puppet configured machines?
Flags: needinfo?(jlund)
I don't see why we can't use it :)
Flags: needinfo?(jlund)
Attachment #8638119 - Flags: review?(mshal)
Attachment #8638119 - Flags: review?(mshal) → review+
Keywords: leave-open
I think we are good: 
12:18:53     INFO - #####
12:18:53     INFO - ##### Running sendchange step.
12:18:53     INFO - #####
12:18:53     INFO - Running pre-action listener: influxdb_recording_pre_action
12:18:53     INFO - Running main action method: sendchange
12:18:53     INFO - 'enable_talos_sendchange' is false; skipping
12:18:53     INFO - Running command: ['c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\Scripts\\python.exe', 'c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\scripts\\buildbot', 'sendchange', '--master', 'buildbot-master81.build.mozilla.org:9301', '--username', 'sendchange-unittest', '--branch', 'try-win32-debug-unittest', '-r', u'56272fbd3fb5a400eb778bda1f3cc208d79becc0', '--username', u'nfitzgerald@mozilla.com', '--comments', 'try: -b do -p all -u cppunit -t none', '--property', 'buildid:20150724110756', '--property', 'pgo_build:False', '--property', 'builduid:8267d8745bbb4bb0bd8de95607be9936', u'https://queue.taskcluster.net/v1/task/DBeC18NBSO6J2eKeFw9N1w/artifacts/public/build/firefox-42.0a1.en-US.win32.zip', u'https://queue.taskcluster.net/v1/task/DBeC18NBSO6J2eKeFw9N1w/artifacts/public/build/test_packages.json']
12:18:53     INFO - Copy/paste: c:\mozilla-build\buildbot-0.8.4-pre-moz6\Scripts\python.exe c:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\buildbot sendchange --master buildbot-master81.build.mozilla.org:9301 --username sendchange-unittest --branch try-win32-debug-unittest -r 56272fbd3fb5a400eb778bda1f3cc208d79becc0 --username nfitzgerald@mozilla.com --comments "try: -b do -p all -u cppunit -t none" --property buildid:20150724110756 --property pgo_build:False --property builduid:8267d8745bbb4bb0bd8de95607be9936 https://queue.taskcluster.net/v1/task/DBeC18NBSO6J2eKeFw9N1w/artifacts/public/build/firefox-42.0a1.en-US.win32.zip https://queue.taskcluster.net/v1/task/DBeC18NBSO6J2eKeFw9N1w/artifacts/public/build/test_packages.json
12:18:59     INFO -  NOTE: --username/-u is deprecated: use --who/-W'
12:18:59     INFO -  change sent successfully
12:18:59     INFO - Return code: 0
12:18:59     INFO - Running post-action listener: influxdb_recording_post_action
12:18:59     INFO - Starting new HTTP connection (1): goldiewilson-onepointtwentyone-1.c.influxdb.com
Looked like sendchange has failed in a different build: 

19:18:54     INFO - ##### Running sendchange step.
19:18:54     INFO - #####
19:18:54     INFO - Running pre-action listener: influxdb_recording_pre_action
19:18:54     INFO - Running main action method: sendchange
19:18:54     INFO - Running command: ['c:\\mozilla-build\\python27\\python.exe', 'c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\scripts\\buildbot', 'sendchange', '--master', 'buildbot-master81.build.mozilla.org:9301', '--username', 'sendchange', '--branch', 'try-win32-talos', '-r', u'ad632449c494bea3eb0f12ea8c92fa4fa2ed37b4', '--username', u'smokey101stair@gmail.com', '--comments', 'try: -b do -p linux,linux64,macosx64,win32,win64 -u mochitest-bc,mochitest-e10s-browser-chrome-1,mochitest-e10s-browser-chrome-2,mochitest-e10s-browser-chrome-3 -t none', '--property', 'buildid:20150724182541', '--property', 'pgo_build:False', '--property', 'builduid:66abb0400c82453d96fc2e5b10c68293', u'https://queue.taskcluster.net/v1/task/rhJcCXZETfm5_AFPzJgxyw/artifacts/public/build/firefox-42.0a1.en-US.win32.zip']
19:18:54     INFO - Copy/paste: c:\mozilla-build\python27\python.exe c:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\buildbot sendchange --master buildbot-master81.build.mozilla.org:9301 --username sendchange --branch try-win32-talos -r ad632449c494bea3eb0f12ea8c92fa4fa2ed37b4 --username smokey101stair@gmail.com --comments "try: -b do -p linux,linux64,macosx64,win32,win64 -u mochitest-bc,mochitest-e10s-browser-chrome-1,mochitest-e10s-browser-chrome-2,mochitest-e10s-browser-chrome-3 -t none" --property buildid:20150724182541 --property pgo_build:False --property builduid:66abb0400c82453d96fc2e5b10c68293 https://queue.taskcluster.net/v1/task/rhJcCXZETfm5_AFPzJgxyw/artifacts/public/build/firefox-42.0a1.en-US.win32.zip
19:18:54     INFO -  Traceback (most recent call last):
19:18:54     INFO -    File "c:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\buildbot", line 3, in <module>
19:18:54     INFO -      from buildbot.scripts import runner
19:18:54     INFO -  ImportError: No module named buildbot.scripts
19:18:54    ERROR - Return code: 1
19:18:54     INFO - The sendchange failed but we don't want to turn the build orange: 1
(In reply to Mark Cornmesser [:markco] from comment #30)
> Looked like sendchange has failed in a different build: 

this is likely due to a try job being pushed that is out of date (prior to https://bugzilla.mozilla.org/show_bug.cgi?id=1175701#c27) with m-c or it's based on an older branch where this change doesn't exist.

The best we can do here is either wait a few days until all devs have merged in m-c or just let the jobs not trigger sendchanges and expect that devs will update to latest m-c.

We should probably merge this sendchange patch across all branches though so we don't break say esr jobs
jlund: has this change been merged everywhere at this point? If so, are we good to r/f?
Flags: needinfo?(jlund)
(In reply to Amy Rich [:arr] [:arich] from comment #32)
> jlund: has this change been merged everywhere at this point? If so, are we
> good to r/f?

I'm assuming that this patch has helped and we are no longer seeing the issue in comment 30/31. I will uplift it everywhere tomorrow.
Flags: needinfo?(jlund)
this bug's patches has been uplifted to m-a but due to current releases and merge week, I have to postpone uploading beta/release/esr till friday or monday
(In reply to Jordan Lund (:jlund) from comment #34)
> this bug's patches has been uplifted to m-a but due to current releases and
> merge week, I have to postpone uploading beta/release/esr till friday or
> monday

we are actually good here as mozharness is not used to build beta/release/esr yet. those branches use buildbot factories/build-steps only.

dustin: a while ago we patched buildbot to support the new path sendchange path[1]. however, in the morharness case that alone wasn't enough as we also had to tell it to use a specific interpreter[2] because of this[3]. Looks like in the buildbot case we use SendChangeStep. Do you think we will need to put `${MOZILLABUILD}python27` after `${MOZILLABUILD}buildbot-0.8.4-pre-moz6\\scripts` in [1]? I worry that will cause spurious results if you think we need to.

[1] https://bugzilla.mozilla.org/attachment.cgi?id=8626290
[2] https://bug1175701.bmoattachments.org/attachment.cgi?id=8638119
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1175701#c23

ni: dustin via irc
I don't understand the situation well enough to make any concrete suggestions.  Changing the order of PATH seems a pretty drastic step, though, and is likely to have other consequences.  I suspect it would be better to modify SendChangeStep to use the proper interpreter.

I brought up the possibility, recently, of reverting support for installing multiple versions of Buildbot side-by-side.  We've had this in puppet for a while because, on Linux and Mac OS X, it's easy (yay symlinks).  The expectation was that we'd be upgrading Buildbot frequently, and that it would be nice to be able to have the "old" and "new" versions installed side-by-side and just flip between them with slavealloc.  This functionality was never developed or used.  On Windows, this has been the source of a lot of headache.  Maybe we should just go back to installing one version of Buildbot in a very predictable path on all systems, not just Windows.
here is how we are calling sendchange via mozharness currently (gpo):
    22:38:42     INFO - Running command: ['c:\\mozilla-build\\python27\\python.exe', 'c:\\mozilla-build\\buildbotve\\scripts\\buildbot', 'sendchange',

and this is buildbot factory based equivalent (gpo):
    'python' 'c:/builds/moz2_slave/m-beta-w32-0000000000000000000/tools/buildfarm/utils/retry.py' '-s' '5' '-t' '1800' '-r' '5', 'buildbot' 'sendchange'
    # with important path bits: PATH=C:\mozilla-build\python27;C:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts;C:\mozilla-build\buildbotve\scripts;


for the puppet win case we have configured the following in mozharness:
    22:38:42     INFO - Running command: ['c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\Scripts\\python.exe', 'c:\\mozilla-build\\buildbot-0.8.4-pre-moz6\\scripts\\buildbot', 'sendchange',


so for buildbot factory win puppet case, with our current PATH above I think we would end up using the `c:\\mozilla-build\\python27\\python.exe` interpreter to call `C:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\buildbot` which is bad as iiuc we don't have buildbot installed in `c:\\mozilla-build\\python27\\python.exe` on the windows puppet scenario.

So there are two options as I see it:

1) patch SendChangeStep to take in a configured path for the interpreter. iiuc SendChangeStep is part of buildbot itself, not buildbotcustom.

2) install buildbot in `c:\\mozilla-build\\python27\\python.exe` on windows puppet machines.

markco, how hard would 2 be. it would mean we don't need any extra conditions or hacks on the automation side.
Flags: needinfo?(mcornmesser)
#1 shouldn't be bad at all -- it's in buildbotcustom:
  https://dxr.mozilla.org/build_buildbotcustom/source/steps/misc.py#74
It should be easy enough to replace `'python'` with a configurable path at
  https://dxr.mozilla.org/build_buildbotcustom/source/steps/misc.py#146

c:\\mozilla-build\\python27 isn't a virtualenv, so installing Buildbot there is not a great idea.
(In reply to Dustin J. Mitchell [:dustin] from comment #38)
> #1 shouldn't be bad at all -- it's in buildbotcustom:
>   https://dxr.mozilla.org/build_buildbotcustom/source/steps/misc.py#74
> It should be easy enough to replace `'python'` with a configurable path at
>   https://dxr.mozilla.org/build_buildbotcustom/source/steps/misc.py#146
> 

ah, not sure why I thought SendChangeStep was part of buildbot. Okay, the one issue here is buildbot is not really aware about how the windows machines are configured (whether they are gpo based or puppet based). So we can't add say the following in buildbot-configs:

['win32']['sendchange_interpreter'] = 'C:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\python.exe'

as that will fail for the current gpo case. Mozharness handles this by saying something like:

if os.path.exists('C:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\python.exe'):
    # change the sendchange python interpreter as this is a puppet machine
    python = 'C:\mozilla-build\buildbot-0.8.4-pre-moz6\scripts\python.exe'

But in buildbot land, we can't really have runtime conditions like that pre-configured. How do we normally support two different slave configured states within the same platform in buildbot world?
That is tricky.. Adding code to call a `stat` on the slave would be a more substantial change to the step.

Stepping back for a minute to note how many "in-transition" things we have going on right now: some builds use mozharness, some don't.  Some builds use puppet-configured hosts, some don't.  Some builds use Buildbot, some don't.  Yikes!

Given all of this, your PATH modifiction suggestion from comment 35 is sounding more attractive.  On non-puppet machines, ${MOZILLABUILD}buildbot-0.8.4-pre-moz6 doesn't exist at all, so adding it won't hurt.  On puppetized machines, use of `python` in the build process will find the virtualenv python, but I *think* that will be OK, as it still has pywin32 and all that suff available.  Would it be possible to amend the path only in the SendChangeStep?
It looks like we are going to make a change on the Puppet configured Windows machines to use the generic buildbotve directory for the active version of buildbot.
Flags: needinfo?(mcornmesser)
These changes can be reverted now. "c:\mozilla-build\buildbotve\scripts\buildbot" now exists within the Puppet configuration.
jlund: Could you revert the changes made for the puppet machines, please? Everything should now be going to the buildbotve directory. The specific version buildbot directories no longer exists.
Flags: needinfo?(jlund)
backing out of mh is kind of awkward as it was part of the large mh->gecko_tree migration.

I had to do this patch by hand. I'll land tomorrow as trees are closed right now and I'm not 100% it won't cause bustage. I am keeping the script core changes in case we want to have multiple search options in exes again in the future.

keeping needinfo to self as reminder
backed out on https://hg.mozilla.org/integration/mozilla-inbound/rev/c05e13ac8ce0

aurora to go
Flags: needinfo?(jlund)
this has been uplifted. the patch has been reverted
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
I kinda feel bad reopening this, but we should now point sendchange to use  "C:\mozilla-build\buildbotve\Scripts\python.exe" . This directory should be static and has all the needed modules needed for sendchange to work available to python within this directory. This is not urgent. I presently have a hack to work around it.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
No longer blocks: 1175635
Duplicate of this bug: 1175635
Did this ever get resolved?
Flags: needinfo?(mcornmesser)
Flags: needinfo?(jlund)
As far as I know there has been no change to this.
Flags: needinfo?(mcornmesser)
missed comment 49. will push a patch to try with requested value for python exe
Flags: needinfo?(jlund)
(In reply to Jordan Lund (:jlund) from comment #54)
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=76d4bf83d0e2

looks sane.

(In reply to Jordan Lund (:jlund) from comment #55)
> Created attachment 8772885 [details]
> Bug 1175701 - change windows python interpreter for buildbot sendchange,
> 
> Review commit: https://reviewboard.mozilla.org/r/65584/diff/#index_header

this ^ shouldn't affect taskcluster builds
Keywords: leave-open
Attachment #8772885 - Flags: review?(mshal) → review+
markco: is what I said below accurate?

11:05:17 <mshal> jlund: so running buildbot/sendchange has to use a specific python exe?
11:05:41 <mshal> I don't quite follow what exactly it's supposed to fix
11:07:12 <jlund> iiuc - sys.executable is resolving to a path on our windows machines that is a link to the real python.exe that we want
11:07:26 <jlund> or at least that's my guess at markco's 'hack' mentioned here: https://bugzilla.mozilla.org/show_bug.cgi?id=1175701#c49
11:08:02 <jlund> so it is working. but he wants to remove the hack and use a static path for buildbot sendchange call
11:08:17 <mshal> oic
Flags: needinfo?(mcornmesser)
also of note:

11:39:02 <Callek> jlund: arr: yea my concern is with removing the hack during the heavy tme-sens releases (last betas, rc/release/beta1) -- specifically both incase a failure to sendchange causes a build bustage and incase something in the build is relying on a side-effect of something installed with the buildbot python and that path....


since there is nothing busted, I'll wait till markco gets back from PTO and chats with callek over comment 57

the patch seems fine and is r+ so I'll land once there are no further concerns and we can remove the temp fix on win machines at the same time.
With the pending move to task cluster, my opinion is to just let sit as is. Though it is not pretty it is functioning.
Flags: needinfo?(mcornmesser)
Status: REOPENED → RESOLVED
Closed: 4 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.