634428 - w32-ix-slave03 needs setup for staging

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Hostname is win32-ix-slave03 so that needs fixing with everything else post-image.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

w32-ix-slave08 has only ever done one job on try, might need keeping an eye one.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

w32-ix-slave08's notion of its hostname is w32-ix-slave0.

I think this needs a "double-check the hostname" step added to PostImage.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Both are in staging now.  Leaving open to put w32-ix-slave08 back onto try.

Comment 4

[Rail Aliiev [:rail]]

w32-ix-slave03 went somehow to production and got a release build, which failed with the following error:
+++ python /e/builds/moz2_slave/rel-192-w32-rpk-1/scripts/scripts/l10n/../..//buildfarm/utils/jsontool.py -k properties.branch buildprops.json
Traceback (most recent call last):
  File "e:/builds/moz2_slave/rel-192-w32-rpk-1/scripts/scripts/l10n/../..//buildfarm/utils/jsontool.py", line 6, in <module>
    import simplejson as json
ImportError: No module named simplejson

buildbottac renamed

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

I think the TAC generator on Windows is seriously broken, and spontaneously regenerates a production file.  Also, there seems to be some service that periodically tries to restart the slave, even if it's turned off.  Hopefully the slave alloc work will eliminate that cruft.

So you may want to shut the system down instead of just moving buildbot.tac.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I heard from catlee over the weekend that Windows systems were seeing spontaneous failures of the bulidslave process, so there's code in place to periodically restart them.  This explains the flapping python.exe checks in nagios, and presumably that periodic restart is also generating new buildbot.tac's that point to production.  That all needs to die with the new slave startup stuff, but in the interim, completely shut down any w32 systems that you don't want running.

I'll move this to staging as directed in the summary.

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

w32-ix-slave03 has the correct hostname and is pointed to sm01.