Last Comment Bug 1060255 - Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.
: Windows Builds fail with configure: error: To build the installer you must ha...
Status: RESOLVED FIXED
[release-impacting]
: intermittent-failure
Product: Release Engineering
Classification: Other
Component: Buildduty (show other bugs)
: unspecified
: x86 Windows XP
-- normal (vote)
: ---
Assigned To: Mark Cornmesser [:markco]
: Justin Wood (:Callek)
:
Mentors:
https://tbpl.mozilla.org/php/getParse...
Depends on:
Blocks: b-2008-ix-0085 b-2008-ix-0115 b-2008-ix-0023 b-2008-ix-0103 b-2008-ix-0014 989531 b-2008-ix-0009 b-2008-ix-0130 b-2008-ix-0114 b-2008-ix-0104 b-2008-ix-0138 b-2008-ix-0122 b-2008-ix-0072 b-2008-ix-0140 b-2008-ix-0149 b-2008-ix-0060 b-2008-ix-0054 b-2008-ix-0059 1062877 b-2008-ix-0002 b-2008-ix-0064 b-2008-ix-0166
  Show dependency treegraph
 
Reported: 2014-08-29 01:19 PDT by Carsten Book [:Tomcat]
Modified: 2014-10-06 03:43 PDT (History)
17 users (show)
See Also:
Crash Signature:
(edit)
Machine State: ---
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description User image Carsten Book [:Tomcat] 2014-08-29 01:19:30 PDT
b2g_b2g-inbound_win32_gecko-debug build on 2014-08-29 00:38:43 PDT for push afe145022092

slave: b-2008-ix-0130

https://tbpl.mozilla.org/php/getParsedLog.php?id=47011046&tree=B2g-Inbound

configure:22686: checking for Unicode NSIS version 2.46 or greater
configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.
*** Fix above errors and then restart with\
               "c:/builds/moz2_slave/b2g-in-w32_g-d-000000000000000/build/mozmake.exe -f client.mk build"
c:/builds/moz2_slave/b2g-in-w32_g-d-000000000000000/build/client.mk:355: recipe for target 'configure' failed
Comment 6 User image Pete Moore [:pmoore][:pete] 2014-08-29 03:35:31 PDT
So at this point in time it is not clear what the source of this problem could be.

Speaking to Mike Hommey, he believes that NSIS is a build dependency and should be part of the Windows image. However, the affected slaves often do not have tracking bugs, and it would therefore appear as though they have not recently been reimaged. Some of these slaves were running jobs successfully a few hours before, so it appears something external may have changed.

However, if something external has changed, it does not appear to be affecting all the Windows build slaves, since others are not exhibiting this problem.

Currently the only reasonable plan we have is to disable offending slaves, until we can work out what is causing this problem.
Comment 7 User image Amy Rich [:arr] [:arich] 2014-08-29 05:16:42 PDT
I presume this is related to bug 989531 which rolled out yesterday.
Comment 8 User image Jordan Lund (:jlund) 2014-08-29 14:57:11 PDT
it sounds like things are fixed up with this: https://bugzilla.mozilla.org/show_bug.cgi?id=989531#c10

https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=b-2008-ix-0138 has been hit this a lot prior to fix. if it greens up, we should resolve this bug and re-enable disabled slaves
Comment 9 User image Phil Ringnalda (:philor) 2014-08-29 21:43:45 PDT
According to https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?class=build&type=b-2008-ix&name=b-2008-ix-0114 I rebooted it at 8:02pm, and it failed this way on a 8:28pm build. I'm disabling the rest of the appaarently-still-affected slaves, too - we've got far too many Windows build slaves, we don't need these broken ones running over the weekend.
Comment 10 User image Phil Ringnalda (:philor) 2014-08-31 09:05:34 PDT
0138 wasn't actually any better, it was just doing a good job of only taking jobs where nobody would look at the way it failed.
Comment 11 User image Q 2014-08-31 16:05:35 PDT
Sounds like we need a deeper dive into this ?
Comment 12 User image Phil Ringnalda (:philor) 2014-08-31 18:57:24 PDT
Unless b-2008-ix-0122 was some separate bustage that just happens to have the same visible symptom, I'd certainly say we do, since it successfully built yesterday, and wound up broken today.
Comment 13 User image Hal Wine [:hwine] (use NI) 2014-09-03 11:54:15 PDT
hit b-2008-ix-0137 on Aug 28 during Thunderbird 31.1.0 release:
 http://buildbot-master84.srv.releng.scl3.mozilla.com:8001/builders/release-comm-esr31-win32_repack_1%2F10/builds/1
Comment 14 User image Mark Cornmesser [:markco] 2014-09-03 12:32:41 PDT
Is there a machine we can use for testing for this? I think we take a machine install NSIS 3.0a2 and then add it to the path, and see if we get the same error. If that works then we can just push that out through a GPO.
Comment 15 User image Hal Wine [:hwine] (use NI) 2014-09-03 12:35:18 PDT
hit b-2008-ix-0114 on Aug 28 during Thunderbird 31.1.0 release repack:
 http://buildbot-master84.srv.releng.scl3.mozilla.com:8001/builders/release-comm-esr31-win32_repack_1%2F10/builds/2
Comment 16 User image Hal Wine [:hwine] (use NI) 2014-09-03 12:38:59 PDT
(In reply to Mark Cornmesser [:markco] from comment #14)
> Is there a machine we can use for testing for this?

b-2008-ix-0114 has been disabled, with this bug given as the reason. You should be able to use that.
Comment 17 User image Mark Cornmesser [:markco] 2014-09-15 13:11:49 PDT
b-2008-ix-0114 has been set up with the files for nsis 3.0a2 and the directory has been added to the path on the machine. 

If this works, I have a GPO ready to push out these changes.
Comment 18 User image Phil Ringnalda (:philor) 2014-09-15 21:53:34 PDT
Slight delay in the testing, since earlier I reenabled 0014 instead of 0114, but now 114 is enabled, waiting for it to take a job where this would have hit.
Comment 19 User image Carsten Book [:Tomcat] 2014-09-15 22:48:11 PDT
(In reply to Phil Ringnalda (:philor) from comment #18)
> Slight delay in the testing, since earlier I reenabled 0014 instead of 0114,
> but now 114 is enabled, waiting for it to take a job where this would have
> hit.

didn't work out ->

https://tbpl.mozilla.org/php/getParsedLog.php?id=48158116&tree=Mozilla-Inbound
configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.

disabled that slave again
Comment 20 User image Mark Cornmesser [:markco] 2014-09-16 12:47:33 PDT
I went back looked at the GPO and the file copy was missing some of the sub-directories. I have corrected this. 

However, am I miss interpreting "your path"? I have added nsis 3.0a2 to the system variable for path. Is that where it needed to be added?
Comment 21 User image Ryan VanderMeulen [:RyanVM] 2014-09-18 10:47:02 PDT
Getting mass failures now. All trees closed.
Comment 22 User image Ryan VanderMeulen [:RyanVM] 2014-09-18 12:10:39 PDT
The GPO changes were backed out and things seem to have gone back to normal. Reopened at 12:10 MVT.
Comment 23 User image Pete Moore [:pmoore][:pete] 2014-10-01 06:01:41 PDT
(In reply to Mark Cornmesser [:markco] from comment #20)

> However, am I miss interpreting "your path"? I have added nsis 3.0a2 to the
> system variable for path. Is that where it needed to be added?
Comment 24 User image Pete Moore [:pmoore][:pete] 2014-10-01 06:07:01 PDT
Hi guys,

Should we close this bug now, or is there more to do?

Many thanks,
Pete
Comment 25 User image Carsten Book [:Tomcat] 2014-10-01 06:52:22 PDT
haven;t seen any new slaves so i think its ok to close from my pov
Comment 26 User image Amy Rich [:arr] [:arich] 2014-10-01 09:24:47 PDT
We rolled back the change that pushed out NSIS. The deployment is still broken (path issues that someone needs to resolve and pass onto relops to fix the GPO), so that bug still needs to be fixed before we can deploy.  I'm not sure if that should be tracked here or in a different bug, though.
Comment 27 User image Pete Moore [:pmoore][:pete] 2014-10-06 03:43:37 PDT
(In reply to Amy Rich [:arich] [:arr] from comment #26)
> We rolled back the change that pushed out NSIS. The deployment is still
> broken (path issues that someone needs to resolve and pass onto relops to
> fix the GPO), so that bug still needs to be fixed before we can deploy.  I'm
> not sure if that should be tracked here or in a different bug, though.

Thanks Amy!

Since bug 989531 is tracking that rollout then, and these symptoms have been solved by the current rollback, I'll close this for now, and it can always be reopened if we roll out again, and cause the same problem.

Pete

Note You need to log in before you can comment on or make changes to this bug.