1060255 - Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Carsten Book [:Tomcat]]

b2g_b2g-inbound_win32_gecko-debug build on 2014-08-29 00:38:43 PDT for push afe145022092

slave: b-2008-ix-0130

https://tbpl.mozilla.org/php/getParsedLog.php?id=47011046&tree=B2g-Inbound

configure:22686: checking for Unicode NSIS version 2.46 or greater
configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.
*** Fix above errors and then restart with\
               "c:/builds/moz2_slave/b2g-in-w32_g-d-000000000000000/build/mozmake.exe -f client.mk build"
c:/builds/moz2_slave/b2g-in-w32_g-d-000000000000000/build/client.mk:355: recipe for target 'configure' failed

Comment 1

[Carsten Book [:Tomcat]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=47007387&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=47011651&tree=B2g-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=47011046&tree=B2g-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=47010662&tree=B2g-Inbound

Comment 2

[Carsten Book [:Tomcat]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=47005048&tree=Fx-Team

Comment 3

[Carsten Book [:Tomcat]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=47014044&tree=B2g-Inbound

Comment 4

[Mark Banner (:standard8)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=46979824&tree=Thunderbird-Esr31
https://tbpl.mozilla.org/php/getParsedLog.php?id=46949663&tree=Thunderbird-Esr31

Comment 5

[Carsten Book [:Tomcat]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=47016174&tree=B2g-Inbound

Comment 6

[Pete Moore [:pmoore][:pete]]

So at this point in time it is not clear what the source of this problem could be.

Speaking to Mike Hommey, he believes that NSIS is a build dependency and should be part of the Windows image. However, the affected slaves often do not have tracking bugs, and it would therefore appear as though they have not recently been reimaged. Some of these slaves were running jobs successfully a few hours before, so it appears something external may have changed.

However, if something external has changed, it does not appear to be affecting all the Windows build slaves, since others are not exhibiting this problem.

Currently the only reasonable plan we have is to disable offending slaves, until we can work out what is causing this problem.

Comment 7

[Amy Rich [:arr] [:arich]]

I presume this is related to bug 989531 which rolled out yesterday.

Comment 8

[Jordan Lund (:jlund)]

it sounds like things are fixed up with this: https://bugzilla.mozilla.org/show_bug.cgi?id=989531#c10

https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=b-2008-ix-0138 has been hit this a lot prior to fix. if it greens up, we should resolve this bug and re-enable disabled slaves

Comment 9

[Phil Ringnalda (:philor)]

According to https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?class=build&type=b-2008-ix&name=b-2008-ix-0114 I rebooted it at 8:02pm, and it failed this way on a 8:28pm build. I'm disabling the rest of the appaarently-still-affected slaves, too - we've got far too many Windows build slaves, we don't need these broken ones running over the weekend.

Comment 10

[Phil Ringnalda (:philor)]

0138 wasn't actually any better, it was just doing a good job of only taking jobs where nobody would look at the way it failed.

Comment 11

[Q]

Sounds like we need a deeper dive into this ?

Comment 12

[Phil Ringnalda (:philor)]

Unless b-2008-ix-0122 was some separate bustage that just happens to have the same visible symptom, I'd certainly say we do, since it successfully built yesterday, and wound up broken today.

Comment 13

[Hal Wine [:hwine] use NI!]

hit b-2008-ix-0137 on Aug 28 during Thunderbird 31.1.0 release:
 http://buildbot-master84.srv.releng.scl3.mozilla.com:8001/builders/release-comm-esr31-win32_repack_1%2F10/builds/1

Comment 14

[Mark Cornmesser [:markco] OOO 2024/04/15]

Is there a machine we can use for testing for this? I think we take a machine install NSIS 3.0a2 and then add it to the path, and see if we get the same error. If that works then we can just push that out through a GPO.

Comment 15

[Hal Wine [:hwine] use NI!]

hit b-2008-ix-0114 on Aug 28 during Thunderbird 31.1.0 release repack:
 http://buildbot-master84.srv.releng.scl3.mozilla.com:8001/builders/release-comm-esr31-win32_repack_1%2F10/builds/2

Comment 16

[Hal Wine [:hwine] use NI!]

(In reply to Mark Cornmesser [:markco] from comment #14)
> Is there a machine we can use for testing for this?

b-2008-ix-0114 has been disabled, with this bug given as the reason. You should be able to use that.

Comment 17

[Mark Cornmesser [:markco] OOO 2024/04/15]

b-2008-ix-0114 has been set up with the files for nsis 3.0a2 and the directory has been added to the path on the machine. 

If this works, I have a GPO ready to push out these changes.

Comment 18

[Phil Ringnalda (:philor)]

Slight delay in the testing, since earlier I reenabled 0014 instead of 0114, but now 114 is enabled, waiting for it to take a job where this would have hit.

Comment 19

[Carsten Book [:Tomcat]]

(In reply to Phil Ringnalda (:philor) from comment #18)
> Slight delay in the testing, since earlier I reenabled 0014 instead of 0114,
> but now 114 is enabled, waiting for it to take a job where this would have
> hit.

didn't work out ->

https://tbpl.mozilla.org/php/getParsedLog.php?id=48158116&tree=Mozilla-Inbound
configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.

disabled that slave again

Comment 20

[Mark Cornmesser [:markco] OOO 2024/04/15]

I went back looked at the GPO and the file copy was missing some of the sub-directories. I have corrected this. 

However, am I miss interpreting "your path"? I have added nsis 3.0a2 to the system variable for path. Is that where it needed to be added?

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Getting mass failures now. All trees closed.

Comment 22

[Ryan VanderMeulen [:RyanVM]]

The GPO changes were backed out and things seem to have gone back to normal. Reopened at 12:10 MVT.

Comment 23

[Pete Moore [:pmoore][:pete]]

(In reply to Mark Cornmesser [:markco] from comment #20)

> However, am I miss interpreting "your path"? I have added nsis 3.0a2 to the
> system variable for path. Is that where it needed to be added?

Comment 24

[Pete Moore [:pmoore][:pete]]

Hi guys,

Should we close this bug now, or is there more to do?

Many thanks,
Pete

Comment 25

[Carsten Book [:Tomcat]]

haven;t seen any new slaves so i think its ok to close from my pov

Comment 26

[Amy Rich [:arr] [:arich]]

We rolled back the change that pushed out NSIS. The deployment is still broken (path issues that someone needs to resolve and pass onto relops to fix the GPO), so that bug still needs to be fixed before we can deploy.  I'm not sure if that should be tracked here or in a different bug, though.

Comment 27

[Pete Moore [:pmoore][:pete]]

(In reply to Amy Rich [:arich] [:arr] from comment #26)
> We rolled back the change that pushed out NSIS. The deployment is still
> broken (path issues that someone needs to resolve and pass onto relops to
> fix the GPO), so that bug still needs to be fixed before we can deploy.  I'm
> not sure if that should be tracked here or in a different bug, though.

Thanks Amy!

Since bug 989531 is tracking that rollout then, and these symptoms have been solved by the current rollback, I'll close this for now, and it can always be reopened if we roll out again, and cause the same problem.

Pete