1448306 - BinScope seems to have stopped working on builds

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[(Away)]

From a recent m-c Win32 opt build task:
12:19:43     INFO - Could not locate binscope at location : C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe
12:19:43     INFO - Binscope wasn't installed or the BINSCOPE env variable wasn't set correctly, skipping this check and exiting...

And from win64 opt:
21:34:27     INFO - BINSCOPE environment variable is not set, can't check DEP/ASLR etc. status.

BinScope verifies that our binaries follow MS security recommendations, so failing to run this tool could lead to uncaught regressions.

First thing to check would be whether "C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe" exists on our builders nowadays.

I don't know who to start with, or even if I'm in the right component. Catlee could you help route this please?

Comment 1

[Chris AtLee [:catlee]]

I'm not sure....It's possible that BINSCOPE isn't being set correctly, you could look at changes to taskcluster/ to see if anything jumps out.

Otherwise, you could ask :grenade or :pmoore to see if anything has changed on the workers lately.

Should failure to run binscope be made into a fatal error?

Comment 2

[(Away)]

:grenade, does "C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe" exist on our builders nowadays?

> Should failure to run binscope be made into a fatal error?

I would claim yes.

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

it looks like binscope is installed at: C:\Program Files\Microsoft BinScope 2014\Binscope.exe

here is a task that lists the contents of C:\Program Files\Microsoft BinScope 2014:
https://tools.taskcluster.net/groups/FHtI9j7uRISF7eQPB8m2Ow/tasks/FHtI9j7uRISF7eQPB8m2Ow/runs/0/logs/public%2Flogs%2Flive.log

i'm not sure how or why the path differs from the one in the mozharness configs. we did recently patch (https://github.com/mozilla-releng/OpenCloudConfig/commit/b58a67f3b54e10085232aa9f39cb7426bf145592) the builder manifests changing the source url for the binscope installer from github (https://github.com/mozilla-releng/OpenCloudConfig/raw/master/userdata/Configuration/FirefoxBuildResources/BinScope_x64.msi) to s3 (https://s3.amazonaws.com/windows-opencloudconfig-packages/binscope/BinScope_x64.msi) but the binary artefact sha512sum for both of those artefacts is identical so i don't see why that patch would have changed the install location.

i think a suitable fix would be to update the paths listed here: https://dxr.mozilla.org/mozilla-central/search?q=binscope
replacing references to:
C:/Program Files (x86)/Microsoft/SDL BinScope/BinScope.exe
with:
C:/Program Files/Microsoft BinScope 2014/Binscope.exe
taking care to also fix the path.join reference (testing/mozharness/configs/builds/taskcluster_base_win32.py)

Comment 4

[(Away)]

14:43:55     INFO - BinScope: The following requested checks were not found: APTCACheck, SNCheck

Binscope 2014 only supports these checks:

C:\Program Files\Microsoft BinScope 2014>binscope -listchecks
Microsoft BinScope 2014
ATLVersionCheck
ATLVulnCheck
AppContainerCheck
CompilerVersionCheck
DBCheck
DefaultGSCookieCheck
ExecutableImportsCheck
FunctionPointersCheck
GSCheck
GSFriendlyInitCheck
GSFunctionSafeBuffersCheck
HighEntropyVACheck
NXCheck
RSA32Check
SafeSEHCheck
SharedSectionCheck
VB6Check
WXCheck

Comment 5

[(Away)]

"Going forward, Binscope will be phased out in favor of BinSkim"

https://blogs.msdn.microsoft.com/secdevblog/2016/08/17/introducing-binskim/

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

if you find a version you'd like us to install on windows infra, let me know or submit a pr to https://github.com/mozilla-releng/OpenCloudConfig

Comment 7

[(Away)]

I don't want to sign up for the work to switch programs. By the time I learned about binskim, I already had some nearly-finished patches to get binscope working. I want to get these landed and file a followup for binskim.

Comment 8

[(Away)]

Attachment: Update checks for BinScope 2014.

For the sake of explicitness, I went ahead and listed out every possible check with a check-or-skip for each.

Comment 9

[(Away)]

Attachment: Allow BinScope to run on clang-cl builds

Comment 10

[(Away)]

Attachment: Update path to BinScope 2014 and make it available to all Windows builds.

Comment 11

[(Away)]

Attachment: Newer Binscope no longer communicates status via return code.

Comment 12

[(Away)]

Attachment: Run Binscope on more files

I'm all ears for a more wildcard-ey way to do this.

Comment 13

[Nathan Froyd [:froydnj]]

Comment on attachment 8963696 [details] [diff] [review]
Update path to BinScope 2014 and make it available to all Windows builds.

Review of attachment 8963696 [details] [diff] [review]:
-----------------------------------------------------------------

Are we able to complain somewhere if the path specified by BINSCOPE does not exist, so we can ensure that we change everything appropriately?

Comment 14

[Nathan Froyd [:froydnj]]

Comment on attachment 8963699 [details] [diff] [review]
Run Binscope on more files

Review of attachment 8963699 [details] [diff] [review]:
-----------------------------------------------------------------

I have no wildcard-y ways to do this ATM.  Maybe file a bug on setting a binscopeCheck flag on binaries or libraries?

Comment 15

[Nathan Froyd [:froydnj]]

Comment on attachment 8963693 [details] [diff] [review]
Update checks for BinScope 2014.

Review of attachment 8963693 [details] [diff] [review]:
-----------------------------------------------------------------

rs=me

Comment 16

[Nathan Froyd [:froydnj]]

Comment on attachment 8963697 [details] [diff] [review]
Newer Binscope no longer communicates status via return code.

Review of attachment 8963697 [details] [diff] [review]:
-----------------------------------------------------------------

Sigh at tools that don't communicate success or failure via exit code...

Comment 17

[(Away)]

> Are we able to complain somewhere if the path specified by BINSCOPE does not
> exist, so we can ensure that we change everything appropriately?

You probably found it moments later, but yes, one of the later patches does exactly that.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2d22f513669f
Update checks for BinScope 2014. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/991e17b4fafa
Allow BinScope to run on clang-cl builds. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/fd3cb62ee635
Update path to BinScope 2014 and make it available to all Windows builds. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/c6669ef7d04d
Newer Binscope no longer communicates status via return code. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/6a806cbc25a7
Run Binscope on more files. r=froydnj

Comment 19

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/2d22f513669f
https://hg.mozilla.org/mozilla-central/rev/991e17b4fafa
https://hg.mozilla.org/mozilla-central/rev/fd3cb62ee635
https://hg.mozilla.org/mozilla-central/rev/c6669ef7d04d
https://hg.mozilla.org/mozilla-central/rev/6a806cbc25a7