Closed Bug 1448306 Opened 4 years ago Closed 4 years ago

BinScope seems to have stopped working on builds

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(firefox61 fixed)

RESOLVED FIXED
Tracking Status
firefox61 --- fixed

People

(Reporter: away, Assigned: away)

References

Details

(Keywords: sec-audit, sec-want)

Attachments

(5 files)

From a recent m-c Win32 opt build task:
12:19:43     INFO - Could not locate binscope at location : C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe
12:19:43     INFO - Binscope wasn't installed or the BINSCOPE env variable wasn't set correctly, skipping this check and exiting...

And from win64 opt:
21:34:27     INFO - BINSCOPE environment variable is not set, can't check DEP/ASLR etc. status.

BinScope verifies that our binaries follow MS security recommendations, so failing to run this tool could lead to uncaught regressions.

First thing to check would be whether "C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe" exists on our builders nowadays.

I don't know who to start with, or even if I'm in the right component. Catlee could you help route this please?
Flags: needinfo?(catlee)
See Also: → 1246550
I'm not sure....It's possible that BINSCOPE isn't being set correctly, you could look at changes to taskcluster/ to see if anything jumps out.

Otherwise, you could ask :grenade or :pmoore to see if anything has changed on the workers lately.

Should failure to run binscope be made into a fatal error?
Flags: needinfo?(catlee)
:grenade, does "C:\Program Files (x86)\Microsoft\SDL BinScope\BinScope.exe" exist on our builders nowadays?

> Should failure to run binscope be made into a fatal error?

I would claim yes.
Flags: needinfo?(rthijssen)
Blocks: 1443590
it looks like binscope is installed at: C:\Program Files\Microsoft BinScope 2014\Binscope.exe

here is a task that lists the contents of C:\Program Files\Microsoft BinScope 2014:
https://tools.taskcluster.net/groups/FHtI9j7uRISF7eQPB8m2Ow/tasks/FHtI9j7uRISF7eQPB8m2Ow/runs/0/logs/public%2Flogs%2Flive.log

i'm not sure how or why the path differs from the one in the mozharness configs. we did recently patch (https://github.com/mozilla-releng/OpenCloudConfig/commit/b58a67f3b54e10085232aa9f39cb7426bf145592) the builder manifests changing the source url for the binscope installer from github (https://github.com/mozilla-releng/OpenCloudConfig/raw/master/userdata/Configuration/FirefoxBuildResources/BinScope_x64.msi) to s3 (https://s3.amazonaws.com/windows-opencloudconfig-packages/binscope/BinScope_x64.msi) but the binary artefact sha512sum for both of those artefacts is identical so i don't see why that patch would have changed the install location.

i think a suitable fix would be to update the paths listed here: https://dxr.mozilla.org/mozilla-central/search?q=binscope
replacing references to:
C:/Program Files (x86)/Microsoft/SDL BinScope/BinScope.exe
with:
C:/Program Files/Microsoft BinScope 2014/Binscope.exe
taking care to also fix the path.join reference (testing/mozharness/configs/builds/taskcluster_base_win32.py)
Flags: needinfo?(rthijssen)
14:43:55     INFO - BinScope: The following requested checks were not found: APTCACheck, SNCheck

Binscope 2014 only supports these checks:

C:\Program Files\Microsoft BinScope 2014>binscope -listchecks
Microsoft BinScope 2014
ATLVersionCheck
ATLVulnCheck
AppContainerCheck
CompilerVersionCheck
DBCheck
DefaultGSCookieCheck
ExecutableImportsCheck
FunctionPointersCheck
GSCheck
GSFriendlyInitCheck
GSFunctionSafeBuffersCheck
HighEntropyVACheck
NXCheck
RSA32Check
SafeSEHCheck
SharedSectionCheck
VB6Check
WXCheck
Assignee: nobody → dmajor
"Going forward, Binscope will be phased out in favor of BinSkim"

https://blogs.msdn.microsoft.com/secdevblog/2016/08/17/introducing-binskim/
Depends on: 1449951
if you find a version you'd like us to install on windows infra, let me know or submit a pr to https://github.com/mozilla-releng/OpenCloudConfig
I don't want to sign up for the work to switch programs. By the time I learned about binskim, I already had some nearly-finished patches to get binscope working. I want to get these landed and file a followup for binskim.
For the sake of explicitness, I went ahead and listed out every possible check with a check-or-skip for each.
Attachment #8963693 - Flags: review?(core-build-config-reviews)
Attachment #8963694 - Flags: review?(core-build-config-reviews)
Attachment #8963696 - Flags: review?(core-build-config-reviews)
Attachment #8963697 - Flags: review?(core-build-config-reviews)
I'm all ears for a more wildcard-ey way to do this.
Attachment #8963699 - Flags: review?(core-build-config-reviews)
Attachment #8963694 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 8963696 [details] [diff] [review]
Update path to BinScope 2014 and make it available to all Windows builds.

Review of attachment 8963696 [details] [diff] [review]:
-----------------------------------------------------------------

Are we able to complain somewhere if the path specified by BINSCOPE does not exist, so we can ensure that we change everything appropriately?
Attachment #8963696 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 8963699 [details] [diff] [review]
Run Binscope on more files

Review of attachment 8963699 [details] [diff] [review]:
-----------------------------------------------------------------

I have no wildcard-y ways to do this ATM.  Maybe file a bug on setting a binscopeCheck flag on binaries or libraries?
Attachment #8963699 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 8963693 [details] [diff] [review]
Update checks for BinScope 2014.

Review of attachment 8963693 [details] [diff] [review]:
-----------------------------------------------------------------

rs=me
Attachment #8963693 - Flags: review?(core-build-config-reviews) → review+
Comment on attachment 8963697 [details] [diff] [review]
Newer Binscope no longer communicates status via return code.

Review of attachment 8963697 [details] [diff] [review]:
-----------------------------------------------------------------

Sigh at tools that don't communicate success or failure via exit code...
Attachment #8963697 - Flags: review?(core-build-config-reviews) → review+
> Are we able to complain somewhere if the path specified by BINSCOPE does not
> exist, so we can ensure that we change everything appropriately?

You probably found it moments later, but yes, one of the later patches does exactly that.
Pushed by dmajor@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2d22f513669f
Update checks for BinScope 2014. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/991e17b4fafa
Allow BinScope to run on clang-cl builds. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/fd3cb62ee635
Update path to BinScope 2014 and make it available to all Windows builds. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/c6669ef7d04d
Newer Binscope no longer communicates status via return code. r=froydnj
https://hg.mozilla.org/integration/mozilla-inbound/rev/6a806cbc25a7
Run Binscope on more files. r=froydnj
Blocks: 1450088
See Also: → 1412918
Blocks: 1450089
You need to log in before you can comment on or make changes to this bug.