Closed
Bug 1186060
(vs2015)
Opened 9 years ago
Closed 8 years ago
Use Visual Studio 2015 Update 1 for release builds
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(firefox42 affected, firefox48 fixed)
RESOLVED
FIXED
mozilla48
People
(Reporter: briansmith, Assigned: gps)
References
(Depends on 1 open bug, Blocks 2 open bugs)
Details
Attachments
(1 file)
Let's make bug 1119082 be about getting Visual Studio 2015 support to the point where we support Visual Studio 2015 as a tier 2 type platform, where we try to get bugs that break Visual Studio 2015 builds fixed ASAP. Let's have *this* bug track making Visual Studio 2015 the compiler for actual releases.
Comment 1•9 years ago
|
||
What's the primary motivation here? Updating toolchains is a fair bit of work, it's good to know why we're doing it.
Reporter | ||
Comment 2•9 years ago
|
||
The primary motivation is to facilitate bug 1186064 to make more C++ language and library features available in Gecko. See http://blogs.msdn.com/b/vcblog/archive/2015/06/19/c-11-14-17-features-in-vs-2015-rtm.aspx about the new features in VS2015. This will allow more Mozilla-isms to be removed and make code easier to write.
Comment 3•9 years ago
|
||
I second that. We absolutely should not let 2013 become the next 2010. I would really like to see us switch to 2015 this year, and also do it through tooltool instead of waiting for ages for the new Visual Studio install to be deployed to all of our build machine, which is a huge part of why updating toolchains is difficult.
Comment 4•9 years ago
|
||
On a separate note, there's something I've never seen talked about, but with the rate at which we're upgrading, maybe that should weigh in: the cost of upgrade for third parties. I'm saying this without knowing how much it costs to upgrade from 2013 to 2015, nor how much it cost to upgrade from 2010 to 2013, but for third parties that do non-open-source derivative work (addons for Firefox, apps based on Gecko, etc. ; those are not hypothetical, there are plenty of them), a bump in the minimum version we require is a hit in the wallet. And we already forced 2013 only 6 months ago. Not saying this is a blocker and we shouldn't bump to 2015, but that those things should be part of the conversation as well, especially considering the timeline.
Assignee | ||
Comment 5•9 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #4) Isn't the concern about wallet impact nullified by Visual Studio Community Edition, which is free for open source? IMO Community Edition is a huge reason to upgrade because everyone will be able to use a free and powerful edition instead of a) paying b) using the feature-pruned Express edition. I do hear the concern about upgrading too frequently. Visual Studio installations are massive and not a lot of fun to deal with.
Comment 6•9 years ago
|
||
(In reply to Gregory Szorc [:gps] from comment #5) > (In reply to Mike Hommey [:glandium] from comment #4) > > Isn't the concern about wallet impact nullified by Visual Studio Community > Edition, which is free for open source? You missed the part where I said non-open-source. To cite one that I'm aware of because they're having problems at the moment (unrelated to MSVC version): Symantec's Norton toolbar is one example.
Comment 7•9 years ago
|
||
MSVC has had a free edition (Express) for years which can be used for open source or closed source development. The only thing that the Community Edition gives you on top of that is the IDE, and since side by side installations of Visual Studio works perfectly fine, you could install VS2015 Express and keep your old 2010/etc edition for debugging and other IDE related stuff. I don't think we are forcing a monetary cost on these consumers.
Comment 8•9 years ago
|
||
Any plans to move this forward? This seems pretty important to me.
Comment 9•9 years ago
|
||
Nobody has given a super-compelling reason to do it, it's a lot of work, and the dependent bugs aren't even all fixed, so I'm not sure it's even feasible yet.
Comment 10•8 years ago
|
||
With bug 1124017 fixed, a VS2015 build of Firefox runs perfectly fine in XP SP2 and SP3! AFAICT, we still need to fix CRT DLL packaging (bug 1204202) and get VS2015 toolchain on the build machines (bug 1186060 to start).
Assignee | ||
Comment 11•8 years ago
|
||
FWIW, Chromium appears to have switched to VS2015 ~2 months ago (https://codereview.chromium.org/1514853004/). A PGO bug fix was cited in https://codereview.chromium.org/1514853004/#msg11.
Assignee | ||
Comment 12•8 years ago
|
||
Preliminary measurements in bug 1250797 indicate that VS2015 PGO builds are ~1.4x faster than VS2013.
Assignee | ||
Comment 13•8 years ago
|
||
Per bug 1250797 comment 12, PGO builds on VS2015 are ~1.8x than VS2013. The speedup increased over comment 12 because the 2nd PGO link phase is nearly 3x faster on VS2015 (the instrumentation phase is only ~2x faster). I haven't yet tested the speedup in automation. But surely there will be a speedup.
Depends on: 1250797
Assignee | ||
Comment 14•8 years ago
|
||
I've been assigned to shepherd this from the build system perspective.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Summary: Use Visual Studio 2015 for release builds → Use Visual Studio 2015 Update 1 for release builds
Assignee | ||
Comment 15•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=08609e0638d4
Assignee | ||
Comment 16•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d5ea35507602
Assignee | ||
Comment 17•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a64ed8bafd11
Assignee | ||
Comment 18•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7e3697dc8cb
Assignee | ||
Comment 19•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5acec37189bb
Assignee | ||
Comment 20•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=19ffbd06c5ae
Assignee | ||
Comment 21•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1a67e90648f4
Assignee | ||
Comment 22•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=964a9857eb5f
Assignee | ||
Comment 23•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ddc9925620d9
Assignee | ||
Comment 24•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e31aaf0de0c8
Assignee | ||
Comment 25•8 years ago
|
||
I appear to have 64-bit builds mostly working (minus the crash reporter): https://treeherder.mozilla.org/#/jobs?repo=try&revision=343a8e570f7a I'm optimistic a PGO try run at https://treeherder.mozilla.org/#/jobs?repo=try&revision=95a24d38f004 will complete in 1-2 hours. Bug 1242005 is preventing 32-bit builds from completing due to a compiler warning being promoted to an error :( Hopefully we can start looking for Talos and other test regressions in the next ~24 hours...
Comment 26•8 years ago
|
||
(In reply to Gregory Szorc [:gps] from comment #25) > I appear to have 64-bit builds mostly working (minus the crash reporter): > https://treeherder.mozilla.org/#/jobs?repo=try&revision=343a8e570f7a The latest patch in bug 623183 should fix the crash reporter bit.
Assignee | ||
Comment 27•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6f9cec2a021f
Assignee | ||
Comment 28•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c4e9e45827be
Assignee | ||
Comment 29•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=681aa58f1d63
Assignee | ||
Comment 30•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d5d82049dce0
Assignee | ||
Comment 31•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b6bab2a1b64d
Assignee | ||
Comment 32•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=32d8ecb70937
Assignee | ||
Comment 33•8 years ago
|
||
(THIS PATCH IS NOT READY FOR FINAL REVIEW BECAUSE OF THE HACKINESS AROUND INSTALLING THE TOOLCHAIN. I'm submitting the patch now to test the waters around the approach.) This commit switches Windows builds from Visual Studio 2013 to Visual Studio 2015 Update 1. Previously, Visual Studio was installed on the builers as part of the base system image. Starting with this commit, we obtain Visual Studio from a pre-generated, self-contained archive containing the executables, Windows SDK, and other support files. This means that new Windows toolchains can be installed without having to modify configuration of machines in automation! The mozconfigs for Visual Studio 2015 are a bit different from existing mozconfigs. Because it appears to be completely redundant and not necessary, the LIBPATH variable has been dropped. The order of paths in PATH, LIB, and INCLUDE has changed. The new order more accurately reflects what would be defined by vcvarsall.bat. As part of switching to Visual Studio 2015, the Universal CRT is now required. So, the 2015 mozconfigs export WIN_UCRT_REDIST_DIR to define the location to those files. The switch to Visual Studio 2015 also involves the switch from the Windows 8.1 SDK to the Windows 10 SDK. However, we still target an old version of Windows, so this hopefully shouldn't have any significant fallout. It's worth noting that switching to Visual Studio 2015 makes builds - especially PGO builds - significantly faster. Our PGO build times in automation are ~1 hour faster. Our regular builds appear to be a few minutes faster. Review commit: https://reviewboard.mozilla.org/r/38833/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/38833/
Attachment #8728174 -
Flags: review?(ted)
Attachment #8728174 -
Flags: review?(ehsan)
Comment 34•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review35499 ::: python/mozbuild/mozbuild/mach_commands.py:324 (Diff revision 1) > + # Hacky Windows only VS2015 toolchain application. > + from cStringIO import StringIO > + import shutil > + import requests > + import zipfile > + > + msvc_dir = os.path.join(self.topsrcdir, 'vstoolchain') > + if os.path.exists(msvc_dir): > + shutil.rmtree(msvc_dir) > + > + os.mkdir(msvc_dir) > + print('downloading Visual Studio 2015 toolchain...') > + toolchain = StringIO(requests.get('https://s3-us-west-2.amazonaws.com/moz-packages/temp/vs2015-3.zip').content) > + toolchain.seek(0) > + print('uncompressing') > + with zipfile.ZipFile(toolchain, mode='r') as zf: > + zf.extractall(msvc_dir) > + print('finished installing Visual Studio 2015 toolchain') WTF is this?
Assignee | ||
Comment 35•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review35499 > WTF is this? The reason for the UPPERCASE comment in the commit message. The plan is to use tooltool in the final version.
Comment 36•8 years ago
|
||
Comment on attachment 8728174 [details] MozReview Request: Bug 1186060 - Build with Visual Studio 2015 Update 1; r?ted, ehsan https://reviewboard.mozilla.org/r/38833/#review35681 Looks good to me, sans the changes in mach_commands.py which I expect will be removed when this package is put into tooltool. Out of curiosity, why haven't you put this into tooltool yet?
Attachment #8728174 -
Flags: review?(ehsan) → review+
Assignee | ||
Comment 37•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=01ae96c776b0
Assignee | ||
Comment 38•8 years ago
|
||
Comment on attachment 8728174 [details] MozReview Request: Bug 1186060 - Build with Visual Studio 2015 Update 1; r?ted, ehsan Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38833/diff/1-2/
Assignee | ||
Comment 39•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e9b167389ff5
Assignee | ||
Comment 40•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review35681 The latest patch has things in tooltool (as an internal file). I was waiting to put things in tooltool because I didn't want to upload a few GB of archives I knew would likely have to be thrown away.
Assignee | ||
Comment 41•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7414dbee645e
Assignee | ||
Comment 42•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7943c53e4341
Assignee | ||
Comment 43•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6e326afc28bd
Assignee | ||
Updated•8 years ago
|
Alias: vs2015
Assignee | ||
Comment 44•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=56b27306d325
Comment 45•8 years ago
|
||
Comment on attachment 8728174 [details] MozReview Request: Bug 1186060 - Build with Visual Studio 2015 Update 1; r?ted, ehsan https://reviewboard.mozilla.org/r/38833/#review36193 ::: browser/config/mozconfigs/win32/l10n-mozconfig:14 (Diff revision 2) > export MOZILLA_OFFICIAL=1 > > if test "$PROCESSOR_ARCHITECTURE" = "AMD64" -o "$PROCESSOR_ARCHITEW6432" = "AMD64"; then > - . $topsrcdir/build/win32/mozconfig.vs2013-win64 > + . $topsrcdir/build/win32/mozconfig.vs2015-win64 > else > . $topsrcdir/build/win32/mozconfig.vs2010 Do we have anything that's still running on 32-bit Windows machines that needs this? (You don't necessarily have to fix this now.) ::: browser/config/tooltool-manifests/win32/releng.manifest:23 (Diff revision 2) > "algorithm": "sha512", > "filename": "sccache.tar.bz2", > "unpack": true > +}, > +{ > +"size": 441571009, Per glandium's other bug, it would be nice to have a version annotation here indicating exactly what version of VC2015 this is. ::: build/win32/mozconfig.vs2015-win64:13 (Diff revision 2) > +export PATH="${vspath}/VC/redist/x86/Microsoft.VC140.CRT:${vspath}/VC/redist/x64/Microsoft.VC140.CRT:${vspath}/SDK/Redist/ucrt/DLLs/x86:${vspath}/SDK/Redist/ucrt/DLLs/x64:${PATH}" > + > +export INCLUDE="${vspath}/VC/include:${vspath}/VC/atlmfc/include:${vspath}/SDK/Include/ucrt:${vspath}/SDK/Include/shared:${vspath}/SDK/Include/um:${vspath}/SDK/Include/winrt:${vspath}/DIASDK/include" > +export LIB="${vspath}/VC/lib:${vspath}/VC/atlmfc/lib:${vspath}/SDK/lib/ucrt/x86:${vspath}/SDK/lib/um/x86:${vspath}/DIASDK/lib" > + > +. $topsrcdir/build/mozconfig.vs-common Do we still need this? https://dxr.mozilla.org/mozilla-central/source/build/mozconfig.vs-common ::: build/win64/mozconfig.vs2015:1 (Diff revision 2) > +vspath="$(cd ${topsrcdir} && pwd)/vs2015u1" These two mozconfigs look similar enough that it feels like you could make them shared with a parameter or two, but I'll leave it up to your judgement whether you think that's warranted. ::: js/src/devtools/automation/winbuildenv.sh:1 (Diff revision 2) > # We will be sourcing mozconfig files, which end up calling mk_add_options with I have...never seen this file before. I assume that since you've got things going on Try you'd notice if you actually broke this!
Attachment #8728174 -
Flags: review?(ted) → review+
Assignee | ||
Comment 46•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review36193 > I have...never seen this file before. I assume that since you've got things going on Try you'd notice if you actually broke this! Well, I don't have SM working in automation yet. See bug 1255686. That's one of the major open issues.
Assignee | ||
Comment 47•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=75b148dde242
Assignee | ||
Comment 48•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0365ae0244d3
Assignee | ||
Comment 49•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=85deef52c235
Assignee | ||
Comment 50•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=aa9d03e0fbd4
Assignee | ||
Comment 51•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0097f669ae5d
Assignee | ||
Comment 52•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d03ab9d3cf4d
Assignee | ||
Comment 53•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4351078c1300
Assignee | ||
Comment 54•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=91ca36427786
Assignee | ||
Comment 55•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=40070b35e02a
Assignee | ||
Comment 56•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce79b371319c
Assignee | ||
Comment 57•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b9dea0a02c72
Assignee | ||
Comment 58•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b222126e31f6
Assignee | ||
Comment 59•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=20fec478a0ca
Assignee | ||
Comment 60•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bbad05e94d7d
Assignee | ||
Comment 61•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c3d750c6b0ea
Assignee | ||
Comment 62•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ae4123b6c7a2
Assignee | ||
Comment 63•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8959e6fc49e7
Assignee | ||
Comment 64•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6e1e1b8b85b9
Assignee | ||
Comment 65•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=19375807e0aa
Assignee | ||
Comment 66•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2972fcd15202
Assignee | ||
Comment 67•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8e1d9322216d
Assignee | ||
Comment 68•8 years ago
|
||
As of yesterday, some SM builds are working. However, some are failing because vcruntime140d.dll isn't on PATH. And it isn't in the zip archive either, so that will need to be updated. Also, the Try spam is to get bug 1124033 sorted out.
Assignee | ||
Comment 69•8 years ago
|
||
According to glandium SM builds are apparently configured incorrectly and shouldn't be using the debug runtime. Will file another bug.
Assignee | ||
Comment 70•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f7906d704ec2
Assignee | ||
Comment 71•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=63721d32a553
Assignee | ||
Comment 72•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=85f14a870957
Assignee | ||
Comment 73•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8060581c5cd8
Assignee | ||
Comment 74•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9f25497849c3
Assignee | ||
Comment 75•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=fa8680594f68
Assignee | ||
Comment 76•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=08bcf938fcd9
Assignee | ||
Comment 77•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e442a949d8de
Assignee | ||
Comment 78•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=87f5a0bd79b3
Assignee | ||
Comment 79•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3b33e3daf339
Assignee | ||
Comment 80•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b579fcf75f79
Assignee | ||
Comment 81•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review36193 > Do we still need this? > https://dxr.mozilla.org/mozilla-central/source/build/mozconfig.vs-common I'm, uh, not sure. Feels like a follow-up bug to me. > These two mozconfigs look similar enough that it feels like you could make them shared with a parameter or two, but I'll leave it up to your judgement whether you think that's warranted. Yes, they are pretty similar. However, I'm leaning towards holding off on this. I'd rather do the tidying as a follow-up, since what this patch implements conforms to the existing "style" of mozconfigs more or less.
Assignee | ||
Comment 82•8 years ago
|
||
Comment on attachment 8728174 [details] MozReview Request: Bug 1186060 - Build with Visual Studio 2015 Update 1; r?ted, ehsan Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38833/diff/2-3/
Assignee | ||
Comment 83•8 years ago
|
||
Comment on attachment 8728174 [details] MozReview Request: Bug 1186060 - Build with Visual Studio 2015 Update 1; r?ted, ehsan Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38833/diff/3-4/
Comment 86•8 years ago
|
||
https://reviewboard.mozilla.org/r/38833/#review36193 > I'm, uh, not sure. Feels like a follow-up bug to me. Can you file a followup on this?
Assignee | ||
Comment 87•8 years ago
|
||
For posterity, the zip file containing the Visual Studio 2015 and SDK files was produced in the following manner: 1) I installed Windows 7 Ultimate 64-bit into a Hyper-V VM running on my personal desktop at home from a physical DVD that was purchased from Microsoft 2) All required updates through Windows Update were installed. This required multiple reboots. This process started a few minutes after finishing the OS install and finished the morning after (I slept in the middle with the host OS locked). 3) The instructions at https://hg.mozilla.org/mozilla-central/file/8ae2e73a7d90/build/docs/toolchains.rst#l43 were followed to install VS2015u1 Community and produce a zip archive. 4) I copied the zip file to a separate Linux VM to upload it to the internal tooltool (I verified the SHA-512 to ensure the file was identical across copying) For the tin foil hat crowd, yes, there is a single point of trust and you have to trust that I didn't act maliciously or that my home machine / VM wasn't compromised. I would encourage people to reproduce my steps and compare against what I uploaded to tooltool. I'm actually quite interested in seeing if someone else can get a bit-for-bit identical zip archive. In theory it is possible. But so far only I have produced the zip archives. I would love for someday this process to be performed in our automation. I'm guessing this is blocked on TaskCluster support for Windows.
Comment 88•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/b942c98f56c4
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox48:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Comment 89•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/559a80645f20
Updated•6 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•