Closed Bug 563318 (msvc2010) Opened 14 years ago Closed 12 years ago

switch trunk builds to use Visual C++ 2010

Categories

(Firefox Build System :: General, defect, P5)

Product:
Firefox Build System
Component:
General
Platform:
x86
Windows Server 2003
Type:
defect

Tracking

(blocking2.0 -, status2.0 wontfix)

Status:
RESOLVED FIXED
Milestone:
mozilla13
Tracking Flags:
Tracking Status
blocking2.0 --- -
status2.0 --- wontfix

People

(Reporter: ted, Assigned: khuey)

References

(Blocks 1 open bug)

Details

(Whiteboard: [sg:want])

Attachments

(1 file)

Disable optimization to work around pgo crash [Checked in: See comment 39]
1.34 KB, patch
bzbarsky
: review+
Details | Diff | Splinter Review 
I'd like to switch trunk builds to use Visual C++ 2010 now that it's been
officially released. This should fix some bugs we have with Breakpad symbol
dumping, and it's also likely to give us a small performance increase.

There are a few small code issues left (marked as blocking this bug), and we'll need to get the compiler installed on the build slaves (also marked as blocking).
Depends on: 563319
Priority: -- → P5
OS: Windows 7 → Windows Server 2003
Whiteboard: [opsi][automation]
Depends on: 515492
No longer depends on: 563316
Someone over in bug 577486 comment 4 indicated that this will mean we won't run on the Windows 2000 OS anymore - is that correct?
It's not clear to me that we'll get this for Firefox 4 anyway, given the dependencies. (I'm also loathe to change compilers post-beta.)
I should point out here that community members made a specific point during the summit to ask for Win2k support, as most k-12 schools are still running it.
I hear comment 4, and would love to support all operating systems everywhere, but as Windows takes away support for Windows 2000 based systems, so should we. If we have hard data on what "most k-12 schools" breaks down to, that would be great signal to add to the conversation.

I've filed bug 579078 to track the Windows 2000 decision.
Regarding MSVC 2010, I noticed a credible claim recently that it "miscompiles ... many C++ programs when optimisations are turned on" [referring to 64-bit builds].

See http://lists.trolltech.com/pipermail/qt-interest/2010-July/025512.html

Something to consider before making a switch, perhaps.
In any event, given the bugs blocking this one, I feel that we're too far into the beta cycle to make this switch for 4.0. We can and should revisit for the future. (Is there a status2.0=wontfix option? I don't see it available.)

That's good to know about x64, although probably a separate issue. (We could ship our x86 and x64 builds with different compilers.)
blocking2.0: --- → -
Blocks: 520651
There's a status2.0 WONTFIX now :-)

I think we really really want this for Firefox.next though.  Being on a six year old compiler is kinda lame.
status2.0: --- → wontfix
(In reply to comment #6)
> Regarding MSVC 2010, I noticed a credible claim recently that it "miscompiles
> ... many C++ programs when optimisations are turned on" [referring to 64-bit
> builds].

And now there's SP1 which apparently has fixes for these problems ("There are specific problems in the x64 version of the C++ compiler (optimizer)."):
http://support.microsoft.com/kb/983509/en-us
Upgrading to VC 2010 will give us improved protection against stack-buffer-overflow exploits [1][2].

[1] http://blogs.technet.com/b/srd/archive/2009/03/20/enhanced-gs-in-visual-studio-2010.aspx
[2] http://blogs.msdn.com/b/ie/archive/2011/03/07/internet-explorer-9-security-part-1-enhanced-memory-protections.aspx
Whiteboard: [opsi][automation] → [opsi][automation][sg:want]
Using an old VC recently caused(?) a security hole, bug 641630.
I'm wondering which version of Firefox will start to use MSVC 2010.  That will be the first version of Firefox to officially drop Win2000 support since MSVC 2010 needs minimium of WinXP SP2/SP3.
We are hoping to target Firefox 8, but we won't know until we get all the dependencies of this bug fixed. When this bug is marked FIXED, we will set the "Target Milestone" to the version that we expect it to ship in.
(In reply to comment #12)
> I'm wondering which version of Firefox will start to use MSVC 2010.  That
> will be the first version of Firefox to officially drop Win2000 support
> since MSVC 2010 needs minimium of WinXP SP2/SP3.

The current plan is Firefox 8.  There will be a post to dev.planning soon about EOL for Win2k and Win XP RTM and SP1.
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #14)

> 
> The current plan is Firefox 8.  There will be a post to dev.planning soon
> about EOL for Win2k and Win XP RTM and SP1.

looks like that plan is seriously out the window as FF8 is now in the Aurora channel and I've recently installed FF8.0a2 on a win2000 machine and it runs on there.
plus I so NO EOL for win2000/xp rtm-sp1 post on dev.planning yet (hence the word "delayed" or "postponed")

sorry Ted & Kyle.  maybe when Firefox 10 or 11 comes out, perhaps either one of those versions will start using MSVC 2010.
Yes, this has been delayed.  We're looking at probably Firefox 10 now.
Blocks: 609976
Depends on: 681893
Depends on: 682182
Depends on: 684227
Blocks: 684500
Blocks: 673518
Reminder - we need to change the config for the update server (bug 682182) prior to switching the compiler, to avoid serving builds that don't work to nightly users.
Blocks: 699246
Blocks: 699247
No longer depends on: 602908
Depends on: 701700
I'm hoping to hit Firefox 12 with this.
I'm going to drive this to completion.  After the CRT is deployed on the debug machines this isn't really a releng bug anymore anyways.
Assignee: nobody → khuey
Blocks: 709193
Reading http://www.zachburlingame.com/2011/03/targeting-windows-2000xp-rtmxp-sp1-from-visual-studio-2010/ it seems the problem that makes win2k and xpsp1 non supported is the CRT, and it looks like it is possible to build with 2010 and link with an older CRT.

It means the choice is not necessarily whether to drop support for win2k and xpsp1.
So it looks like the settings at that url actually just use the msvc2008 compiler :(

OTOH, there are interesting things in
http://stackoverflow.com/questions/2484511/can-i-use-visual-studio-2010s-c-compiler-with-visual-studio-2008s-c-runtim

One of them is to declare the two missing functions from old kernel32.dll in one of our dlls (mozutils would be a good candidate)
It will make the app less secure to replace EncodePointer/DecodePointer with dummy functions. Moreover, it is not likely to be supported by Microsoft.
pre xp2 support is not that hard to drop now there smaller os marketshare and we need to compete with chrome on speed/performance
I have a fix for the VS2010 runtime library not supporting pre XP SP2.  If that works (which I believe it does), then we won't have to drop support for any operating system merely by just switching to VS2010.

I will file a bug and attach a patch after more testing and investigation.
(In reply to Ehsan Akhgari [:ehsan] from comment #24)
> I have a fix for the VS2010 runtime library not supporting pre XP SP2.  If
> that works (which I believe it does), then we won't have to drop support for
> any operating system merely by just switching to VS2010.
> 
> I will file a bug and attach a patch after more testing and investigation.

Ehsan, how do you know that EncodePointer/DecodePointer is the only important dependency on SP2+ that the library has?

Please CC me on the new bug.
(In reply to Brian Smith (:bsmith) from comment #25)
> Ehsan, how do you know that EncodePointer/DecodePointer is the only
> important dependency on SP2+ that the library has?
> 
> Please CC me on the new bug.

+1 on both :-)
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #26)
> (In reply to Brian Smith (:bsmith) from comment #25)
> > Ehsan, how do you know that EncodePointer/DecodePointer is the only
> > important dependency on SP2+ that the library has?
> > 
> > Please CC me on the new bug.
> 
> +1 on both :-)

This project turned out to be a lot trickier than it seemed to be at first.  We've asked help from Microsoft about this, and I have suspended my efforts until we hear back from them.
(In reply to Ehsan Akhgari [:ehsan] from comment #27)
> (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #26)
> > (In reply to Brian Smith (:bsmith) from comment #25)
> > > Ehsan, how do you know that EncodePointer/DecodePointer is the only
> > > important dependency on SP2+ that the library has?
> > > 
> > > Please CC me on the new bug.
> > 
> > +1 on both :-)
> 
> This project turned out to be a lot trickier than it seemed to be at first. 
> We've asked help from Microsoft about this, and I have suspended my efforts
> until we hear back from them.

did you hear back 

i dont think you will get a response unless you pay since thats not a supported platform if im not mistaken
(In reply to gabe from comment #28)
> (In reply to Ehsan Akhgari [:ehsan] from comment #27)
> > (In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #26)
> > > (In reply to Brian Smith (:bsmith) from comment #25)
> > > > Ehsan, how do you know that EncodePointer/DecodePointer is the only
> > > > important dependency on SP2+ that the library has?
> > > > 
> > > > Please CC me on the new bug.
> > > 
> > > +1 on both :-)
> > 
> > This project turned out to be a lot trickier than it seemed to be at first. 
> > We've asked help from Microsoft about this, and I have suspended my efforts
> > until we hear back from them.
> 
> did you hear back 

No word from Microsoft yet.
You may want to do this before/after ESR:
*before: get rid of old environments at Mozilla sooner.
*after: (last) longer support to run on older systems.
Component: Release Engineering → Build Config
Product: mozilla.org → Core
QA Contact: release → build-config
Whiteboard: [opsi][automation][sg:want] → [sg:want]
Version: other → unspecified
Windows SQLite 3.7.9 PGO builds crash when built with MSVC 2005. From my personal experience, they do not with MSVC 2010. With an ESR based on Fx10 about to ship, can this move forward?
We're still waiting on metrics data to determine how many users we'd be dropping.
No longer blocks: RequireWin7SDK
https://hg.mozilla.org/mozilla-central/rev/fc1202d75165
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Unfortunately this had to come out because it was causing a crash during the profiling run that look suspiciously like a crash bholley saw during the profiling run on a try push last week.

https://hg.mozilla.org/mozilla-central/rev/b506d48ef7aa
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
It appears to be a compiler bug ... I'm testing a workaround now.
Are we using Visual C++ 2010 with SP1 [10.0.40219.10] or without it?
This successfully works around the problem, and shows no perf regression on the microbenchmark bz constructed.
Attachment #593522 - Flags: review?(bzbarsky)
Comment on attachment 593522 [details] [diff] [review]
Disable optimization to work around pgo crash
[Checked in: See comment 39]

r=me, but please file a bug on our side tracking reverting this, and put _that_ bug number in the comment (possibly in addition to this one)?
Attachment #593522 - Flags: review?(bzbarsky) → review+
Blocks: 723158
Depends on: 617819
(In reply to Ehsan Akhgari [:ehsan] from comment #29)
> 
> No word from Microsoft yet.

The solution described here (http://stackoverflow.com/questions/2484511/can-i-use-visual-studio-2010s-c-compiler-with-visual-studio-2008s-c-runtim/3502056#3502056) seems feasible.
The author says "With some effort it would be possible to use LoadLibrary / GetProcAddress to provide the real functionality where OS offers is", so we wouldn't lose security.
http://support.microsoft.com/kb/2005279
It's possible to use the VC++ 2008 runtime using VC++ 2010.
We punted on the issue and simply dropped support for Win2k and WinXP <SP2.
(In reply to Marco Castelluccio from comment #40)
> (In reply to Ehsan Akhgari [:ehsan] from comment #29)
> > 
> > No word from Microsoft yet.
> 
> The solution described here
> (http://stackoverflow.com/questions/2484511/can-i-use-visual-studio-2010s-c-
> compiler-with-visual-studio-2008s-c-runtim/3502056#3502056) seems feasible.
> The author says "With some effort it would be possible to use LoadLibrary /
> GetProcAddress to provide the real functionality where OS offers is", so we
> wouldn't lose security.

To elaborate, that would prevent us from using certain newer compiler features, which is why it was not acceptable.
Attachment #593522 - Attachment description: Disable optimization to work around pgo crash → Disable optimization to work around pgo crash [Checked in: See comment 39]
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #18)
> I'm hoping to hit Firefox 12 with this.

Most likely Firefox 13.  Asa Dotzler recently wrote a blog on his site that FF 12 will be the last version to support Win2000:
http://weblogs.mozillazine.org/asa/archives/2012/01/end_of_firefox_win2k.html

(In reply to Ehsan Akhgari [:ehsan] from comment #43)
> (In reply to Marco Castelluccio from comment #40)
> > (In reply to Ehsan Akhgari [:ehsan] from comment #29)
> > > 
> > > No word from Microsoft yet.
> > 
> > The solution described here
> > (http://stackoverflow.com/questions/2484511/can-i-use-visual-studio-2010s-c-
> > compiler-with-visual-studio-2008s-c-runtim/3502056#3502056) seems feasible.
> > The author says "With some effort it would be possible to use LoadLibrary /
> > GetProcAddress to provide the real functionality where OS offers is", so we
> > wouldn't lose security.
> 
> To elaborate, that would prevent us from using certain newer compiler
> features, which is why it was not acceptable.

So Ehsan you still haven't heard from Microsoft.  Guess you may have to abandon your little project altogether and accept reality of support for Win2000 and WinXP RTM/SP1 being dropped.
(In reply to erpman1 from comment #44)
> Most likely Firefox 13.  Asa Dotzler recently wrote a blog on his site that
...
> So Ehsan you still haven't heard from Microsoft.  Guess you may have to
> abandon your little project altogether and accept reality of support for
> Win2000 and WinXP RTM/SP1 being dropped.

The switch has already been made, which is why this bug is marked as resolved fixed (see comment 39).
Target Milestone: --- → mozilla13
Depends on: 726570
>Are we using Visual C++ 2010 with SP1 [10.0.40219.10] or without it?

For the record (and in case anyone else wonders), we're on MSVC 2010 *without* SP1.
(In reply to Gian-Carlo Pascutto (:gcp) from comment #46)
> >Are we using Visual C++ 2010 with SP1 [10.0.40219.10] or without it?
> 
> For the record (and in case anyone else wonders), we're on MSVC 2010
> *without* SP1.

Ah, that's good to know - unfortunately SP1 would fix a compiler bug, from what I heard. See http://support.microsoft.com/kb/983509 - esp. the "C++ Compiler" section.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #47)
> Ah, that's good to know - unfortunately SP1 would fix a compiler bug, from
> what I heard. See http://support.microsoft.com/kb/983509 - esp. the "C++
> Compiler" section.

I don't think it would fix the compiler bugs we've hit (judging from my local testing with MSVC 2010 SP1).
it would fix a compiler optimisations bug where SSE2 instructions are used in the SSE1 profile; causing illegal instruction errors on SSE1 processors as a result.
(In reply to Gian-Carlo Pascutto (:gcp) from comment #46)
> >Are we using Visual C++ 2010 with SP1 [10.0.40219.10] or without it?
> 
> For the record (and in case anyone else wonders), we're on MSVC 2010
> *without* SP1.

May I ask, why SP1 not used?

It seems that SP1 fixes many important bugs/issues:
http://blogs.msdn.com/b/jasonz/archive/2010/12/20/visual-studio-2010-service-pack-1-beta-feedback.aspx
(In reply to Danial Horton from comment #49)
> it would fix a compiler optimisations bug where SSE2 instructions are used
> in the SSE1 profile; causing illegal instruction errors on SSE1 processors
> as a result.

Which doesn't affect us.  We don't compile with -arch:SSE

(In reply to Boaz Dodin from comment #51)
> (In reply to Gian-Carlo Pascutto (:gcp) from comment #46)
> > >Are we using Visual C++ 2010 with SP1 [10.0.40219.10] or without it?
> > 
> > For the record (and in case anyone else wonders), we're on MSVC 2010
> > *without* SP1.
> 
> May I ask, why SP1 not used?

Because that's what got installed on the machine and I didn't pay close enough attention when qualifying the install :-)

> It seems that SP1 fixes many important bugs/issues:
> http://blogs.msdn.com/b/jasonz/archive/2010/12/20/visual-studio-2010-service-
> pack-1-beta-feedback.aspx

Of that list, the only issue that might affect us is https://connect.microsoft.com/VisualStudio/feedback/details/336316.  The rest are all related to .NET, x64, or STL things that we don't use.  Returning a copy of a non-POD type constructed on the stack isn't a common pattern in our codebase, but that one is worth watching out for.
the c++ runtime memory leak may also be indirectly tripped though.
Blocks: 741004
No longer blocks: 723158
Blocks: 750661
No longer blocks: 750661
Blocks: 790228
No longer blocks: 684500
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: