752004 - (winclang) Allow building Firefox with clang-cl on Windows

Status: RESOLVED FIXED

(Firefox :: General, defect)

Comment 1

[(not currently active) Ted Mielczarek]

I assume the bug summary should be more like "allow building Firefox with Clang on Windows"? The goal is not to change our official builds or anything like that, just to make it possible?

Comment 2

[(no longer active)]

That's correct!

Comment 4

[Nathan Froyd [:froydnj]]

Brief update on this:

I spent some quality time debugging things yesterday and discovered that the entire JS engine was being compiled with MSVC while the rest of Gecko was being compiled with clang-cl.  That resulted in bug 1233542.  In an ideal world, this shouldn't have mattered; what appears to be happening is that template function instantiations are producing different code between MSVC and clang-cl (unsurprising), but substituting clang-cl's version for MSVC's version in libxul (the linker selects which version to use; it's worth noting that combining compilation units with different implementations of a particular template instantiation is undefined behavior, I believe) causes no end of problems (a little surprising; I'm not entirely certain whether this is a bug in clang-cl).  Fixing the template instantiations to only happen once would be...difficult, I think, and also bad for performance.

Once bug 1233542 is fixed, we're still falling back to MSVC for some JS engine files due to https://llvm.org/bugs/show_bug.cgi?id=25875  And that causes the same problems as before, due to some template function instantiations being different between MSVC and clang-cl.

We still fall back to MSVC on a (very) few Gecko files, but those are straightforward to fix.

Most of the dependent bugs I've been filing have to do with reducing warning spam; once all those patches are in the tree, we might be able to compile Firefox with clang-cl on try (since the logs will no longer overflow due to massive warning spam).

Comment 5

[Nathan Froyd [:froydnj]]

The simple startup crashes from comment 4 have been resolved once bug 1234860 winds its way onto mozilla-central.  We still crash on startup, though, with the assert added in bug 1177819.  We don't hit that assert in an MSVC-compiled Gecko, so I'm not quite sure what's going on yet.

Painting performance in a clang-cl-compiled Gecko also appears to be terrible; the UI doesn't paint properly, and the painting appears to be somewhat slow in any event.  One more thing to debug.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

(In reply to Nathan Froyd [:froydnj] from comment #5)
> Painting performance in a clang-cl-compiled Gecko also appears to be
> terrible; the UI doesn't paint properly, and the painting appears to be
> somewhat slow in any event.  One more thing to debug.

Can you screenshot the painting problem?

Comment 7

[Nathan Froyd [:froydnj]]

Attachment: screenshot of the painting problem at startup

(In reply to Jeff Muizelaar [:jrmuizel] from comment #6)
> (In reply to Nathan Froyd [:froydnj] from comment #5)
> > Painting performance in a clang-cl-compiled Gecko also appears to be
> > terrible; the UI doesn't paint properly, and the painting appears to be
> > somewhat slow in any event.  One more thing to debug.
> 
> Can you screenshot the painting problem?

Attached.  The browser chrome doesn't get painted, and the content is only painted if you mouseover it.  Once the mouse moves outside of the browser window, the content area is painted black again.

Ctrl-L and typing a URL does seem to work, but eventually hits the crashes about native anonymous content.

Comment 8

[(no longer active)]

(In reply to Nathan Froyd [:froydnj] from comment #5)
> The simple startup crashes from comment 4 have been resolved once bug
> 1234860 winds its way onto mozilla-central.  We still crash on startup,
> though, with the assert added in bug 1177819.  We don't hit that assert in
> an MSVC-compiled Gecko, so I'm not quite sure what's going on yet.

I think the easiest way to deal with this issue is to figure out a way to fix bug 1209680 and find the mis-compiled object file(s)...

Comment 9

[(no longer active)]

I have managed to successfully build with clang-cl x86-64 both debug and optimized.  Basic browsing on Windows Server 2012 seems to work just fine.

Comment 10

[Jeff Muizelaar [:jrmuizel]]

(In reply to :Ehsan Akhgari from comment #9)
> I have managed to successfully build with clang-cl x86-64 both debug and
> optimized.  Basic browsing on Windows Server 2012 seems to work just fine.

But I'm able to get crashes browsing to yahoo.com on Windows 7.

Comment 11

[(no longer active)]

Nathan, have you been seeing painting issue with a recent build?

Comment 12

[Nathan Froyd [:froydnj]]

(In reply to :Ehsan Akhgari from comment #11)
> Nathan, have you been seeing painting issue with a recent build?

I haven't tried in a couple of weeks.  I was doing 32-bit builds and I see that your recent successes have been with 64-bit.  Have you tried the 32-bit versions as well?

Comment 13

[Jeff Muizelaar [:jrmuizel]]

I built a 32-bit build and it seems to work fine. I'm running the mochitest browser-chrome now and it seems to be passing.

Comment 14

[(no longer active)]

Here is a try push with Windows x86 and x86-64 builds and tests in both debug and optimized: <https://treeherder.mozilla.org/#/jobs?repo=try&revision=f787abf6f20f>  The x86 debug build timed out, and the rest succeeded, but we are failing quite a few tests.

Comment 15

[Nathan Froyd [:froydnj]]

With only minimal changes and a patch or two still under review, I can make a Firefox with clang-cl build.  A --disable-optimize build seems OK, while an optimized build still shows the black painting problems in the screenshot attached to this bug.  (e.g. the newtab page is black everywhere there isn't a visible element)  Is there an obvious place to start looking for graphics bugs like that?

Comment 16

[Jeff Muizelaar [:jrmuizel]]

(In reply to Nathan Froyd [:froydnj] from comment #15)
> With only minimal changes and a patch or two still under review, I can make
> a Firefox with clang-cl build.  A --disable-optimize build seems OK, while
> an optimized build still shows the black painting problems in the screenshot
> attached to this bug.  (e.g. the newtab page is black everywhere there isn't
> a visible element)  Is there an obvious place to start looking for graphics
> bugs like that?

This is a build without any fallback to msvc cl right? I don't have any real suggestions of where to start debugging it. I can try to reproduce and debug the problem next week.

However, you could try to reduce the issue by trying to bisect the compilation.
i.e. have a wrapper around the compiler that chooses to add -O2 based on whether the file is in a list of files. With that infrastructure in place, you can reduce that list using http://delta.tigris.org/. Once we have the compilation unit narrowed down. We can probably decompose -O2 into individual passes and figure out exactly which pass is causing the change. That should give us an idea of what's going wrong.

Comment 17

[Ting-Yu Chou [:ting] (away)]

I tried to build m-c with clang-cl (r284701), and it failed with error "cannot use throw with exceptions disabled" in crashreporter/jsoncpp/src/lib_json/json_value.cpp. Should I fallback to the version prior r242176 [1], or are we following the latest clang?

[1] http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20150713/133143.html

Comment 18

[Nathan Froyd [:froydnj]]

(In reply to Ting-Yu Chou [:ting] from comment #17)
> I tried to build m-c with clang-cl (r284701), and it failed with error
> "cannot use throw with exceptions disabled" in
> crashreporter/jsoncpp/src/lib_json/json_value.cpp. Should I fallback to the
> version prior r242176 [1], or are we following the latest clang?
> 
> [1]
> http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20150713/133143.html

We are (mostly) following the latest clang, with the caveat that m-c is not buildable by default (bug 1298418).  So you have two options:

1. Use an earlier version of clang-cl for now; or
2. Figure out what we need to do to make things work with r242176.

Comment 19

[Nathan Froyd [:froydnj]]

(In reply to Ting-Yu Chou [:ting] from comment #17)
> I tried to build m-c with clang-cl (r284701), and it failed with error
> "cannot use throw with exceptions disabled" in
> crashreporter/jsoncpp/src/lib_json/json_value.cpp. Should I fallback to the
> version prior r242176 [1], or are we following the latest clang?
> 
> [1]
> http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20150713/133143.html

Actually, how does this break for you?  What is the command-line you're using?  We don't pass -f{no-}exceptions to clang-cl, so I'm at a bit of a loss to see how this commit affects the build.

Comment 20

[Ting-Yu Chou [:ting] (away)]

I just followed the instructions on MDN [1], and here [2] is where the "-fexceptions" comes in.

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Building_Firefox_on_Windows_with_clang-cl
[2] https://dxr.mozilla.org/mozilla-central/rev/f0f1aaf051d6798e1e73d1feee07ca847333167a/toolkit/crashreporter/jsoncpp/src/lib_json/moz.build#20

Comment 21

[Ting-Yu Chou [:ting] (away)]

Just to get some ideas about how far we're from clang-cl buildable. With clang r284701, m-c revision 215f96861176 [1] is buildable if bug 1312313, 1312309, 1311175, 1298418 are addressed.

[1] https://hg.mozilla.org/mozilla-central/rev/215f9681176

Comment 22

[Ting-Yu Chou [:ting] (away)]

Anyone knows do we have clang-cl build in automation for monitoring the buildable status?

Comment 23

[Nathan Froyd [:froydnj]]

(In reply to Ting-Yu Chou [:ting] from comment #22)
> Anyone knows do we have clang-cl build in automation for monitoring the
> buildable status?

I am working on that exact thing this quarter.

Comment 24

[(not currently active) Ted Mielczarek]

Considering that bug 1443590 is fixed, I'm going to have to conclude that this is fixed as well. :)