1657913 - Windows build bustage with MSVC 16.7.0 update and later.

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Attachment: build log

Today I installed the newly-released MSVC 2019 16.7.0 update and am hitting a lot of build bustage.

The primary error appears to be:

C:\PROGRA2\MICROS1\2019\COMMUN1\VC\Tools\MSVC\14271.291\include\yvals_core.h(494,2): error: STL1000: Unexpected compiler version, expected Clang 10.0.0 or newer.

There's a lot of other issues as well (see the attached log), but I'm not sure how many of them are separate issues vs. fallout from the initial bustage. We should probably at least direct people away from installing this update for the time-being.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

I see some other possibly-different errors in utility as well:

C:\PROGRA2\MICROS1\2019\COMMUN1\VC\Tools\MSVC\14271.291\include\utility(212,5): note: copy constructor is implicitly deleted because 'pair<unsigned char, unsigned char>' has a user-declared move constructor

Comment 2

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

I tried downloading LLVM 10.0 to replace the mozbuild clang but somehow the Windows precompiled binary distribution doesn't include some required tools e.g. llvm-dlltool. I'm building the LLVM myself to get it.

Edit: The only missing part was llvm-dlltool, everything went well after adding it.

Comment 4

[mac198442]

Good news is if you install vs2017 along side vs2019 you can include ac_add_options --with-visual-studio-version=2017 in your mozconfig and get builds back working.

Comment 5

[Honza Bambas (:mayhemer)]

(In reply to mac198442 from comment #4)

Good news is if you install vs2017 along side vs2019 you can include ac_add_options --with-visual-studio-version=2017 in your mozconfig and get builds back working.

Doesn't work for me...

Comment 6

[(Away)]

Here is an old clang-10 build from automation. We had to back it out for perf regressions but if you only care about building then maybe it can get you unblocked: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VzRmR7vqQmynv_6T14oguA/runs/0/artifacts/public/build/clang.tar.bz2

Comment 7

[Jean-Yves Avenard [:jya]]

(In reply to Honza Bambas (:mayhemer) from comment #5)

(In reply to mac198442 from comment #4)

Good news is if you install vs2017 along side vs2019 you can include ac_add_options --with-visual-studio-version=2017 in your mozconfig and get builds back working.

Doesn't work for me...

Didn't work for me either. I had to uninstall VS2019 for now.

Comment 8

[Mike Hommey [:glandium]]

IIRC, the msvc installer allows to install a bunch of older versions.

Comment 9

[Jean-Yves Avenard [:jya]]

(In reply to Mike Hommey [:glandium] from comment #8)

IIRC, the msvc installer allows to install a bunch of older versions.

Older being 2017, not dot release.

Apparently, you can go back to an earlier dot release, but only with enterprise and pro version.

Comment 10

[Mike Hommey [:glandium]]

In my community installer, I can see choices for:

• MSVC v140 - VS 2015 C++ x64/x86 build tool (v14.00)
• MSVC v141 - VS 2017 C++ x64/x86 build tool (v14.16)
• MSVC v142 - VS 2019 C++ x64/x86 build tool (v14.21)
• MSVC v142 - VS 2019 C++ x64/x86 build tool (v14.22)
• MSVC v142 - VS 2019 C++ x64/x86 build tool (v14.23)
• MSVC v142 - VS 2019 C++ x64/x86 build tool (v14.24)
• MSVC v142 - VS 2019 C++ x64/x86 build tool (v14.25)

And similar range of versions of SDKs (ATL and MFC).

Comment 11

[Frank-Rainer Grahl (:frg)]

And similar range of versions of SDKs (ATL and MFC).

Does not work. Tried it. Uninstalled the latest toolset version and installed a previous one. configure will not find the c compiler because the directory structure is a bit different for the vswhere output and file locations Just did go back to 16.6.5.

Too late now if you don't have a backup but I always check out an offline repo with vs_community.exe --layout D:\Soft\Microsoft\VS2019\VS2019-16.7 --lang en-US
Replace the directory with you favorite place. Also always kill the online update task after installation.

Comment 12

[Jorg K (CEST = GMT+2)]

Has anyone tried 16.7.1 which I got offered today?

Comment 13

[Ryan VanderMeulen [:RyanVM]]

(In reply to Jorg K (CEST = GMT+2) from comment #12)

Has anyone tried 16.7.1 which I got offered today?

No change.

Comment 14

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

(In reply to Jorg K (CEST = GMT+2) from comment #12)

Has anyone tried 16.7.1 which I got offered today?

It uses the same MSVC 14.27.29110.

Comment 15

[Chamal De Silva]

Clang installed inside .mozbuild directory is version 9.0.1

c:/Users/<Username>/.mozbuild/clang/bin/clang-cl.exe --version
clang version 9.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: c:\Users\<Username>.mozbuild\clang\bin

Is it necessary to update this clang version?

Comment 16

[Gabriel S.]

(In reply to Chamal De Silva from comment #15)

Clang installed inside .mozbuild directory is version 9.0.1

c:/Users/<Username>/.mozbuild/clang/bin/clang-cl.exe --version
clang version 9.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: c:\Users\<Username>.mozbuild\clang\bin

Is it necessary to update this clang version?

Yes, but version 10 was rolled back as per https://bugzilla.mozilla.org/show_bug.cgi?id=1616692 - I suspect it may take more effort to get Firefox compiling than just upgrading the binary.

Comment 17

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

(In reply to Gabriel S. from comment #16)

(In reply to Chamal De Silva from comment #15)

Clang installed inside .mozbuild directory is version 9.0.1

c:/Users/<Username>/.mozbuild/clang/bin/clang-cl.exe --version
clang version 9.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: c:\Users\<Username>.mozbuild\clang\bin

Is it necessary to update this clang version?

Yes, but version 10 was rolled back as per https://bugzilla.mozilla.org/show_bug.cgi?id=1616692 - I suspect it may take more effort to get Firefox compiling than just upgrading the binary.

The bug says it got some sanity failures after compiling, and the build itself has been okay on my machine.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

(In reply to :dmajor from comment #6)

Here is an old clang-10 build from automation. We had to back it out for perf regressions but if you only care about building then maybe it can get you unblocked: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VzRmR7vqQmynv_6T14oguA/runs/0/artifacts/public/build/clang.tar.bz2

Confirmed that this works for me.

EDIT: Other than the resulting build crashing on startup, that is :\

Comment 19

[(Away)]

Here's a build of clang 11.0.0 rc2 that we're planning to switch over to in bug 1660340: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/aZ1BC14qS3W_WxHtTw4s1A/runs/0/artifacts/public/build/clang.tar.zst If your machine doesn't know what to do with .zst files, taskcluster/scripts/misc/zstdpy is an option.

Comment 20

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

(In reply to :dmajor from comment #19)

Here's a build of clang 11.0.0 rc2 that we're planning to switch over to in bug 1660340: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/aZ1BC14qS3W_WxHtTw4s1A/runs/0/artifacts/public/build/clang.tar.zst If your machine doesn't know what to do with .zst files, taskcluster/scripts/misc/zstdpy is an option.

Works perfectly 😀

Comment 22

[Masatoshi Kimura [:emk]]

Is this still an issue with the latest clang?

Comment 23

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

It has been okay with clang 11.

Comment 24

[Masatoshi Kimura [:emk]]

Then, why is this bug still open?

Comment 25

[Frank-Rainer Grahl (:frg)]

Will break again with 16.8 but I have not tested with the latest prerelease level. Would be nice to have an option to use a specific toolchain from VS2019. See comment 10.

Comment 26

[Masatoshi Kimura [:emk]]

Why do you know that it will be broken with 16.8 without testing it? Even if it is, a new bug should be filed for 16.8 instead of muddying an existing bug.

Comment 27

[Frank-Rainer Grahl (:frg)]

Why do you know that it will be broken with 16.8 without testing it?
I did test 16.8 preview 2 and it was broken because of new language features support in the headers which clang 11 didn't like. I just did not test the latest preview as stated.

Comment 28

[Kagami Rosylight [:saschanaz] (they/them) (OOO until 2024-10-06)]

Can we open a new bug rather than covering newer updates here? That way the discussion can be specific to the failing update. (I don't think the new update will require clang 12 as it doesn't exist.)

Comment 29

[(Away)]

Yes please.