Closed Bug 1318143 (vs2017) Opened 8 years ago Closed 7 years ago

Officially support building with Visual Studio 2017

Categories

(Firefox Build System :: General, defect)

Unspecified
Windows
defect
Not set
normal

Tracking

(firefox53 wontfix, firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox53 --- wontfix
firefox55 --- fixed

People

(Reporter: RyanVM, Assigned: ted)

References

(Blocks 1 open bug)

Details

(Keywords: meta)

Attachments

(4 files)

The release candidate build of Visual Studio 2017 was announced today. This bug tracks making it an officially supported compiler.
https://blogs.msdn.microsoft.com/visualstudio/2016/11/16/visual-studio-2017-rc/
For starters, the directory structure of Visual Studio 2017 has completely changed. The things we care about live in:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24629\bin

Within that, there are different subdirectories for different cl.exe compiler (and cross-compiler) options.
HostX64\x64
HostX64\x86
HostX86\x64
HostX86\x86

The HKLM\SOFTWARE\Microsoft\VisualStudio\15.0\Setup\VC key (which is what we've used for detecting past versions, at least with Mozillabuild) doesn't exist. Looks like the only reference to that path in the registry is the "15.0" key at HKLM\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VS7.

So this is going to take a little bit of refactoring work to detect the installation and to assemble the relevant toolchain paths afterwards.
We were looking at the HKLM\SOFTWARE\{WOW6432Node\,}Microsoft\VisualStudio\VC\*\*\*\Compiler keys, not HKLM\SOFTWARE\Microsoft\VisualStudio\15.0\Setup\VC
s/We were/We are/
Depends on: 1318182
Depends on: 1310067
Turning this into a meta bug.
Alias: vs2017
Keywords: meta
Depends on: 1318189
Depends on: 1318193
compiling uriloader_prefetch0.cpp causes ICE (C1001: f:\dd\vctools\compiler\cxxfe\sl\p1\c\outdname.c) by nsThreadUtils.h  I will report it to MS via connect.  This doesn't occurs on VS"15" preview 4.  It is from RC.
(In reply to Makoto Kato [:m_kato] from comment #5)
> compiling uriloader_prefetch0.cpp causes ICE (C1001:
> f:\dd\vctools\compiler\cxxfe\sl\p1\c\outdname.c) by nsThreadUtils.h  I will
> report it to MS via connect.  This doesn't occurs on VS"15" preview 4.  It
> is from RC.

This will be fixed in next release. (http://connect.microsoft.com/VisualStudio/feedback/details/3111942/ice-in-f-dd-vctools-compiler-cxxfe-sl-p1-c-outdname-c-line-4872)
Depends on: 1319971
I asked Microsoft the proper way to locate VS2017 at https://developercommunity.visualstudio.com/content/problem/2813/cant-find-registry-entries-for-visual-studio-2017.html. I suspect the answer will make glandium's head explode.
Depends on: 1345331
As of today's mozilla-central, the tree should be buildable with VS2017. We don't have a good start-shell script for it though; you have to hack together your own.
We shouldn't require a start-shell script for it. Configure can find older versions of VS from a start-shell.bat shell, it should be enhanced to find VS2017.
(In reply to David Major [:dmajor] from comment #9)
> As of today's mozilla-central, the tree should be buildable with VS2017. We
> don't have a good start-shell script for it though; you have to hack
> together your own.

Maybe, get_vc_paths and vc_compiler_path won't work when VS2015 isn't installed.  We need VS2017's install path detection for moz.configure.  (Although Microsoft post a blog to detect install path, all way uses COM...)
FWIW, get_vs_paths and vc_compiler_path are essentially bypassed if you pass the right variables in the environment or via mozconfig.
(In reply to Mike Hommey [:glandium] from comment #12)
> FWIW, get_vs_paths and vc_compiler_path are essentially bypassed if you pass
> the right variables in the environment or via mozconfig.

compiler path such as CC and CXX is no problem, but LINK still sets VS2015 path even if I calling C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat.  (LINK is absolute path of link.exe)
(In reply to Makoto Kato [:m_kato] (PTO until 3/20) from comment #11)
> Maybe, get_vc_paths and vc_compiler_path won't work when VS2015 isn't
> installed.  We need VS2017's install path detection for moz.configure. 
> (Although Microsoft post a blog to detect install path, all way uses COM...)

We should figure out how to do this. It would be nice to not have to use COM, but whatever works.
I'm a noob with my first major patch that I need to test on Windows.  I just now installed VS2017.  Are there build instructions to follow from there? 8)
Found an installer finally, sorry for comment 16.
(In reply to Xavier Fung from comment #8)
> Microsoft has posted some ways to find out the VS instances:
> https://blogs.msdn.microsoft.com/vcblog/2017/03/06/finding-the-visual-c-
> compiler-tools-in-visual-studio-2017/

Per this, the registry method we're currently using in configure no longer works. There are some COM APIs you can use, but Microsoft also provides a redistributable `vswhere.exe` that can output JSON:
https://github.com/Microsoft/vswhere

I think maybe the path of least resistance here would be to just check a copy of that binary into the tree and call it from configure. That would also make it possible to locate MSVC in WSL (bug 1311729), which we wouldn't be able to do from Python (since Python in WSL is a Linux binary).
I looked at the number of source files in that vswhere project and... ugh.

I'm normally opposed to checking binaries into the tree. But for this, it might be justified. If we go that path, we'll want to also vendor the source because we don't like checking in binaries without corresponding source.

I built a release build of vswhere.exe and it weighs in at 376kb. That feels a bit large. The VS Solution is already optimizing the binaries for size. Not sure where the bloat is. I really wish the binary was a few hundred kb smaller.
The bark is worse than the bite in terms of source code. Most of the source is for things we don't need, like XML formatting. I wonder if we could extract the useful bits and produce a smaller binary?

Also, if Rust can speak COM, that provides a nifty way to not have to check in a binary :)
(In reply to Gregory Szorc [:gps] from comment #20)
> Also, if Rust can speak COM, that provides a nifty way to not have to check
> in a binary :)

Rust can do this with some plumbing.  Python could too for comment 14?
Yeah, Python can do it. The comtypes package seems reasonable and doable.

WSL might throw us for a loop, since presumably you need a win32 binary to use COM. I suppose we could download a self-contained Python for Windows distribution under WSL so we can invoke win32 Python processes. We may need that for other things in WSL. e.g. we make use of win32 APIs for performance reasons in a few places since the POSIX APIs that Python 2.7 uses are slow on Windows and presumably also slower-than-win32 under WSL. So it isn't too much of a stretch to imagine our WSL cross-compiling build invoking a mix of native/ELF and win32 Python.
Having mixed Pythons in a WSL build env sounds sorta horrible to me, TBH. I'd much rather just have a checked-in vswhere.exe to solve this particular problem.
I have a patch using vswhere.exe that passes configure using VS2017, will get it up for your perusal once I ensure that it builds everything OK.
Assignee: nobody → ted
Depends on: 1358169
Well, no such luck, I hit bug 1358169 in the interim.
OK, with those patches I'm able to successfully run configure from a start-shell.bat shell for both VC 2015 and 2017. I added a second patch that adds --with-visual-studio-version so you can choose to build with 2015 even if you have 2017 installed without having to have the environment configured in your shell.
Comment on attachment 8860089 [details]
bug 1318143 - add support for detecting Visual Studio 2017 to configure.

https://reviewboard.mozilla.org/r/132108/#review135080

I still have a somewhat negative reaction to checking vswhere.exe into the tree. I'd prefer we download it via tooltool at run-time until we know it is the solution going forward. But at only 200kb compressed, it wouldn't be the biggest mistake and waste of space in the history of the repo.

::: build/moz.configure/windows.configure:260
(Diff revision 2)
> +        if c_compiler.version < '19.10':
> -        path = os.path.join(os.path.dirname(vc_path), 'DIA SDK')
> +            path = os.path.join(os.path.dirname(vc_path), 'DIA SDK')
> +        else:
> +            # This would be easier if we had the installationPath that
> +            # get_vc_paths works with, since 'DIA SDK' is relative to that.
> +             path = os.path.normpath(os.path.join(vc_path, r'..\..\..\..\..\..\..\DIA SDK'))

How do you feel about adding the DIA SDK path to `get_vc_paths()`? That feels slightly better than doing what is effectively post-processing here. I'm not sure the implications for the various dependencies in this file though. So feel free to drop.
Attachment #8860089 - Flags: review?(gps) → review+
Comment on attachment 8860104 [details]
bug 1318143 - add a --with-visual-studio-version to choose which VS version configure chooses when multiple versions are installed.

https://reviewboard.mozilla.org/r/132114/#review135082

Since this will choose VS2017 by default, please send out an announcement to dev-platform when this lands.

Also, I'm not convinced that making the VS2017 the default is best. I'd rather wait until automation is upgraded so there is less divergence between local and official environments. I'll defer to your judgement here.
Attachment #8860104 - Flags: review?(gps) → review+
It's only really an issue if someone has both VS2015 and VS2017 installed, and I have provided an escape hatch in the form of --with-visual-studio-version. I will post about that on dev-builds before landing this. This should still be better than the current situation which is "I spent a bunch of time installing VS2017 and I can't get a build working at all".

I'm going to try getting VS2017 builds stood up in Taskcluster ASAP.
Attached image Clipboard.png
With the patch I am able to build a x64 comm-central suite. I am running into an error when I try to do an x86 build. Additional paths might be needed to be set here for the x86 compiler. Didn't try to build an x86 Firefox yet but probably be the same. This is a vm with VS2017 only. I completly removed VS2015 from it. 

I had to remove "ac_add_options --enable-warnings-as-errors" from the mozconfig. VS2017 seems to put out additional warnings.
From a comment in https://reviews.llvm.org/D30991# it seems the x86 cross compiler needs parts in the x64 native compiler. 

+++ snip +++

The new 2017 toolchains are split up by host architecture and target architecture:
HostX64/x64. Native compilation. Compile for x64 on an x64 host.
HostX64/x86. Cross-compilation. Compile for x86 on an x64 host.
HostX86/x64. Cross-compilation. Compile for x64 on an x86 host
HostX86/x86. Native compilation. Compile for x86 on an x86 host.

The cross compiling toolchains (HostX64/x86, HostX86/x64) are incomplete as they rely on components containing in the corresponding native toolchains (e.g. HostX64/x86 uses components from HostX64/x64).
You can see this in the vcvars bat scripts supplied with Visual Studio. They setup %PATH% in the same way when cross-compiling.
mspdbcore.dll is an example of something missing from the cross-compiling toolchains. There are probably others but I didn't check.
(In reply to Frank-Rainer Grahl from comment #34)
> From a comment in https://reviews.llvm.org/D30991# it seems the x86 cross
> compiler needs parts in the x64 native compiler. 

Thanks for the info! I updated my patch and was able to build a 32-bit Firefox locally.
gps: If you could give the first patch a quick look, I had to make a change to accommodate the 64->32 cross-toolchain. I simply changed `get_vc_paths` to provide a list of paths in the dicts it yields, and vs_compiler_path passes on the list to get appened to `PATH`.
Flags: needinfo?(gps)
Comment on attachment 8860089 [details]
bug 1318143 - add support for detecting Visual Studio 2017 to configure.

https://reviewboard.mozilla.org/r/132108/#review136366
Flags: needinfo?(gps)
Comment on attachment 8860089 [details]
bug 1318143 - add support for detecting Visual Studio 2017 to configure.

https://reviewboard.mozilla.org/r/132108/#review135080

> How do you feel about adding the DIA SDK path to `get_vc_paths()`? That feels slightly better than doing what is effectively post-processing here. I'm not sure the implications for the various dependencies in this file though. So feel free to drop.

I'll file a followup to refactor this a bit. It's a little convoluted at the moment because `get_vc_paths` is simply used to set `PATH` to find the compiler, and then the stuff in `windows.configure` works backwards from that. :-/
Pushed by tmielczarek@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6eb93543bf16
add support for detecting Visual Studio 2017 to configure. r=gps
https://hg.mozilla.org/integration/autoland/rev/fc094009b7c1
add a --with-visual-studio-version to choose which VS version configure chooses when multiple versions are installed. r=gps
Backed out for Windows build bustage:

https://hg.mozilla.org/integration/autoland/rev/4baf1f4e5df5b6989fdf1e14d3020ee9978a636d
https://hg.mozilla.org/integration/autoland/rev/78fc62215d7cfc6eed98b395d4c0dbc65f161dbf

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=fc094009b7c163b6c923170190e1c4c236163463&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=94184726&repo=autoland

20:36:13     INFO -  checking for a shell... C:/mozilla-build/msys/bin/sh.exe
20:36:13     INFO -  checking for host system type... i686-pc-mingw32
20:36:13     INFO -  checking for target system type... i686-pc-mingw32
20:36:13     INFO -  checking for a shell... C:/mozilla-build/msys/bin/sh.exe
20:36:14     INFO -  checking for host system type... i686-pc-mingw32
20:36:14     INFO -  checking for target system type... i686-pc-mingw32
20:36:14     INFO -  checking whether cross compiling... no
20:36:14     INFO -  Traceback (most recent call last):
20:36:14     INFO -    File "z:/build/build/src/configure.py", line 124, in <module>
20:36:14     INFO -      sys.exit(main(sys.argv))
20:36:14     INFO -    File "z:/build/build/src/configure.py", line 29, in main
20:36:14     INFO -      sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 362, in run
20:36:14     INFO -      self._value_for(option)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 430, in _value_for
20:36:14     INFO -      return self._value_for_option(obj)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 447, in _value_for_option
20:36:14     INFO -      not self._value_for(implied_option.when)):
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 153, in result
20:36:14     INFO -      return self._func(resolved_args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 152, in <genexpr>
20:36:14     INFO -      for d in self.dependencies)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 106, in result
20:36:14     INFO -      for d in self.dependencies]
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 106, in result
20:36:14     INFO -      for d in self.dependencies]
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 106, in result
20:36:14     INFO -      for d in self.dependencies]
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 106, in result
20:36:14     INFO -      for d in self.dependencies]
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 427, in _value_for
20:36:14     INFO -      return self._value_for_depends(obj, need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 436, in _value_for_depends
20:36:14     INFO -      return obj.result(need_help_dependency)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\util.py", line 925, in method_call
20:36:14     INFO -      cache[args] = self.func(instance, *args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 107, in result
20:36:14     INFO -      return self._func(*resolved_args)
20:36:14     INFO -    File "z:\build\build\src\python\mozbuild\mozbuild\configure\__init__.py", line 955, in wrapped
20:36:14     INFO -      return new_func(*args, **kwargs)
20:36:14     INFO -    File "z:/build/build/src/build/moz.configure/toolchain.configure", line 472, in vc_compiler_path
20:36:14     INFO -      data = all_versions[-1][1]
20:36:14     INFO -  IndexError: list index out of range
20:36:14     INFO -  *** Fix above errors and then restart with\
20:36:14     INFO -                 "z:/build/build/src/mozmake.EXE -f client.mk build"
20:36:14     INFO -  client.mk:379: recipe for target 'configure' failed
20:36:14     INFO -  mozmake.EXE: *** [configure] Error 1
Flags: needinfo?(ted)
Blocks: 1359598
There were two dumb things broken in those patches, I don't know why I didn't push to try:
1) vc_compiler_path didn't cope with the situation where vswhere.exe returned an empty list, which happens on the build machines because they don't actually have Visual C++ installed system-wide.
2) test_toolchain_configure.py was broken because it mocks process execution but it wasn't set up to mock vswhere.exe. I changed it to just return an empty list for all invocations currently, since we're not testing this code yet anyway.
Flags: needinfo?(ted)
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #43)
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=b9c983a3a44274891c341335549f461b445d3ce4

This is still failing on the buildbot Windows builds, it looks like vswhere is finding the system VS2013 install and we wind up using link.exe from there. I'll see what I can do to fix that.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #46)
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=60afcf53d61b8314cbd9e621ead8efd282e28a39

OK, I don't have a machine with VS2013 handy to test with, and I didn't feel like spending the time installing it in a VM just to test this, so I put in a workaround where we'll just skip any legacy versions < MSVC2015, and the buildbot Windows builds are green now. The changes I made are pretty trivial, so I'm just going to re-land.
https://hg.mozilla.org/integration/mozilla-inbound/rev/311336b5377de438e6e71e4ad4eca6b18da2b37c
bug 1318143 - add support for detecting Visual Studio 2017 to configure. r=gps

https://hg.mozilla.org/integration/mozilla-inbound/rev/93d41c6f2a14d0c557828fcdd505c47e71fa9fbb
bug 1318143 - add a --with-visual-studio-version to choose which VS version configure chooses when multiple versions are installed. r=gps
https://hg.mozilla.org/mozilla-central/rev/311336b5377d
https://hg.mozilla.org/mozilla-central/rev/93d41c6f2a14
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Attached image ClipboardVS2015.png
Good News, Everyone!

Looks like the VS2015 x86 cross compiler has the same problem as VS2017 and also needs the path for the x64 compiler appended. You still need to use start-shell-msvc2015.bat for it. Using only start-shell.bat doesn't work here.
(In reply to Frank-Rainer Grahl from comment #50)
> Looks like the VS2015 x86 cross compiler has the same problem as VS2017 and
> also needs the path for the x64 compiler appended. You still need to use
> start-shell-msvc2015.bat for it. Using only start-shell.bat doesn't work
> here.

Please file a new bug. Have it block or depend on this one.

(In other news, we either need a new meta bug or we should reopen this one. I'll defer to Ted.)
Flags: needinfo?(ted)
Depends on: 1360631
Depends on: 1361750
(In reply to Gregory Szorc [:gps] from comment #51)
> (In other news, we either need a new meta bug or we should reopen this one.
> I'll defer to Ted.)

Yeah, I should have put these patches on a different bug, but oh well! We should probably just open a new meta bug.
Flags: needinfo?(ted)
Depends on: 1362872
Depends on: 1362923
Depends on: 1371179
Blocks: 1371439
Depends on: 1371465
Depends on: 1371485
See Also: → 1376051
Depends on: 1395907
No longer depends on: 1371179
Depends on: 1408695
Blocks: 1408789
Blocks: 1414297
Blocks: 1414299
Blocks: 1414304
Blocks: 1414306
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: