1534615 - Run gn directly as part of the build system

Status: RESOLVED DUPLICATE of bug 1654407

(Firefox Build System :: General, enhancement)

Description

[Dan Minor [:dminor]]

It would be nice to run gn directly as part of the build system rather than relying upon checking in json files and code generating moz.build files.

The current process is problematic while doing updates. For instance, in Bug 1496359 we've been asked to cherry pick an upstream webrtc.org commit. That commit moves files, which means I have to edit generated files to workaround moz.build validation errors just to be able to run the gn generation steps.

It also triggers bugs in the moz.build code generation, I think because the paths in the checked in json files for non-Tier 1 platforms do not match paths that exist after the patch is applied.

When doing a full webrtc.org update, I workaround these problems be removing all json files except for the current platform I'm trying to get to build, but that is not ideal. It's also not great to have to ask everyone who supports a non-Tier 1 platform to have to regenerate json files if we cherry pick a patch that affects gn files.

The generated json files are also quite large and more or less impossible to review (see also Bug 1526287).

My recollection is that we originally decided to check in json files rather than run gn itself was that it was not possible to build gn outside of Chromium and we didn't want the Firefox build system to depend on Chromium. It looks like gn has now moved to it's own repo (https://gn.googlesource.com/gn/) so it seems like we could revisit this decision.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

The build system part of this should be relatively straightforward and I'd very much like to find time to work on that, I think the remaining question is what hoops upstreams may yet need to go through to build and package their own gn.

Comment 2

[Jan Beich]

Did you mean downstreams? If so see https://repology.org/project/gn/versions

Comment 3

[Mike Hommey [:glandium]]

So, one problem here is that gn changes and in-tree BUILD.gn files don't. Which is kind of a similar problem with compilers, but compilers tend to be more behaved wrt compatibility. And as it turns out, I tried gn master over the week-end, and it fails with what we currently have in the tree, while the older version we build on automation (listed in taskcluster/scripts/misc/build-gn-common.sh) works.

Comment 4

[Dan Minor [:dminor]]

I think right now only webrtc.org is using gn, but if we have more third party libraries that use it, we might also end up with the problem where we're stuck updating them all at the same time to keep them on mutually compatible versions of gn. I suppose we could keep several versions of gn in automation, and tag parts of the tree with which version of gn to run. That sounds kind of horrifying.