1380405 - Build should not create files in the source tree

Status: NEW

(Firefox Build System :: General, defect)

Description

[Jim Blandy :jimb]

The mach build process should leave the source tree untouched; all data should live in the object directory. Putting generated files in the source tree causes various problems:

- Deleting one's object directory does not give one the clean slate that it ought (stale .pyc files are a pain)

- The files end up accidentally included in tarballs or other packages.

- Deciding whether two builds can live side by side requires more judgement than it should.

Comment 1

[Jim Blandy :jimb]

Right now the offenders are:

- .pyc files. I don't know much about Python, but surely we can get it to put these elsewhere?

- configure scripts. The design of autoconf requires that these live in the source tree. I'm not sure we should fix this, but that's open to debate.

- mozbuild transitional old-configure scripts. I assume(?) mozbuild could put these wherever it wants?

- testing/mozbase/mozlog/mozlog.egg-info

- third_party/python/psutil, a bunch of .o and .so files that totally don't belong there; bug 841678 is about tripping over them

Looking through my tree, I have things like Emacs backup files and other developer tool detritus. These probably belong in my personal .hgignore file.

And of course, I have build subdirectories. Putting those in the source tree is pretty widespread and handy.

Comment 2

[Michael Shal [:mshal]]

(In reply to Jim Blandy :jimb from comment #0)
> - Deleting one's object directory does not give one the clean slate that it
> ought (stale .pyc files are a pain)

It might be worth investigating whether using 'python -B' / PYTHONDONTWRITEBYTECODE=1 causes a noticeable performance hit for us. If not, then we could just avoid writing *.pyc altogether.

Comment 3

[Gregory Szorc [:gps]]

Bug 685957 (and 795995) have the history regarding .pyc files. It is a non-trivial problem. The build system invokes enough Python and Python startup overhead is enough of a problem that we need the benefits of .pyc files.

Comment 4

[Ryan VanderMeulen [:RyanVM]]

node_modules is also created in the source dir these days if linting is used

Comment 5

[Jim Blandy :jimb]

I think enumerating know problems here is okay, but let's be sure to create blockers once we want to have much discussion of a particular instance. For example, let's make sure all discussion of .pyc files go in 685957. (And if the idea's already covered there, no need to repeat it here.)