979629 - Editing included .ini files doesn't cause install targets to be regenerated

Status: REOPENED

(Firefox Build System :: General, defect)

Description

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

STR:
1. Do a normal build.
2. Modify a mochitest.ini file and add a skip-if directive to one of the tests.
3. Run |mach build|.
4. Observe that the test is still in your objdir.

If I run |make install-tests|, the test is correctly removed. Here's my mozconfig:

. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-dbg
mk_add_options MOZ_MAKE_FLAGS="-j12"
ac_add_options --enable-debug
ac_add_options --disable-optimize
ac_add_options --enable-valgrind
ac_add_options --enable-gczeal
ac_add_options --with-ccache

Comment 1

[(not currently active) Ted Mielczarek]

A top-level build does run this. Are you doing a non-top-level build?

Comment 2

[Gregory Szorc [:gps]]

Are you sure your STR are accurate? Please paste(bin) output to confirm.

What I suspect is happening is interaction with the NO_REMOVE environment variable.

Because not every file in _tests is currently defined in manifests and thus available to the build config, we don't remove unaccounted files from _tests during non-full builds. If we removed unaccounted files during partial builds (or during test execution as part of the mach commands), those files would not be replaced by the build, likely leading to failures.

If you perform a full tree build, unreferenced files from _tests should be deleted. I know this works because my full-tree build prints:

 From _tests: Kept 15802 existing; Added/updated 0; Removed 985 files and 128 directories.

If you perform a partial tree build (or run a mach testing command), we see no deletions occur:

 $ mach xpcshell-test toolkit/modules/
 From _tests: Kept 15802 existing; Added/updated 0; Removed 0 files and 0 directories.

We can do the right thing and prune unaccounted files from _tests on *every* action once the build config knows about every file in _tests. Ted has made great strides towards this. But as you can see, we still have nearly 1000 unaccounted files lingering.

Comment 3

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

I can't reproduce this anymore.

Comment 4

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

This is happening to me again. Ted, where is the code that runs |make install-tests| for a toplevel build? I can try and see why it's not running.

Comment 5

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Attachment: out

This is the output of |mach build| from the objdir after adding a new test to a manifest (toolkit/mozapps/extensions/test/xpcshell/xpcshell-shared.ini). The test is not getting symlinked into the objdir. When I run |make install-tests|, the symlink appears.

Comment 6

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

I sorta figured this out. The title of the bug is wrong, which is what caused the confusion.

Let's say I change toolkit/mozapps/extensions/test/xpcshell/xpcshell-shared.ini. This file is included by toolkit/mozapps/extensions/test/xpcshell/xpcshell.ini. Then I run |mach build|. For some reason, the dependencies are such that we don't do the "Reticulating splines..." step. Consequently, _build_manifests/install/tests is not updated. So we actually do run |make install-tests|, but it doesn't do the right thing. If I run |mach build-backend| and then |mach build| (or just |make install-tests|) then everything works fine.

I'm not sure where we make config.status (I think this is the important target) depend on the .ini files, but we're apparently missing included .ini files.

Comment 7

[(not currently active) Ted Mielczarek]

Duplicate of bug 994255?