1074289 - |mach build file| does not always build

Status: NEW

(Firefox Build System :: Mach Core, enhancement)

Description

[Tomasz Kołodziejski [:tomasz]]

Steps to reproduce:

1. cd $MOZ/toolkit/content/widgets
2. modify autocomplete.xml
3. mach build .
(output:
 0:00.41 /usr/bin/make -C /Users/mozilla/gecko-dev/obj-ff-dbg -j8 -s backend.RecursiveMakeBackend
 0:00.54 /usr/bin/make -C toolkit/content -j8 -s widgets
 0:00.57 Your build was successful!)

Expected:
Firefox got updated and changes are visible.

Result:
Changes didn't get picked up.

To build it correclty I have to |mach build ..|. It outputs:
 0:00.41 /usr/bin/make -C /Users/mozilla/gecko-dev/obj-ff-dbg -j8 -s backend.RecursiveMakeBackend
 0:00.54 /usr/bin/make -C toolkit/content -j8 -s
 0:00.83 /usr/bin/make -C toolkit/library -j8 -s
 0:00.96 /usr/bin/make -C browser/app -j8 -s
 0:01.45 Your build was successful!

As far as I know it is a very much known issue.

Comment 1

[Tomasz Kołodziejski [:tomasz]]

I'll work on it :-)

Comment 2

[Gregory Szorc [:gps]]

Doing a partial tree build doesn't work for all file types. It is a known issue.

Comment 3

[Tomasz Kołodziejski [:tomasz]]

My understanding of the expected behavior of mach was like this:
1. run: mach build $MOZ/toolkit/content/widgets
2. mach goes up the obj tree to find directory with Makefile
3. when it finds one it assumes that we want to build that dir
4. run dumbmake on that dir to get additional dependencies

What I don't understand is (http://hg.mozilla.org/mozilla-central/file/5e704397529b/python/mozbuild/mozbuild/util.py#l188):

    A regular (non-Makefile) file resolves to the nearest directory at
    or above the file containing a Makefile, and an appropriate
    target.

What does it mean appropriate target?

In this picture I'm missing make_target parts in dumbmake.py. Especially, in my usual case what happens is that make_target is non-empty (it's equal to content or even content/autocomplete.xml) and so add_extra_dependencies in http://hg.mozilla.org/mozilla-central/file/5e704397529b/python/mozbuild/dumbmake/dumbmake.py#l108 skips the adding extra dependencies step. When I don't skip it it starts to work as I want. But I don't really understand what was the reason for skipping it in the first place. If you can explain that a little bit gps I'd be happy.

Please keep in mind that I mainly work inside $MOZ/{toolkit,browser} directories (I mostly just "build" js).

Comment 4

[Gregory Szorc [:gps]]

The "appropriate target" bit probably refers to how you can magically append a make target to the paths accepted by |mach build|. e.g. |mach build toolkit/export| is equivalent to |make -C toolkit export|.

The extra "continue" there may be a legitimate bug. But I'd have to page all of dumbmake into my head.

With all the work glandium has been doing to optimize tier traversal, the days of dumbmake become fewer and fewer. I'm tempted to provide a make target for browser developers that does all the JS/CSS/HTML optimally, in much the same way Gecko developers can now build libxul optimally.

Comment 5

[Tomasz Kołodziejski [:tomasz]]

Try:

mach build browser/base/content/newtab

It reports the target_pairs to be [(u'browser/base', u'content/newtab')]. For it to work we just need to build browser/base and its dumbmake dependecies. It certainly does not work like that right now. I don't know make very well (but I'd be very happy to learn about that as an intern if you had some time one day gps!) so I'm not sure if target content/newtab will ever make sense.

So it looks like resolve_target_to_make finds the right make_dir but I don't know how the make_target is useful at all. Maybe I'm missing the c++ context to understand that.

I'd love to discuss that so I'm marking needinfo?ing you gps.

Comment 6

[Gregory Szorc [:gps]]

browser/base/content/newtab resolves to [('browser/base', 'content/newtab')] because browser/base is the first parent directory to have a moz.build/Makefile.in file.

This behavior is wrong in this case but it is not always wrong.

I don't think we can achieve perfection here until all build targets are known up front. That's a lot of work. We're not getting there any time soon.

I think a good compromise would be to have the build system print the expanded make targets before execution. e.g.

  $ mach build browser/base/content/newtab
  no Makefile found in requested path; building content/newtab target of browser/base

Comment 7

[Tomasz Kołodziejski [:tomasz]]

I'll un-assign myself as it looks like I don't understand the build system enough to be able to fix this in the general case.

However, for all my use at Mozilla (fx-team) I'm using my script that simply walks through the obj dir to find the closes ancestor dir with Makefile: https://github.com/neojski/mach-watch. It works for all the js/jsm/css files I have ever edited. I also automatically build my js/jsm/css when file is changed and it works very well.

Comment 8

[Tomasz Kołodziejski [:tomasz]]

It looks like I should have set [e-mail] (it has the same description as [e-mail] which is confusing).