1407403 - Handle IPDL generation in the tup backend

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Michael Shal [:mshal]]

We need to implement the _handle_ipdl_sources function in the tup backend, and do something similar to ipc/ipdl/Makefile.in. I think this is made slightly tricky for tup since the outputs aren't known ahead of time.

Comment 1

[Michael Shal [:mshal]]

Attachment: Bug 1407403 - Handle IPDL code generation in the tup backend;

IPDL generation consists of a single ipdl.py invocation on all the
*.ipdl and *.ipdlh files in the tree. Unfortunately the headers that are
generated are placed into directories corresponding to the namespaces
declared within the IDPL, and can't be determined solely based on the
input filename. As such, we have to do a quick parsing of the AST in
order to grab the namespaces and calculate the filenames for the .h
files.

The downside of this approach is that all *.ipdl/*.ipdlh files are
considered inputs for the tup backend, so editing one will perform
moz.build generation before running ipdl.py. This could be fixed by
either standardizing the generated header layout, or fixing tup to
somehow allow unspecified outputs.

Review commit: https://reviewboard.mozilla.org/r/193102/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/193102/

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8922100 [details]
Bug 1407403 - Handle IPDL code generation in the tup backend;

https://reviewboard.mozilla.org/r/193102/#review198380

Another unfortunate build system hack :/

But I don't see an obvious easy way around this. Given that perfect is very much the enemy for Tup support at this juncture, let's land this and optimize/improve it later.

::: python/mozbuild/mozbuild/backend/tup.py:452
(Diff revision 1)
> +        sys.path.append(mozpath.join(self.environment.topsrcdir, 'ipc', 'ipdl'))
> +        import ipdl

We shouldn't need to do this. But I don't want to scope bloat you.

Comment 3

[Nathan Froyd [:froydnj]]

I don't think anybody would complain too loudly if the ipdl namespacing of headers went away, FWIW.  Put them under mozilla/ipdl/ if necessary.

Comment 4

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8922100 [details]
Bug 1407403 - Handle IPDL code generation in the tup backend;

https://reviewboard.mozilla.org/r/193102/#review198386

::: python/mozbuild/mozbuild/backend/tup.py:473
(Diff revision 1)
> +	    (filepath, ext) = os.path.splitext(filename)
> +	    (dirname, basename) = os.path.split(filepath)

nit: parens around these tuples aren't necessary

Comment 5

[(not currently active) Ted Mielczarek]

(In reply to Nathan Froyd [:froydnj] from comment #3)
> I don't think anybody would complain too loudly if the ipdl namespacing of
> headers went away, FWIW.  Put them under mozilla/ipdl/ if necessary.

I mentioned this yesterday, but this was annoying for my "upload generated sources" patch as well. Let's file a followup for that since mshal already has this working.

Comment 6

[Nathan Froyd [:froydnj]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> (In reply to Nathan Froyd [:froydnj] from comment #3)
> > I don't think anybody would complain too loudly if the ipdl namespacing of
> > headers went away, FWIW.  Put them under mozilla/ipdl/ if necessary.
> 
> I mentioned this yesterday, but this was annoying for my "upload generated
> sources" patch as well. Let's file a followup for that since mshal already
> has this working.

Done, filed bug 1411969.

Comment 7

[Michael Shal [:mshal]]

Comment on attachment 8922100 [details]
Bug 1407403 - Handle IPDL code generation in the tup backend;

https://reviewboard.mozilla.org/r/193102/#review198588

::: python/mozbuild/mozbuild/backend/tup.py:473
(Diff revision 1)
> +	    (filepath, ext) = os.path.splitext(filename)
> +	    (dirname, basename) = os.path.split(filepath)

Fixed, thanks!

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8922100 [details]
Bug 1407403 - Handle IPDL code generation in the tup backend;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/193102/diff/1-2/

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/26c1d50db1c1
Handle IPDL code generation in the tup backend; r=gps

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for py-compat failure: python/mozbuild/mozbuild/backend/tup.py:470:124 | inconsistent use of tabs and spaces in indentation:

https://hg.mozilla.org/integration/autoland/rev/99c40288511ac903e786ea0cb3fa74c1d2324fed

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=139868325&repo=autoland

Comment 11

[Michael Shal [:mshal]]

Hmm, no idea how those got in there. Editor settings seem fine...

Comment 12

[Michael Shal [:mshal]]

Comment on attachment 8922100 [details]
Bug 1407403 - Handle IPDL code generation in the tup backend;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/193102/diff/2-3/

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bcb6adf5707c
Handle IPDL code generation in the tup backend; r=gps

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/bcb6adf5707c