1437182 - Associate object files with linkables in the emitter

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Comment 1

[Chris Manchester (limited bugmail, email directly)]

Attachment: Bug 1437182 - Note object files associated with linkables in the emitter.

Review commit: https://reviewboard.mozilla.org/r/219198/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/219198/

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Attachment: Bug 1437182 - Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter.

Review commit: https://reviewboard.mozilla.org/r/219200/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/219200/

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8949886 [details]
Bug 1437182 - Note object files associated with linkables in the emitter.

https://reviewboard.mozilla.org/r/219198/#review225360

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8949887 [details]
Bug 1437182 - Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter.

https://reviewboard.mozilla.org/r/219200/#review225362

I'll r+ this. I'm not too keen about not using sets. But the code is already not using sets. I doubt there are major performance implications. So it should be fine.

::: python/mozbuild/mozbuild/frontend/data.py:322
(Diff revision 1)
> +        return (sorted(list(WebIDLCodegenManager.GLOBAL_DEFINE_FILES)) +
> +                sorted(dict(self.unified_source_mapping).keys()))

I think this should return a set and sorting should be done by the caller. But this will work as well.

::: python/mozbuild/mozbuild/frontend/data.py:368
(Diff revision 1)
>                                          unified_prefix='UnifiedProtocols',
>                                          unified_suffix='cpp',
>                                          files_per_unified_file=16))
>  
> +    def all_source_files(self):
> +        return sorted(dict(self.unified_source_mapping).keys())

The inner expression here can just be `set(i[0] for i in self.unified_source_mapping)`, since `self.unified_source_mapping` is a list of 2-tuples. Casting to a dict only to throw away the values only adds overhead.

The sorting can be done here or in the caller. Generally speaking, I like keeping things as sets for as long as possible. It is a much more efficient data structure than lists. I'm pretty sure I wrote a lot of this code and am not sure why I didn't use sets more. Perhaps it was before I was fully enlightened about their awesomeness?

Comment 5

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8949886 [details]
Bug 1437182 - Note object files associated with linkables in the emitter.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/219198/diff/1-2/

Comment 6

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8949887 [details]
Bug 1437182 - Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/219200/diff/1-2/

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/19497c8c5b6f
Note object files associated with linkables in the emitter. r=gps
https://hg.mozilla.org/integration/autoland/rev/588efc6f3520
Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter. r=gps

Comment 8

[Andreea Pavel [:apavel]]

Backed out for failing gecko decision task

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=588efc6f352087699feca72841c7a1fa0f8438d1

Failure log: https://hg.mozilla.org/integration/autoland/rev/4b8c961c0a25bbb44c5231da23ca118d5ed1dfdf

Backout: https://hg.mozilla.org/integration/autoland/rev/4b8c961c0a25bbb44c5231da23ca118d5ed1dfdf

Comment 9

[Chris Manchester (limited bugmail, email directly)]

(In reply to Andreea Pavel [:apavel] from comment #8)
> Backed out for failing gecko decision task
> 
> Push with failures:
> https://treeherder.mozilla.org/#/
> jobs?repo=autoland&revision=588efc6f352087699feca72841c7a1fa0f8438d1
> 
> Failure log:
> https://hg.mozilla.org/integration/autoland/rev/
> 4b8c961c0a25bbb44c5231da23ca118d5ed1dfdf
> 
> Backout:
> https://hg.mozilla.org/integration/autoland/rev/
> 4b8c961c0a25bbb44c5231da23ca118d5ed1dfdf

"ImportError: No module named mozwebidlcodegen" in the decision task. 

This isn't failing on try, or when I run the decision task's command locally. I guess moving this to a local import would fix it.

Comment 10

[Chris Manchester (limited bugmail, email directly)]

Oh, I bet this has something to do with sparse profiles.

Comment 11

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8949886 [details]
Bug 1437182 - Note object files associated with linkables in the emitter.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/219198/diff/2-3/

Comment 12

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8949887 [details]
Bug 1437182 - Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/219200/diff/2-3/

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/caad7f97fe38
Note object files associated with linkables in the emitter. r=gps
https://hg.mozilla.org/integration/autoland/rev/f41883aac000
Take IPDL/WebIDL generated sources into account when associating object files with linkables in the emitter. r=gps

Comment 14

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/caad7f97fe38
https://hg.mozilla.org/mozilla-central/rev/f41883aac000