Annotate processed moz.build files with tier info

RESOLVED FIXED in mozilla27

Status

RESOLVED FIXED
5 years ago
6 months ago

People

(Reporter: gps, Assigned: glandium)

Tracking

Trunk
mozilla27

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

5 years ago
I will explain this better in the description for the soon-to-be-attached patch.
(Reporter)

Comment 1

5 years ago
Created attachment 796895 [details] [diff] [review]
Annotate read moz.build files with tier info

This patch annotates the processed moz.build sandboxes with tier
metadata. The goal of this patch is to eventually enable the make-based
backend to easily generate a make file that incurs submakes for every
directory. i.e. instead of performing recursive traversal, we'll just
list each directory as a flat list and have make evaluate each one using
whatever concurrency it is capable of. This will likely be done
differently for export, libs, tools, etc. But that's followup bug
territory.

Since we may add additional metadata later, I decided to make the
metadata a dict rather than hardcode a "tier" argument/annotation
everywhere.
Attachment #796895 - Flags: review?(mh+mozilla)
(Reporter)

Updated

5 years ago
Assignee: nobody → gps
(Reporter)

Comment 2

5 years ago
Created attachment 796906 [details] [diff] [review]
Part 2: Write tier directory info into make backend

While I was here, I figured we should write out the full list of tier
directories in the root backend.mk file. Next step is to modify
/Makefile.in and the tier traversal rules to leverage this data to build
non-recursively.
Attachment #796906 - Flags: review?(mh+mozilla)
(Assignee)

Comment 3

5 years ago
(In reply to Gregory Szorc [:gps] from comment #1)
> This patch annotates the processed moz.build sandboxes with tier
> metadata. The goal of this patch is to eventually enable the make-based
> backend to easily generate a make file that incurs submakes for every
> directory. i.e. instead of performing recursive traversal, we'll just
> list each directory as a flat list and have make evaluate each one using
> whatever concurrency it is capable of.

I'm close to a patch doing that already, but i didn't need that tier info.
(Assignee)

Comment 4

5 years ago
Comment on attachment 796895 [details] [diff] [review]
Annotate read moz.build files with tier info

Review of attachment 796895 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/frontend/reader.py
@@ +642,5 @@
>          # It's very tempting to use a set here. Unfortunately, the recursive
>          # make backend needs order preserved. Once we autogenerate all backend
>          # files, we should be able to convert this to a set.
> +        recurse_info = []
> +        dirs = set()

Just use an OrderedDict for recurse_info, and don't keep a separate dirs list, which will be recurse_info.keys().
Attachment #796895 - Flags: review?(mh+mozilla) → feedback+
(Assignee)

Comment 5

5 years ago
Created attachment 804814 [details] [diff] [review]
Annotate read moz.build files with tier info

Since i was going to add stuff on top of this, i needed it updated, so this is your patch with my review comment applied. Just getting it through you in case you have something to add.
Attachment #804814 - Flags: review?(gps)
(Assignee)

Updated

5 years ago
Attachment #796895 - Attachment is obsolete: true
(Assignee)

Updated

5 years ago
Assignee: gps → mh+mozilla
Status: NEW → ASSIGNED
(Reporter)

Updated

5 years ago
Attachment #804814 - Flags: review?(gps) → review+
(Assignee)

Comment 6

5 years ago
Comment on attachment 796906 [details] [diff] [review]
Part 2: Write tier directory info into make backend

Review of attachment 796906 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not convinced this tier summary is any useful outside the recursive make backend. That is, it doesn't seem to me something the emitter should be emitting.
Attachment #796906 - Flags: review?(mh+mozilla)
(Assignee)

Updated

5 years ago
Blocks: 917086
https://hg.mozilla.org/mozilla-central/rev/fdaa11265464
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27

Updated

6 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.