Closed Bug 914731 Opened 9 years ago Closed 9 years ago

[Gaia] Parallel builds are broken (again)

Categories

(Firefox OS Graveyard :: Gaia, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gsvelto, Assigned: gsvelto)

Details

Attachments

(2 files)

The optimize-clean rule seem to fail during parallel builds. From a quick glance it seems to me that it needs to happen after webapp-optimize while in the current system it may run in parallel to it leading to the error.
I'm testing this patch right now, I did a few tens of runs and haven't hit the problem again so I think this should fix the issue.

Long-term it would be better if we could encode these operations (generating optimized l10n files, etc...) as real makefile rules so we have proper dependencies and more parallelism.
Assignee: nobody → gsvelto
Status: NEW → ASSIGNED
Attachment #802461 - Flags: review?(timdream)
Comment on attachment 802461 [details] [diff] [review]
[PATCH] Serialize the webapp optimization and packaging steps

Yuren, could you review this? Thanks.
Attachment #802461 - Flags: review?(timdream) → review?(yurenju.mozilla)
Gabriele, your patch looks good but I would like to know how to reproduce this bug, could you provide the steps?
(In reply to Yuren Ju [:yurenju] from comment #3)
> Gabriele, your patch looks good but I would like to know how to reproduce
> this bug, could you provide the steps?

It seems to happen when doing a parallel build with a large number of jobs (and cores/hardware threads to back them up), see bug 810382 comment 7. I could reproduce it on my box but I couldn't find a reliable way to make it happen. In general it seems to happen once every few runs if I touch one of the HTML files in between which causes the corresponding l10n optimized file to be regenerated. But it happens like once every 5-10 runs on my box which has only 2 cores / 4 threads; maybe on a wider box it should happen more often.
Comment on attachment 802461 [details] [diff] [review]
[PATCH] Serialize the webapp optimization and packaging steps

that's not easy to reproduce even I tried -j30 on my box. but this change make dependencies of makefile more clean so I would like to give r+ if this can solve parallel build issue.
Attachment #802461 - Flags: review?(yurenju.mozilla) → review+
This patch looks good to me too fwiw
Thanks for the review!

(In reply to Yuren Ju [:yurenju] from comment #5)
> that's not easy to reproduce even I tried -j30 on my box. but this change
> make dependencies of makefile more clean so I would like to give r+ if this
> can solve parallel build issue.

Yeah, it's hard to reproduce. What I did is to serialize those dependencies so that they happen in the same order as when make is run with -j1.
Attachment #802983 - Attachment description: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/12117 → [PULL REQUEST] Serialize the webapp optimization and packaging steps
Merged in master:

https://github.com/mozilla-b2g/gaia/commit/bc795d7e3d10c8295c1368704a46926dcd0ca1de

Fingers crossed in the hope we don't hit more of these issues in Gaia.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
(In reply to Michael Vines [:m1] [:evilmachines] from comment #6)
> This patch looks good to me too fwiw

If you hit the problem again please re-open this bug. On my side I've being building Gaia since yesterday with this patch applied and I haven't hit the issue anymore.
You need to log in before you can comment on or make changes to this bug.