Closed Bug 985837 Opened 10 years ago Closed 10 years ago

emulator-kk builds crash make half the time (which it describes as "failed to build")


(Release Engineering :: General, defect)

Gonk (Firefox OS)
Not set


(Not tracked)



(Reporter: philor, Unassigned)



(Keywords: intermittent-failure)


(2 files)

Like I'd find it to be a build improvement, just like it said on the tin, except people keep retriggering them until they turn green, instead of accepting the blessing of a quick red.

"Maybe we can force to use -j <10."
this is definitely make crashing, as opposed to bash or nice:
Mar 20 03:54:56 bld-linux64-spot-332 kernel: make[12579]: segfault at 0 ip 0000000000416f61 sp 00007fff78be61c0 error 4 in make[400000+28000]
Mar 20 03:54:56 bld-linux64-spot-332 abrt[24810]: Can't read /proc/12579/exe link
we've got make-3.81-19.el6.x86_64 installed
Hidden on m-i, m-c and b-i, since they couldn't see the visiblity requirement of <5% failure if they climbed a ladder.
And hidden on mozilla-aurora, but it seems I'm back on the &onlyunstarred=1 so rest assured I'll keep you updated on how frequently they fail. doesn't actually set -j anything.

It looks like .config has:


And has this in it:

echo MAKE_FLAGS=-j$((CORE_COUNT + 2)) > .tmp-config

So the simplest way of overriding this would be to do:

echo "MAKE_FLAGS=-j4" >> .config

after running and before running

Otherwise we could add an option to that allows the default -j setting to be set when is run.
no idea if this will help or not, but worth a shot
Attachment #8396860 - Flags: review?(aki)
Comment on attachment 8396860 [details] [diff] [review]
force -j4 for emulator-kk

Thanks Catlee.
Attachment #8396860 - Flags: review?(aki) → review+
Attachment #8396860 - Flags: checked-in+
That seems to have taken care of the "half the time" problem: now they fail to build 100% of the time.

let's try -j1 then
Attachment #8397819 - Flags: review?(rail)
Attachment #8397819 - Flags: review?(rail) → review+
Attachment #8397819 - Flags: checked-in+
in production
in production
Okay, almost sort of builds fairly often now, except that it hits bug 934890 an awful lot of the time,
Still hidden, still a constant source of failure, but not from this :)
Closed: 10 years ago
Resolution: --- → FIXED
FTR backed out this change since it seems to be working well in staging without the workaround.
Should be a fun bit of statistical analysis, figuring out whether the (still continuing) 14400 second timeouts have decreased enough to make the rate of these worth tolerating.
Resolution: FIXED → ---
Something here went live today
make 3.82 seems to staging
Chris, what are next steps here? :-)
Flags: needinfo?(catlee)
Depends on: 1017166
Blocks: 1017545
KK builds hidden on all trees for too many failures; will unhide in bug 1017545, once bug 1017166 fixes the failures here.
Flags: needinfo?(catlee)
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.