Nightly build bustage due to pymake overflowing stack RuntimeError: maximum recursion depth exceeded

RESOLVED FIXED in mozilla27

Status

Firefox Build System
General
P1
normal
RESOLVED FIXED
5 years ago
4 months ago

People

(Reporter: gps, Assigned: gps)

Tracking

Trunk
mozilla27
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Assignee)

Description

5 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=28423679&tree=Mozilla-Central&full=1#error0

You trace the log back and pymake is processing mochitest installations when it hits the error.

This is a legit bug in pymake where it's little brain can't deal with very large amounts of rules.

We can work around it by converting "large" directories to use manifests.

I performed file counts for directories:

    392     ./content/html/content/test
    431     ./intl/icu/source/data/coll
    437     ./js/src/jit-test/tests/ion
    442     ./layout/base/crashtests
    455     ./js/src/jit-test/tests/debug
    476     ./dom/webidl
    503     ./layout/reftests/svg
    518     ./layout/generic/crashtests
    532     ./dom/tests/mochitest/dom-level1-core
    560     ./extensions/spellcheck/hunspell/tests/unit/data
    588     ./content/base/test
    637     ./layout/reftests/margin-collapsing
    642     ./intl/icu/source/data/locales
    669     ./dom/tests/mochitest/dom-level2-html
    1185    ./js/src/jit-test/tests/basic
    3035    ./layout/reftests/bugs 

We know from the log that dom/tests/mochitest/dom-level1-core completes without issue. Let's convert some directories to manifests to work around this issue.
(Assignee)

Updated

5 years ago
Assignee: nobody → gps
Status: NEW → ASSIGNED
Priority: -- → P1
(Assignee)

Comment 1

5 years ago
Created attachment 810775 [details] [diff] [review]
Convert large directories to mochitest manifests

Convert large directories to mochitest manifests.

[test_HTMLDocument12] is a bit wonky. It appears the inactive test
filtering in emitter.py isn't working. Will file a bug.
Attachment #810775 - Flags: review?(ted)
Comment on attachment 810775 [details] [diff] [review]
Convert large directories to mochitest manifests

Review of attachment 810775 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/base/test/mochitest.ini
@@ +305,5 @@
> +[test_bug373181.xhtml]
> +[test_bug375314.html]
> +[test_bug378969.html]
> +[test_bug380418.html]
> +[test_bug380418.html^headers^]

aren't such files support files?
(Assignee)

Comment 3

5 years ago
Created attachment 810783 [details] [diff] [review]
Increase pymake recursion limit (again)

Alternative hack.
Attachment #810783 - Flags: review?(mh+mozilla)
(Assignee)

Updated

5 years ago
Blocks: 874210
Comment on attachment 810783 [details] [diff] [review]
Increase pymake recursion limit (again)

Review of attachment 810783 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/pymake/make.py
@@ +24,2 @@
>    try:
> +      sys.setrecursionlimit(100000)

Can we go for 10000 instead?
Attachment #810783 - Flags: review?(mh+mozilla) → review+
(Assignee)

Updated

5 years ago
Attachment #810775 - Attachment is obsolete: true
Attachment #810775 - Flags: review?(ted)
The Windows nightly retrigger ran fine. Thanks for the quick fix!
Target Milestone: --- → mozilla27
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Updated

5 years ago
Depends on: 925766

Updated

4 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.