Closed Bug 1317041 Opened 9 years ago Closed 8 years ago

Intermittent mozmake.EXE[4]: *** [symbolsfullarchive] Error 18

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox58 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla58
Tracking Flags:
Tracking Status
firefox58 --- fixed

People

(Reporter: aryx, Assigned: marco)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Patch
3.42 KB, patch
gps
: review+
Details | Diff | Splinter Review
https://treeherder.mozilla.org/logviewer.html#?job_id=39103333&repo=mozilla-inbound 21:19:11 INFO - adding: xul.pdb/0C50CE1269DD411EB0A93E9EAA4E37FF1/xul.pd_ (176 bytes security) (deflated 1%) 21:19:11 INFO - adding: xul.pdb/0C50CE1269DD411EB0A93E9EAA4E37FF1/xul.sym (176 bytes security) (deflated 84%) 21:19:11 INFO - Makefile:337: recipe for target 'symbolsfullarchive' failed 21:19:11 INFO - mozmake.EXE[4]: *** [symbolsfullarchive] Error 18 21:19:11 INFO - mozmake.EXE[4]: Leaving directory 'c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/src/obj-firefox' 21:19:11 INFO - c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/src/build/moz-automation.mk:122: recipe for target 'automation/buildsymbols' failed 21:19:11 INFO - mozmake.EXE[3]: *** [automation/buildsymbols] Error 2
Depends on: 1314713
this is specific to windowns 7 taskcluster builds in debug mode- :ted, is there anything you can think of to fix this or reduce this failure?
Flags: needinfo?(ted)
10:33:14 INFO - File "z:\build\build\src\python\mozbuild\mozpack\mozjar.py", line 748, in crc32 10:33:14 INFO - return zlib.crc32(self._data.getvalue()) & 0xffffffff 10:33:14 INFO - MemoryError We're hitting an OOM trying to create the symbols zip file, presumably because the content is very large and the Python we're using on Windows is 32-bit. Specifically, this line is trying to calculate the CRC32 checksum for the zip file, for which it needs the entire uncompressed contents of what it's compressing, which is failing here. There are a few things we could do to fix this: 1) Switch our Windows builds to use a 64-bit Python. 2) Fix this mozjar code to calculate the crc32 as data is written to the object instead of all-at-once at the end. The zlib.crc32 function we're using says in its docs that you can use it incrementally: https://docs.python.org/2/library/zlib.html#zlib.crc32 We'd have to add a field to the `Deflater` object to hold the running crc32, then in `write`: https://dxr.mozilla.org/mozilla-central/rev/13d241d08912be31884f9d0d0e805b25343d6c0a/python/mozbuild/mozpack/mozjar.py#705 do something like: args = (data,) if self._crc is None else (data, self._crc) self._crc = zlib.crc32(*args) (assuming we initialized self._crc to None).
Flags: needinfo?(ted)
:ted- this is trending to be a more frequent failure- possibly this is something you can look at
Flags: needinfo?(ted)
I put a summary of possible fixes in comment 15, we just need to pick one and execute it.
Flags: needinfo?(ted)
:ted, is this something you would do- I am trying to determine who is doing the next step here and both of the options looks like build system tooling.
Flags: needinfo?(ted)
Assignee: nobody → mcastelluccio
Blocks: 1413015
Status: NEW → ASSIGNED
I'm trying out option 2, but option 1 will probably be needed to fully fix bug 1413015.
Attached patch Patch — — Splinter Review
Attachment #8924283 - Flags: review?(ted)
Comment on attachment 8924283 [details] [diff] [review] Patch Review of attachment 8924283 [details] [diff] [review]: ----------------------------------------------------------------- LGTM. I'll land this.
Attachment #8924283 - Flags: review?(ted) → review+
Pushed by gszorc@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/f47cb574c96e Calculate crc32 incrementally; r=gps
Flags: needinfo?(ted)
https://hg.mozilla.org/mozilla-central/rev/f47cb574c96e
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox58: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla58
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: