807680 - DXR builds failing with out-of-disk-space

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Joshua Cranmer [:jcranmer]]

See, e.g., http://ftp.mozilla.org/pub/mozilla.org/dxr/tinderbox-builds/mozilla-central-mock/dxr-mozilla-central-bm35-build1-build22.txt.gz or http://ftp.mozilla.org/pub/mozilla.org/dxr/tinderbox-builds/mozilla-central-mock/dxr-mozilla-central-bm35-build1-build23.txt.gz

Comment 1

[[e-mail] (:bhearsum)]

Any idea how much space a successful dxr build would need? We currently ensure we have 4gb, need to bump the "-s" argument to https://hg.mozilla.org/build/tools/file/default/scripts/dxr/dxr.sh#l39 to a better value.

Comment 2

[Joshua Cranmer [:jcranmer]]

Offhand, no. Try 6GB for now?

Comment 3

[[e-mail] (:bhearsum)]

Attachment: bump to 6

Comment 4

[[e-mail] (:bhearsum)]

OK, I triggered a new build with this checked in.

Comment 5

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=16669077&tree=Firefox

10.49 GB of space available
...
gzip: stdout: No space left on device

Comment 6

[Joshua Cranmer [:jcranmer]]

The output tarball takes up ~400MB of space (round up to find space padding needed for extra mozilla-central code). Disk-space ran out somewhere during www/ indexing, so 10.5 GB gives us enough space to at least build and do the sqlite-generation for indexing. du -s -h of the last extracted successful tarball comes out to 2.2 GB (of which ~650MB comes from the .dxr_xref directory that should already build). Total necessary space then comes out to 10.5 + 0.4 + 2.2 - .65 = 12.45 (which ought to guarantee that it doesn't run out of space). Given how far we get, I think we might be able to fit in 12GB, but 13GB should make things work easily on the safe side...

Comment 7

[[e-mail] (:bhearsum)]

(In reply to Joshua Cranmer [:jcranmer] from comment #6)
> The output tarball takes up ~400MB of space (round up to find space padding
> needed for extra mozilla-central code). Disk-space ran out somewhere during
> www/ indexing, so 10.5 GB gives us enough space to at least build and do the
> sqlite-generation for indexing. du -s -h of the last extracted successful
> tarball comes out to 2.2 GB (of which ~650MB comes from the .dxr_xref
> directory that should already build). Total necessary space then comes out
> to 10.5 + 0.4 + 2.2 - .65 = 12.45 (which ought to guarantee that it doesn't
> run out of space). Given how far we get, I think we might be able to fit in
> 12GB, but 13GB should make things work easily on the safe side...

Thanks for the info Joshua. I bumped it up to 14! There's a new job running now, we'll see how it goes.

Comment 8

[[e-mail] (:bhearsum)]

Most recent rebuild hung midbuild:
essed kind ClassTemplate
nsGNOMEShellService.cpp
Makefile:137: browser_459906.js is disabled for intermittent failures. Bug 766044
/builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/components/BrowserComponents.manifest: WARNING: no useful preprocessor directives found
eParm
Unprocessed /builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/locales/en-US/searchplugins/bing.xml: WARNING: no preprocessor directives found
/builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/locales/en-US/searchplugins/eBay.xml: WARNING: no preprocessor directives found
kind ClassTemplate
/builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/locales/en-US/searchplugins/twitter.xml: WARNING: no preprocessor directives found
/builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/locales/en-US/searchplugins/wikipedia.xml: WARNING: no preprocessor directives found
/builds/slave/dxr-mozilla-central/dxr-build-env/src/browser/locales/en-US/searchplugins/yahoo.xml: WARNING: no preprocessor directives found
nsModule.cpp
/bu

Unlikely to be related to disk space, trying again....

Comment 9

[[e-mail] (:bhearsum)]

My build hung again. I filed bug 808156 on that.

Comment 10

[[e-mail] (:bhearsum)]

Looks like this worked, I see green builds.