503338 - Fix "hdiutil: create failed - Device not configured" issue in build queue / partner-repack.py

Status: VERIFIED FIXED

(Websites Graveyard :: byob.mozilla.com, defect)

Description

[Les Orchard [:lorchard]]

> * After running awhile, my offline queue appears to fail for Mac builds with "hdiutil: create failed - Device not configured".  The hdiutil command used by the repack script doesn't seem to like the detached login session.  Not sure what the solution is - has build / release engineering solved this somehow?  See also: http://developer.apple.com/documentation/Darwin/Reference/Manpages/man1/hdiutil.1.html 

I believe this is handled by release automation. I'll send email to coop and see how they handle it in the current build environment.

Comment 1

[Kev Needham [:kev]]

Coop, if you have some cycles, can you review with Les and I? We're in BYOB, and need a little help with implementation.

Comment 2

[Chris Cooper [:coop] (he/him)]

Possibly related to bug 499506? We should probably make .dmg unpacking more robust in the same style as installdmg.sh.

Comment 3

[Les Orchard [:lorchard]]

FWIW, the relevant section from the manpage, though still not sure of a fix:

[ENXIO]            
Device not configured.  This error is returned by DiskImages when its kernel driver
or framework helper cannot be contacted.  The former usually means the IOHDIXCon-troller IOHDIXController
troller kernel extension can't be loaded.  The latter usually means Foundation's
distributed objects RPC mechanism cannot be configured.  DO doesn't work under dead
mach bootstrap contexts such as exist in a reattached screen(1) session.  Root users
can take advantage of StartupItemContext(8) (in /usr/libexec) to access the startup
item mach bootstrap context.

Comment 4

[Les Orchard [:lorchard]]

Oh, and fwiw: I've been trying to run a PHP script that manages a work queue in a detached screen session, and later using nohup.  Running the queue in another context might help, but wondering how this is handled elsewhere

Comment 5

[Les Orchard [:lorchard]]

One more update from our meeting today: The work queue I've got can be run periodically from a crontab in one-shot exhaustion mode, so if hdiutil is happy running as root from cron that could be a solution.

Comment 6

[Les Orchard [:lorchard]]

Looks like this is working as a launchd scheduled task.

r30236

Comment 7

[Kev Needham [:kev]]

cool. so instead of a queue, you're just running a cron job now that checks for builds and executes when there's a job (or jobs) in the queue?

Comment 8

[Les Orchard [:lorchard]]

Sort of.  A launchd script exhausts the queue every 10 seconds.  There's a lock file, so if the exhaustion takes longer than 10 seconds, additional parallel runs are prevented.  

The queue also handles things like moving build files around (eg. from private review to public download), since the file ownership / permissions tend to deny the web app from doing so.

Comment 9

[Carsten Book [:Tomcat]]

have not seen this during testing so marking verified