1328238 - OS X 10.10 debug test run into hdiutil: attach failed - image data corrupted for installer.dmg (local variable 'appDir' referenced before assignment)"

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Carsten Book [:Tomcat]]

like https://treeherder.mozilla.org/logviewer.html#?job_id=65815818&repo=autoland#L1679 - seems to hit OS 10.10 debug on autoland only for some reason

retrigger also failed with this error

Comment 1

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Attachment: script2.PNG

I was not able to test on OS X 10.10 debug as I couldn't retrieve the properties of buildprops.json.
For example by checking the logs of a failed job (1) and manually add these properties into buildprops.json of my loaner (with the name of the machine changed) I hit other errors when running  the script from step 8 (2)(3).

Is there a way to test on a loaner this job using specific platform,branch and buildername? 
All these fails only took place on OSX 10.10 autoland debug  (4)


(1)https://treeherder.mozilla.org/logviewer.html#?job_id=65815821&repo=autoland#L371-L393
(2)script2.png
(3)error.png 
(4) https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=141fe3b986cd80d7781545b24413462735c76545&filter-searchStr=OSX

Comment 2

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Attachment: error.png

Comment 3

[Intermittent Failures Robot]

36 failures in 76 pushes (0.474 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 36

Platform breakdown:
* osx-10-10: 36

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1328238&startday=2017-01-03&endday=2017-01-03&tree=all

Comment 4

[Intermittent Failures Robot]

36 failures in 563 pushes (0.064 failures/push) were associated with this bug in the last 7 days. 

This is the #34 most frequent failure this week. 

Repository breakdown:
* autoland: 36

Platform breakdown:
* osx-10-10: 36

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1328238&startday=2017-01-02&endday=2017-01-08&tree=all

Comment 5

[Justin Wood (:Callek)]

Looking at the code, and the log...


It *looks* like there was a corruption downloading the .dmg (or the .dmg it was using was corrupt in some way!?)

This corresponds to error line:

> hdiutil: attach failed - image data corrupted

from https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozinstall/mozinstall/mozinstall.py?q=path%3Amozinstall.py&redirect_type=single#249

The log then goes on to print a traceback:


> 00:36:50    ERROR -  Traceback (most recent call last):
> 00:36:50    ERROR -    File "/builds/slave/test/build/venv/bin/mozinstall", line 9, in <module>
> 00:36:50    ERROR -      load_entry_point('mozInstall==1.12', 'console_scripts', 'mozinstall')()
> 00:36:50    ERROR -    File "/builds/slave/test/build/venv/lib/python2.7/site-packages/mozinstall/mozinstall.py", line 328, in install_cli
> 00:36:50    ERROR -      install_path = install(src, options.dest)
> 00:36:50    ERROR -    File "/builds/slave/test/build/venv/lib/python2.7/site-packages/mozinstall/mozinstall.py", line 114, in install
> 00:36:50    ERROR -      install_dir = _install_dmg(src, dest)
> 00:36:50    ERROR -    File "/builds/slave/test/build/venv/lib/python2.7/site-packages/mozinstall/mozinstall.py", line 274, in _install_dmg
> 00:36:50    ERROR -      subprocess.call('hdiutil detach %s -quiet' % appDir,
> 00:36:50    ERROR -  mozinstall.mozinstall.InstallError: Failed to install "/builds/slave/test/installer.dmg (local variable 'appDir' referenced before assignment)"

This traceback corresponds to https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozinstall/mozinstall/mozinstall.py?q=path%3Amozinstall.py&redirect_type=single#274

Which is in a finally: clause of a try block, inside the try block *after* the attaching of the image is where data is set, so in actual practice, we don't have a mounted /Volume/ to detatch from in this case, and is why this is failing...

Comment 6

[Andrei Obreja [:aobreja NOT AVAILABLE][:buildduty]]

Last occurrence was 2 weeks ago,seems the problem only happened in 03 January and only for (autoland,debug)(1) and since then some of the failed tests ran on the same machines and were successfully. 
The .dmg that was used couldn't be corrupt as other machines also used that .dmg and the tests finished without fail.
Will change the status of this bug to resolved for now.



(1) https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=141fe3b986cd80d7781545b24413462735c76545&filter-searchStr=OSX