Open Bug 958812 Opened 10 years ago Updated 2 years ago

Intermittent mozinstall test.py | Failed to install .../Installer-Stubs/firefox.dmg

Categories

(Testing :: Mozbase, defect)

x86_64
macOS
defect

Tracking

(Not tracked)

People

(Reporter: ewong, Unassigned)

References

(Depends on 1 open bug)

Details

Attachments

(3 files)

Getting these errors:

======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 119, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 146, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 119, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 146, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 119, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 146, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 119, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 231, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

----------------------------------------------------------------------
Ran 231 tests in 88.188s

FAILED (errors=9, skipped=2)
make[2]: *** [test.py-run] Error 1
make[2]: Target `check' not remade because of errors.
make[1]: *** [check] Error 2
make: *** [check] Error 2
Summary: Errors with mozinstall section of the tests. → Errors in the mozinstall section of the tests.
InstallError: Failed to install "/builds/slave/c-cen-t-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

Erk! All these tests appear to be hard coded to use "firefox".

http://mxr.mozilla.org/comm-central/source/mozilla/testing/mozbase/mozinstall/tests/test.py#24

24         cls.dmg = os.path.join(here, 'Installer-Stubs', 'firefox.dmg')
25         cls.exe = os.path.join(here, 'Installer-Stubs', 'firefox.exe')
26         cls.zipfile = os.path.join(here, 'Installer-Stubs', 'firefox.zip')
27         cls.bz2 = os.path.join(here, 'Installer-Stubs', 'firefox.tar.bz2')
Depends on: 895940
Flags: needinfo?(ahalberstadt)
(In reply to Philip Chee from comment #1)
> Erk! All these tests appear to be hard coded to use "firefox".
> 
> http://mxr.mozilla.org/comm-central/source/mozilla/testing/mozbase/
> mozinstall/tests/test.py#24
> 
> 24         cls.dmg = os.path.join(here, 'Installer-Stubs', 'firefox.dmg')
> 25         cls.exe = os.path.join(here, 'Installer-Stubs', 'firefox.exe')
> 26         cls.zipfile = os.path.join(here, 'Installer-Stubs', 'firefox.zip')
> 27         cls.bz2 = os.path.join(here, 'Installer-Stubs', 'firefox.tar.bz2')

They're hardcoded because those are dummy binaries that are checked into the tree. Can you provide some more context here? Are these happening when you run them locally? Or is it on try? If the latter can you post a log?
Flags: needinfo?(ahalberstadt)
These are tests running on the SeaMonkey tbpl. Unfortunately:
1. Our tbpl is sick. Ewong knows how to get the logs though.
2. It's Chinese New Year so ewong and myself won't be around much for the next few days.
(In reply to Andrew Halberstadt [:ahal] from comment #2)
> (In reply to Philip Chee from comment #1)
> > Erk! All these tests appear to be hard coded to use "firefox".
> > 
> > http://mxr.mozilla.org/comm-central/source/mozilla/testing/mozbase/
> > mozinstall/tests/test.py#24
> > 
> > 24         cls.dmg = os.path.join(here, 'Installer-Stubs', 'firefox.dmg')
> > 25         cls.exe = os.path.join(here, 'Installer-Stubs', 'firefox.exe')
> > 26         cls.zipfile = os.path.join(here, 'Installer-Stubs', 'firefox.zip')
> > 27         cls.bz2 = os.path.join(here, 'Installer-Stubs', 'firefox.tar.bz2')
> 
> They're hardcoded because those are dummy binaries that are checked into the
> tree. Can you provide some more context here? Are these happening when you
> run them locally? Or is it on try? If the latter can you post a log?

It's on SeaMonkey's trunk OSX64 build.

Going to attach the log.
Attached file make_check.log
This is the log from one of SeaMonkey's OSX64 trunk builds during the |make check| step.
(In reply to Edmund Wong (:ewong) from comment #5)
> Created attachment 8371142 [details]
> make_check.log
> 
> This is the log from one of SeaMonkey's OSX64 trunk builds during the |make
> check| step.

Andrew: log attached to comment 5.
Flags: needinfo?(ahalberstadt)
This looks familiar, it might already be fixed upstream. Henrik, do you remember seeing this before?
Flags: needinfo?(ahalberstadt) → needinfo?(hskupin)
Attachment #8371142 - Attachment mime type: text/x-log → text/plain
Flags: needinfo?(hskupin)
It's hard to see the real problem given that we mangle the real underlying issue. I think it would be good to know if the latest code which has been landed on inbound (not merged to m-c yet) works for you. Please let me know if the problem still persists.
Flags: needinfo?(ewong)
This happened on a try push as well:
https://tbpl.mozilla.org/php/getParsedLog.php?id=34974965&tree=Try
Component: Testing Infrastructure → Mozbase
Product: SeaMonkey → Testing
QA Contact: hskupin
Summary: Errors in the mozinstall section of the tests. → Intermittent mozinstall test.py | Failed to install .../Installer-Stubs/firefox.dmg
We used to hit this a lot in RelEng, and eventually figured out that 'hdiutil attach' can return before the image is actually attached. We wrote this tool to cope:
http://hg.mozilla.org/build/tools/file/19569ea375a7/buildfarm/utils/installdmg.sh

Basically we poll the mount point until files show up.

Maybe we're seeing the same thing here?
Yeah, definitely wouldn't hurt to add a similar check to mozinstall either way.
That's exactly what I'm trying to do on bug 969334, because that's exactly what we also had in our own install script before moving to mozinstall:

https://github.com/mozilla/mozmill-automation/blob/hotfix-1.5/mozmill_automation/install.py#L113

So I think if that is the reason we can get those failures fixed via bug 969334.
Depends on: 969334
(In reply to Henrik Skupin (:whimboo) [away 02/24 - 02/28] from comment #8)
> It's hard to see the real problem given that we mangle the real underlying
> issue. I think it would be good to know if the latest code which has been
> landed on inbound (not merged to m-c yet) works for you. Please let me know
> if the problem still persists.

Sure.  Thanks
Flags: needinfo?(ewong)
On SeaMonkey's beta and aurora builds, we're getting this every time. (trunk
builds are busted during compile due to a different bug.


======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 129, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 156, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 129, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 156, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_get_binary (test.TestMozInstall)
Test mozinstall's get_binary method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 58, in test_get_binary
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_install (test.TestMozInstall)
Test mozinstall's install capability
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 129, in test_install
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

======================================================================
ERROR: test_uninstall (test.TestMozInstall)
Test mozinstall's uninstall capabilites
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/test.py", line 156, in test_uninstall
    installdir = mozinstall.install(self.dmg, self.tempdir)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 121, in install
    install_dir = _install_dmg(src, dest)
  File "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/mozinstall/mozinstall.py", line 249, in _install_dmg
    for appFile in os.listdir(appDir):
InstallError: Failed to install "/builds/slave/c-beta-osx64/build/mozilla/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg"

----------------------------------------------------------------------
Ran 339 tests in 98.204s

FAILED (errors=9, skipped=18)
make[2]: *** [test.py-run] Error 1
make[2]: Target `check' not remade because of errors.
make[1]: *** [check] Error 2
make: *** [check] Error 2
I don't have a mac machine handy to try to reproduce/test this, but if :catlee's theory is correct, then this should fix it (I think).

Edmund, could you try pushing this to the seamonkey tbpl? (Wherever it is that it picks up mozbase from). Even if this doesn't fix the problem, it makes it so we don't cover up the exception with an "InstallError" so it should give us a better idea of what is going on.
Flags: needinfo?(ewong)
I manually applied the patch to an osx64 c-b build tree and ran |make -k check| manually.  The results, I've attached; however, I'm not sure if it's 'right'
because of the Permission issues.  (iiuc, the xpcshell tests are run under the user seabld..  which doesn't seem to have access to /Volume/Stub. ) I'll need
to confer with :Callek if I indeed missed something.
Flags: needinfo?(ewong)
I'm not sure how deeply bug 969334 is involved here but install on OS X is kinda broken. I wish that I would have had time yet to get this fixed.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.