1154047 - *** AUS:SVC readStatusFile - status: failed: 7, path: /tmp/tmpAyKRdJ.binary-update-tests/updates/0/update.status

Status: RESOLVED INVALID

(Remote Protocol :: Marionette, defect)

Description

[Armen [:armenzg]]

Attachment: update_failure_7.txt

I see a lot of these failures in today's run [output]

The timeout is reached in here [1]

I see an update notification with state "pending" being notified, however, the test is expecting the state to be change to "applied" (IIUC)
Could we wait for a state change instead of waiting for the state to be "applied"?

I think this is what is happening:
* We're "attempting to stage update" [3]
** We don't raise an exception
* I think we set the state "failed" (from "pending" in here [4])
** even though reading "FOTA" there makes me think of b2g
* We read status 7 "readStatusFile - status: failed: 7" [5]

In general, how can we take a screen shot when things go bad?

I'm considering adding a general time out of 30 seconds.
On a good run, we can pass both tests in 50 seconds.

[1] https://github.com/mozilla/firefox-ui-tests/blob/master/firefox_puppeteer/ui/about_window/window.py#L132
[2] https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/nsUpdateService.js#
[3] https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/nsUpdateService.js#91
[4] https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/nsUpdateService.js#1329
[5] https://hg.mozilla.org/releases/mozilla-beta/file/default/toolkit/mozapps/update/nsUpdateService.js#l4602

[output]
*** AUS:SVC Downloader:_verifyDownload called
*** AUS:SVC Downloader:_verifyDownload downloaded size == expected size.
*** AUS:SVC Downloader:_verifyDownload hashes match.
*** AUS:SVC Downloader:onStopRequest - setting state to: pending
*** AUS:SVC canStageUpdatesSession - testing write access /tmp/tmpAyKRdJ.binary-update-tests/update.test
*** AUS:SVC canStageUpdatesSession - testing write access /tmp/update.test
*** AUS:SVC canStageUpdatesSession - able to stage updates
*** AUS:SVC Downloader:onStopRequest - attempting to stage update: Firefox 38.0 Beta 3
Xlib:  extension "RANDR" missing on display ":2".
*** AUS:SVC readStatusFile - status: failed: 7, path: /tmp/tmpAyKRdJ.binary-update-tests/updates/0/update.status
*** AUS:SVC UpdateManager:refreshUpdateStatus - Notifying observers that the update was staged. state: pending, status: failed: 7
*** UTM:SVC TimerManager:notify - notified timerID: browser-cleanup-thumbnails
*** UTM:SVC TimerManager:notify - notified @mozilla.org/browser/search-service;1
*** AUS:SVC readStatusFile - status: pending, path: /tmp/tmpAyKRdJ.binary-update-tests/updates/0/update.status
*** UTM:SVC TimerManager:notify - notified @mozilla.org/updates/update-service;1
*** UTM:SVC TimerManager:notify - notified @mozilla.org/browser/experiments-service;1
1428945046785   addons.manager  INFO    Skipping background update check
*** UTM:SVC TimerManager:notify - notified @mozilla.org/addons/integration;1
11:11.94 LOG: MainThread WARNING Failed to gather test failure debug.
Traceback (most recent call last):

  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette/runner/base.py", line 554, in gather_debug
    rv['source'] = marionette.page_source

  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1095, in page_source
    response = self._send_message('getPageSource', 'value')

  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)

  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 697, in _send_message
    "Connection timed out", status=errors.ErrorCodes.TIMEOUT)

TimeoutException: TimeoutException: Connection timed out


11:11.94 TEST_END: MainThread ERROR, expected PASS
Traceback (most recent call last):
  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/firefox_ui_tests/update/direct/test_direct_update.py", line 21, in test_update
    self.download_and_apply_available_update(force_fallback=False)
  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/firefox_ui_harness/testcases/update.py", line 155, in download_and_apply_available_update
    about_window.wait_for_update_applied()
  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/about_window/window.py", line 134, in wait_for_update_applied
    message='Update has been applied.')
  File "/home/cltbld/temp/build-tools/release/updates/venv/lib/python2.7/site-packages/marionette_driver/wait.py", line 143, in until
    cause=last_exc)
TimeoutException: TimeoutException: Timed out after 300.1 seconds with message: Update has been applied.

Comment 1

[Armen [:armenzg]]

Maybe related to this:
[[e-mail] updates]$ mkdir temp
mkdir: cannot create directory `temp': No space left on device

Comment 2

[Armen [:armenzg]]

It was a disk space issue (bug 1154060).

chmanchester, do we want to deal with this issue? I don't think so but want to run it through you.

Comment 3

[Chris Manchester (limited bugmail, email directly)]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #2)
> It was a disk space issue (bug 1154060).
> 
> chmanchester, do we want to deal with this issue? I don't think so but want
> to run it through you.

I believe this is bug 1083131. There is some discussion in that bug about a fix and it looks tractable, but if it's straightforward to manage on the machine itself that's probably sufficient. If that isn't possible for some reason I can check into that bug this week.

Comment 4

[Armen [:armenzg]]

Out of the 26GB used on /tmp only 1GB was used by fx-ui-updates.
Once bug 1154060 is fixed, we won't see this issue anymore.
Until that is fixed, I can add a "rm -rf /tmp/*" in verify.sh

Marking it as INVALID unless you have an objection.