[WinXP] TEST-UNEXPECTED-ERROR | test_ssl_status_after_restart.py TestSSLStatusAfterRestart.test_ssl_status_after_restart | TimeoutException: TimeoutException: Timed out after 5.1 seconds

RESOLVED WONTFIX

Status

RESOLVED WONTFIX
3 years ago
3 years ago

People

(Reporter: whimboo, Assigned: whimboo)

Tracking

({intermittent-failure})

unspecified
mozilla47
Unspecified
Windows XP
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox43 disabled, firefox44 disabled, firefox45 fixed, firefox46 fixed, firefox47 fixed, firefox-esr38 disabled)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Assignee)

Description

3 years ago
The test fails currently on Windows only with the following failure:

Traceback (most recent call last):
  File "c:\jenkins\workspace\mozilla-central_remote\venv\lib\site-packages\marionette_client-0.12-py2.7.egg\marionette\marionette_test.py", line 296, in run
    testMethod()
  File "c:\jenkins\workspace\mozilla-central_remote\firefox-ui-tests\firefox_ui_harness\decorators.py", line 15, in wrapper
    return target(self, *args, **kwargs)
  File "c:\jenkins\workspace\mozilla-central_remote\firefox-ui-tests\firefox_ui_harness\decorators.py", line 23, in wrapper
    return target(self, *args, **kwargs)
  File "c:\jenkins\workspace\mozilla-central_remote\firefox-ui-tests\firefox_ui_tests\remote\security\test_ssl_status_after_restart.py", line 61, in test_ssl_status_after_restart
    self.verify_certificate_status(item)
  File "c:\jenkins\workspace\mozilla-central_remote\firefox-ui-tests\firefox_ui_tests\remote\security\test_ssl_status_after_restart.py", line 74, in verify_certificate_status
    Wait(self.marionette).until(lambda _: self.identity_popup.is_open)
  File "c:\jenkins\workspace\mozilla-central_remote\venv\lib\site-packages\marionette_driver-0.5-py2.7.egg\marionette_driver\wait.py", line 143, in until
    cause=last_exc)

TimeoutException: TimeoutException: Timed out after 5.0 seconds

The reason here is that the appropriate tab is not selected before.  This happens because the test does not wait until the session has been restored.

As long as we cannot wait for the appropriate sessionrestored event, we should add a sleep. Maybe the best location for that is directly in the FirefoxTestCase.restart() method. That way each test will benefit from it.
(Assignee)

Comment 1

3 years ago
This failure seems to only happen for the Windows XP node in SCL3. All other Windows machines are working fine. I wonder if it is something related to this machine then. I will see if I can find a local XP machine for tests.
OS: Unspecified → Windows XP
(Assignee)

Updated

3 years ago
Whiteboard: [test-failure]
(Assignee)

Updated

3 years ago
Blocks: 1165215
(Assignee)

Comment 2

3 years ago
Created attachment 8611467 [details] [review]
skip patch for Windows XP

This is the skip patch for Windows XP only. Maybe someone will have the time to investigate. I don't have it at the moment. I will land this change once Travis CI is passing.
(Assignee)

Comment 3

3 years ago
The test has been disabled for Windows XP now:
https://github.com/mozilla/firefox-ui-tests/commit/ad6817abb55b67f106ed20cd40a00374037fde08 (master)

Backports I will do soon once all other failures are fixed, which also need a backport.
status-firefox38: --- → affected
status-firefox39: --- → affected
status-firefox40: --- → affected
status-firefox41: --- → disabled
status-firefox-esr38: --- → affected
(Assignee)

Comment 4

3 years ago
The patch as landed doesn't work because the machines report:

version: 5.1.2600
os: win
service_pack: Service Pack 3
bits: 32
processor: x86

That means os_version == "5.1" does not match. I feel this is a bug in handling OS versions in the manifest parser. It should not take the build number into account. I will check this.
(Assignee)

Comment 5

3 years ago
The problem is that we haven't had a new release of mozinfo since Oct 2013! Features like os_version are not available via the packages on PyPI. I'm going to release version 0.8 via bug 1169168 to unblock us.
(Assignee)

Updated

3 years ago
Depends on: 1169168
(Assignee)

Comment 6

3 years ago
I also pushed a version bump for mozinfo to 0.8:
https://github.com/mozilla/firefox-ui-tests/commit/6aa74d806b373ba29cee83d90e0ebe501192c48b

If that works I will have to backport both patches down to esr38.
(Assignee)

Comment 7

3 years ago
Having our tests for Nightly run in production now, I can also see those timeouts on other versions of Windows like Vista, 7, and 8. So I think we should skip it for all Windows, and check soon what's going on.
OS: Windows XP → Windows
(Assignee)

Updated

3 years ago
No longer blocks: 1165215
(Assignee)

Comment 10

3 years ago
While working on bug 1128968 I noticed that we missed to retrieve the elements for the identity popup again after the restart. That's why it was failing:

https://github.com/mozilla/firefox-ui-tests/commit/2c38349a1494aa6b43f82789785638f467ec031b#diff-c2229f1aac10fba9af79defdcaf44a78R59

I'm not sure why it's working on some platforms but not all. I would wait with further investigation until bug 1177417 has been solved.
Depends on: 1177417
(Assignee)

Updated

3 years ago
No longer depends on: 1177417
(Assignee)

Updated

3 years ago
Keywords: intermittent-failure
(Assignee)

Updated

3 years ago
Keywords: intermittent-failure
Product: Mozilla QA → Testing
(Assignee)

Comment 11

3 years ago
I did a couple of local runs for this test on Windows and I was not able to see this failure anymore. That means I would like to try to unskip this test. A simple backout will not work due to the remote->functional migration. So I will come up with a new patch.
(Assignee)

Comment 12

3 years ago
Created attachment 8705079 [details] [review]
Github PR (unskip test)
Attachment #8705079 - Flags: review?(mjzffr)
(Assignee)

Updated

3 years ago
Blocks: 1237735
Attachment #8705079 - Flags: review?(mjzffr) → review+
(Assignee)

Comment 13

3 years ago
Merged PR to mozilla-central:
https://github.com/mozilla/firefox-ui-tests/commit/ca7e96357b92678e65ac75c62fbbf400ff6c5f3b

If we don't have any failures I would like to also get it enabled for mozilla-aurora which will be the next ESR release.
status-firefox38: disabled → ---
status-firefox39: disabled → ---
status-firefox40: disabled → ---
status-firefox41: disabled → ---
status-firefox43: --- → disabled
status-firefox44: --- → disabled
status-firefox45: --- → disabled
status-firefox46: --- → fixed
Target Milestone: --- → mozilla46
(Assignee)

Updated

3 years ago
Keywords: intermittent-failure
(Assignee)

Comment 14

3 years ago
So all Windows platforms seem to work except for Win XP when we see this failure again now:

https://treeherder.mozilla.org/logviewer.html#?job_id=3035636&repo=mozilla-central

I will leave this running for the weekend and check if we constantly fail.
(Assignee)

Comment 15

3 years ago
This failure always reproduces on WinXP via the nodes mm-win-xp-32-3 and mm-win-xp-32-4. The permanent issue with OS X 10.10 is bug 1198611. It was working fine since lately. I will check both bugs in more detail tomorrow.
(Assignee)

Comment 16

3 years ago
The failure on WinXP was not visible the last two days. Maybe something has been changed here. So it's not permanent. I would leave this test enabled on mozilla-central for now and also enable it in aurora. That way we might get a better idea what's going on if it happens again.

Backported backout patch to mozilla-aurora:
https://github.com/mozilla/firefox-ui-tests/commit/e570df4679256c5f4972680937ef89f3db449ea7
status-firefox45: disabled → affected
status-firefox46: fixed → affected
(Assignee)

Updated

3 years ago
OS: Windows → Windows XP
Summary: Failure in test_ssl_status_after_restart.py because test doesn't wait after restart for session restored → [WinXP] TEST-UNEXPECTED-ERROR | test_ssl_status_after_restart.py TestSSLStatusAfterRestart.test_ssl_status_after_restart | TimeoutException: TimeoutException: Timed out after 5.1 seconds
(Assignee)

Updated

3 years ago
Whiteboard: [test-failure]
15 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-aurora: 11
* mozilla-central: 4

Platform breakdown:
* windowsxp: 15

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1167179&startday=2016-01-18&endday=2016-01-24&tree=all
(Assignee)

Comment 18

3 years ago
Nowadays this is a permanent failure on both mozilla-central and mozilla-aurora. So we should get this turned off again for WinXP only. I don't think it makes that much sense right now for investigation.
(Assignee)

Comment 19

3 years ago
Created attachment 8711633 [details] [review]
Github PR (Re-skip test but on WinXP only)

This is similar to attachment 8611467 [details] [review] but with an updated path.
Attachment #8611467 - Attachment is obsolete: true
Attachment #8711633 - Flags: review?(mjzffr)
(Assignee)

Comment 20

3 years ago
Created attachment 8711635 [details] [diff] [review]
Disable patch (v1) [mozilla-central]

Adapted patch for mozilla-central on hg.mo.
Attachment #8711635 - Flags: review?(mjzffr)
Attachment #8711633 - Flags: review?(mjzffr) → review+
Attachment #8711635 - Flags: review?(mjzffr) → review+
(Assignee)

Comment 21

3 years ago
I landed the Github patch across all affected branches:

https://github.com/mozilla/firefox-ui-tests/commit/0c64b99f0c69c216b3720887609e6047cdd5b74f (central)
https://github.com/mozilla/firefox-ui-tests/commit/e124685d218196b8673f5f03a836c5bd1af87d22 (aurora)
https://github.com/mozilla/firefox-ui-tests/commit/764b070502acfccdf110e67762aff7624a54f6da (beta)

I will also land it on inbound now. I leave the status flags for proper landing on hg.mo.
status-firefox47: --- → affected

Comment 23

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/48292bc3676d
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
status-firefox47: affected → fixed
Resolution: --- → FIXED

Updated

3 years ago
Target Milestone: mozilla46 → mozilla47
(Assignee)

Comment 24

3 years ago
Attachment 8711635 [details] [diff] is a testonly change and does not affect the product. It constantly fails for us on WinXP, so we would like to have it skipped down to the upcoming 45.0 ESR.
Whiteboard: [checkin-needed-aurora][checkin-needed-beta]
https://hg.mozilla.org/releases/mozilla-aurora/rev/560a24bbebce
 https://hg.mozilla.org/releases/mozilla-beta/rev/07bd7f4a0a1b
status-firefox45: affected → fixed
status-firefox46: affected → fixed
Whiteboard: [checkin-needed-aurora][checkin-needed-beta]
(Assignee)

Comment 26

3 years ago
Actually I will mark this bug as wontfix given that it only affects a platform which we can hopefully get rid of in the next months. That's why I don't want to put in any other work.
Resolution: FIXED → WONTFIX
You need to log in before you can comment on or make changes to this bug.