1142344 - Intermittent test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised

Status: RESOLVED FIXED

(Testing :: Marionette Client and Harness, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

14:45:52 INFO - TEST-START | test_profile_management.py TestLog.test_can_restart_the_browser
14:45:52 INFO - ERROR:root:[Errno 10054] An existing connection was forcibly closed by the remote host
14:46:16 INFO - TEST-PASS | test_profile_management.py TestLog.test_can_restart_the_browser | took 24419ms
14:46:16 INFO - TEST-START | test_profile_management.py TestLog.test_change_preset
14:46:35 INFO - TEST-PASS | test_profile_management.py TestLog.test_change_preset | took 18590ms
14:46:35 INFO - TEST-START | test_profile_management.py TestLog.test_clean_profile
14:46:53 INFO - TEST-PASS | test_profile_management.py TestLog.test_clean_profile | took 18320ms
14:46:53 INFO - TEST-START | test_profile_management.py TestLog.test_in_app_restart_the_browser
14:47:09 ERROR - TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
14:47:09 INFO - Traceback (most recent call last):
14:47:09 INFO - File "C:\slave\test\build\venv\lib\site-packages\marionette\marionette_test.py", line 296, in run
14:47:09 INFO - testMethod()
14:47:09 INFO - File "C:\slave\test\build\tests\marionette\tests\testing\marionette\client\marionette\tests\unit\test_profile_management.py", line 63, in test_in_app_restart_the_browser
14:47:09 INFO - "return SpecialPowers.getBoolPref('marionette.test.restart');")
14:47:09 INFO - TEST-INFO took 15447ms
14:47:09 INFO - TEST-START | test_profile_management.py TestLog.test_preferences_are_set
14:47:09 INFO - TEST-PASS | test_profile_management.py TestLog.test_preferences_are_set | took 48ms
14:47:09 INFO - TEST-START | test_set_window_size.py TestSetWindowSize.test_that_we_can_get_and_set_window_size

Comment 1

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7498506
repository: mozilla-inbound
start_time: 2015-03-11T14:39:20
who: wkocher[at]mozilla[dot]com
machine: t-w864-ix-087
buildname: Windows 8 64-bit mozilla-inbound pgo test marionette
revision: 6039462e84b0

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50690)
Return code: 10

Comment 2

[Chris Manchester (limited bugmail, email directly)]

I gave this some retriggers. It's still sort of a bad test, maybe I should just kill that assertion.

Comment 3

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7505933
repository: mozilla-inbound
start_time: 2015-03-11T18:11:53
who: wkocher[at]mozilla[dot]com
machine: t-w864-ix-031
buildname: Windows 8 64-bit mozilla-inbound pgo test marionette
revision: 6039462e84b0

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50695)
Return code: 10

Comment 4

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7505934
repository: mozilla-inbound
start_time: 2015-03-11T18:12:18
who: wkocher[at]mozilla[dot]com
machine: t-w864-ix-162
buildname: Windows 8 64-bit mozilla-inbound pgo test marionette
revision: 6039462e84b0

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50694)
Return code: 10

Comment 5

[Chris Manchester (limited bugmail, email directly)]

2 out of five is quite bad, I think that assertion is just tickling a race, I'll get rid of it.

Comment 6

[Chris Manchester (limited bugmail, email directly)]

Attachment: MozReview Request: bz://1142344/chmanchester

/r/5227 - Bug 1142344 - Remove a racy assertion added in marionette test for frequently failing on a pgo build.; r=jgriffin

Pull down this commit:

hg pull review -r 3e3f08e01cbd838771edacd24f20ac6471092c95

Comment 7

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=1155334
repository: mozilla-central
start_time: 2015-03-12T05:09:21
who: tomcat[at]mozilla[dot]com
machine: t-w864-ix-053
buildname: Windows 8 64-bit mozilla-central pgo test marionette
revision: b6329532e4e9

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50696)
Return code: 10

Comment 8

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7529702
repository: mozilla-inbound
start_time: 2015-03-12T05:51:37
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-121
buildname: Windows 8 64-bit mozilla-inbound pgo test marionette
revision: ddfcc47b47fc

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50705)
Return code: 10

Comment 9

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2246988
repository: fx-team
start_time: 2015-03-12T08:15:24
who: tomcat[at]mozilla[dot]com
machine: t-w864-ix-018
buildname: Windows 8 64-bit fx-team opt test marionette
revision: edc1012c711b

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Return code: 10

Comment 10

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8576357 [details]
MozReview Request: bz://1142344/chmanchester

https://reviewboard.mozilla.org/r/5225/#review4259

Do we undersrtand why this is racy?  I'm guessing restart() can sometimes do the wrong thing, but it's not clear.

Comment 11

[Chris Manchester (limited bugmail, email directly)]

https://reviewboard.mozilla.org/r/5225/#review4261

Only a theory, but restart kills the browser in a way that means user prefs aren't persisted before shutdown, which is what this test is trying to verify. From what I've seen it does this when the browser is in _some stage_ of persisting the prefs, so before bug 1141679 that meant the pref service knew about the pref but the value wasn't what was set right before shutdown, and after it meant that the pref service didn't know about the pref at all (most of the time).

The "blow away a user pref" behavior isn't really desired in the first place, so I don't think this assertion is adding much value.

Comment 12

[Chris Manchester (limited bugmail, email directly)]

checkin-needed without try because this only deletes a flaky assertion in a test.

Comment 13

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7542318
repository: mozilla-inbound
start_time: 2015-03-12T10:29:53
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-020
buildname: Windows 8 64-bit mozilla-inbound opt test marionette
revision: b09215fcb02f

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50707)
Return code: 10

Comment 14

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2251396
repository: fx-team
start_time: 2015-03-12T12:25:11
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-135
buildname: Windows 8 64-bit fx-team pgo test marionette
revision: 622fb308750a

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Return code: 10

Comment 15

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2255373
repository: fx-team
start_time: 2015-03-12T15:54:06
who: wkocher[at]mozilla[dot]com
machine: t-w864-ix-045
buildname: Windows 8 64-bit fx-team pgo test marionette
revision: 72bf69607e72

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50706)
Return code: 10

Comment 16

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7562310
repository: mozilla-inbound
start_time: 2015-03-12T18:13:58
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-013
buildname: Windows 8 64-bit mozilla-inbound pgo test marionette
revision: 43eb116a884d

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50707)
Return code: 10

Comment 17

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2257533
repository: fx-team
start_time: 2015-03-12T18:29:31
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-052
buildname: Windows 8 64-bit fx-team pgo test marionette
revision: 5a79c295029a

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50697)
Return code: 10

Comment 18

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=7570801
repository: mozilla-inbound
start_time: 2015-03-12T22:13:17
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-136
buildname: Windows 7 32-bit mozilla-inbound pgo test marionette
revision: 91abcf24159b

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 51258)
Return code: 10

Comment 19

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2260522
repository: fx-team
start_time: 2015-03-13T04:09:44
who: tomcat[at]mozilla[dot]com
machine: t-w864-ix-122
buildname: Windows 8 64-bit fx-team pgo test marionette
revision: f3fd3b32f8a4

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Exception happened during processing of request from ('127.0.0.1', 50706)
Return code: 10

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b068aca286b5

Comment 21

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=b2g-inbound&job_id=1507702
repository: b2g-inbound
start_time: 2015-03-13T06:36:15
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-037
buildname: Windows 8 64-bit b2g-inbound pgo test marionette
revision: d0c157139a7e

TEST-UNEXPECTED-FAIL | test_profile_management.py TestLog.test_in_app_restart_the_browser | AssertionError: JavascriptException not raised
Return code: 10

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b068aca286b5

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We want to uplift the patch on bug 1141679, so uplifting this patch would also be necessary. Setting ni? for David, who is most likely going to do it.

Comment 25

[David Burns :automatedtester]

https://hg.mozilla.org/releases/mozilla-aurora/rev/919d851ad569

Comment 27

[Chris Manchester (limited bugmail, email directly)]

Attachment: MozReview Request: Bug 1142344 - Remove a racy assertion added in marionette test for frequently failing on a pgo build.; r=jgriffin

Comment 28

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.