1095514 - Intermittent testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Reporter

Description

•

10 years ago

Android 2.3 Emulator mozilla-central opt test robocop-1

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=597935&repo=mozilla-central

04:52:15 WARNING - TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T07:10:31
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=597935
repository: mozilla-central
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-281
buildname: Android 2.3 Emulator mozilla-central opt test robocop-1
revision: 17e190839086

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T07:19:54
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=fx-team&job_id=1129369
repository: fx-team
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-375
buildname: Android 2.3 Emulator fx-team opt test robocop-1
revision: 3357e9e81614

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T07:24:56
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3687530
repository: mozilla-inbound
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-1281
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 9b212f882730

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Richard Newman [:rnewman]

Updated

•

10 years ago

Assignee: nobody → bnicholson

Blocks: 1068425

Status: NEW → ASSIGNED

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T08:48:51
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3670915
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-794
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: afb918496f14

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T08:53:07
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=b2g-inbound&job_id=795844
repository: b2g-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-965
buildname: Android 2.3 Emulator b2g-inbound opt test robocop-1
revision: b9c062ed4146

Can't download from http://ftp.mozilla.org.proxxy1.srv.releng.use1.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/b2g-inbound-android/1415371245/fennec-36.0a1.en-US.android-arm.apk to /builds/slave/test/build/fennec-36.0a1.en-US.android-arm.apk!
Caught exception: timed out
Caught exception: timed out
Caught exception: timed out
TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T08:58:16
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=fx-team&job_id=1132259
repository: fx-team
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-169
buildname: Android 2.3 Emulator fx-team opt test robocop-1
revision: 4beef3d7299c

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T09:04:30
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3692460
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-1142
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 527d5142b7c5

Can't download from http://ftp.mozilla.org.proxxy1.srv.releng.usw2.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1415373282/fennec-36.0a1.en-US.android-arm.tests.zip to /builds/slave/test/build/fennec-36.0a1.en-US.android-arm.tests.zip!
Caught exception: timed out
Caught exception: timed out
Caught exception: timed out
TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Mark Finkle (:mfinkle) (use needinfo?)

Comment 8

•

10 years ago

Attached image testdistro-checktiles-failure.jpeg (obsolete) — Details

Screenshot

Mark Finkle (:mfinkle) (use needinfo?)

Comment 9

•

10 years ago

All of the current failures are Android 2.3 emulator runs.

Brian Nicholson (:bnicholson)

Assignee

Comment 10

•

10 years ago

Made a try build this morning to add a couple waitForTexts in the pin site code. Let's see if it helps: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=8b05b71c5eba

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T16:19:34
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3707026
repository: mozilla-inbound
who: wkocher[at]mozilla[dot]com
machine: tst-linux64-spot-525
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 970e11385295

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-07T17:24:43
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3708308
repository: mozilla-inbound
who: wkocher[at]mozilla[dot]com
machine: tst-linux64-spot-1284
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 67c6993d405c

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-10T01:04:23
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3727486
repository: mozilla-inbound
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-490
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 961d11a08333

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-10T05:05:37
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=608911
repository: mozilla-central
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-413
buildname: Android 2.3 Emulator mozilla-central opt test robocop-1
revision: d380166816dd

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-10T05:05:41
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=608320
repository: mozilla-central
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-010
buildname: Android 2.3 Emulator mozilla-central opt test robocop-1
revision: d380166816dd

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.AssertionFailedError: Text string: 'Pin Site' is not found!

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-10T09:02:38
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3735533
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-752
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 3dc89d160b3a

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.AssertionFailedError: Text string: 'Pin Site' is not found!

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-10T12:23:30
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3738169
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-1285
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 4d3c5b18b800

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.AssertionFailedError: Text string: 'Pin Site' is not found!

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T05:20:23
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3761022
repository: mozilla-inbound
who: tomcat[at]mozilla[dot]com
machine: tst-linux64-spot-572
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: ac0d616b8870

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T08:40:18
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3765792
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-490
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 51572056e797

Can't download from http://ftp.mozilla.org.proxxy1.srv.releng.usw2.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1415717286/fennec-36.0a1.en-US.android-arm.tests.zip to /builds/slave/test/build/fennec-36.0a1.en-US.android-arm.tests.zip!
Caught exception: timed out
Caught exception: timed out
Caught exception: timed out
Can't download from http://ftp.mozilla.org.proxxy1.srv.releng.usw2.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1415717286/fennec-36.0a1.en-US.android-arm.apk to /builds/slave/test/build/fennec-36.0a1.en-US.android-arm.apk!
Caught exception: timed out
Caught exception: timed out
Caught exception: timed out
TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.AssertionFailedError: Text string: 'Pin Site' is not found!

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T10:21:00
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3768550
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-449
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 2f642ef2068a

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Brian Nicholson (:bnicholson)

Assignee

Comment 21

•

10 years ago

Attached patch Make distribution tile tests more robust — Details — Splinter Review

After clicking Pin Site, there is a small delay before the top site item is actually pinned, and we occasionally click the tile before its pinned state has been updated. This creates a verifyPinned helper, along with some additional assertions/fixes to make the test more robust.

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=dbf5a5f3e77f

Attachment #8519196 - Attachment is obsolete: true

Attachment #8520776 - Flags: review?(rnewman)

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T11:54:55
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3773689
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-1134
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 7325d5ccfe2e

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Richard Newman [:rnewman]

Comment 23

•

10 years ago

Comment on attachment 8520776 [details] [diff] [review]
Make distribution tile tests more robust

Review of attachment 8520776 [details] [diff] [review]:
-----------------------------------------------------------------

Sounds fair to me!

Attachment #8520776 - Flags: review?(rnewman) → review+

Brian Nicholson (:bnicholson)

Assignee

Comment 24

•

10 years ago

https://hg.mozilla.org/integration/fx-team/rev/217c7f7d9f48

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T12:52:49
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-inbound&job_id=3775160
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-537
buildname: Android 2.3 Emulator mozilla-inbound opt test robocop-1
revision: 56f7242d0794

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Brian Nicholson (:bnicholson)

Assignee

Comment 26

•

10 years ago

Comment on attachment 8520776 [details] [diff] [review]
Make distribution tile tests more robust

Approval Request Comment
[Feature/regressing bug #]: Failures caused by tests added in bug 1068425.
[User impact if declined]: Intermittent test failures.
[Describe test coverage new/current, TBPL]: Failure is gone in try push (comment 21).
[Risks and why]: No risk (test-only fix).
[String/UUID change made/needed]: None.

Attachment #8520776 - Flags: approval-mozilla-aurora?

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-11T16:42:34
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=b2g-inbound&job_id=817063
repository: b2g-inbound
who: wkocher[at]mozilla[dot]com
machine: tst-linux64-spot-344
buildname: Android 2.3 Emulator b2g-inbound opt test robocop-1
revision: a926116946f8

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

seth
https://tbpl.mozilla.org/php/getParsedLog.php?id=52381854&tree=Try
Android 2.3 Emulator try opt test robocop-1 on 2014-11-11 14:38:23
revision: d7a020f1361a
slave: tst-linux64-spot-782

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)

Comment 29

•

10 years ago

https://hg.mozilla.org/mozilla-central/rev/217c7f7d9f48

Status: ASSIGNED → RESOLVED

Closed: 10 years ago

Resolution: --- → FIXED

Target Milestone: --- → Firefox 36

Ryan VanderMeulen [:RyanVM]

Comment 30

•

10 years ago

Comment on attachment 8520776 [details] [diff] [review]
Make distribution tile tests more robust

Test-only changes don't need approval. SHIP IT.

Attachment #8520776 - Flags: approval-mozilla-aurora?

Ryan VanderMeulen [:RyanVM]

Updated

•

10 years ago

status-firefox34: --- → wontfix

status-firefox35: --- → affected

status-firefox36: --- → fixed

status-firefox-esr31: --- → unaffected

Ryan VanderMeulen [:RyanVM]

Comment 31

•

10 years ago

That said, the subtests in question don't exist on Aurora?

Ryan VanderMeulen [:RyanVM]

Comment 32

•

10 years ago

Ah, bug 1068425 hasn't been uplifted yet.

Comment hidden (Legacy TBPL/Treeherder Robot)

submit_timestamp: 2014-11-12T11:09:30
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=3088378
repository: try
who: jryans[at]gmail[dot]com
machine: tst-linux64-spot-555
buildname: Android 2.3 Emulator try opt test robocop-1
revision: a430aa2f5175

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Comment hidden (Legacy TBPL/Treeherder Robot)

botond
https://tbpl.mozilla.org/php/getParsedLog.php?id=52509242&tree=Try
Android 2.3 Emulator try opt test robocop-1 on 2014-11-12 18:03:55
revision: 64cb0111f38c
slave: tst-linux64-spot-366

TEST-UNEXPECTED-FAIL | testDistribution | Exception caught - junit.framework.ComparisonFailure: expected:<......> but was:<...,"pin":true...>

Ryan VanderMeulen [:RyanVM]

Comment 35

•

10 years ago

https://hg.mozilla.org/releases/mozilla-aurora/rev/0d83fa850744

status-firefox34: wontfix → unaffected

status-firefox35: affected → fixed

Nobody; OK to take it and work on it

Updated

•

3 years ago

Product: Firefox for Android → Firefox for Android Graveyard

testdistro-checktiles-failure.jpeg 10 years ago Mark Finkle (:mfinkle) (use needinfo?) 34.16 KB, image/png		Details
Make distribution tile tests more robust 10 years ago Brian Nicholson (:bnicholson) 5.15 KB, patch	rnewman : review+	Details \| Diff \| Splinter Review