Intermittent cluster of failures on a single build: test_XHR.html | wrong blob size, test_pluginstream_asfileonly.html | content doesn't match, test_xhr2.html | 'blob' responseType failed, test_pluginstream_asfileonly.html | content doesn't match

RESOLVED FIXED

Status

()

Core
DOM
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: mbrubeck, Assigned: michal)

Tracking

({intermittent-failure})

11 Branch
x86
Linux
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox11+ fixed)

Details

(Whiteboard: [qa-])

(Reporter)

Description

5 years ago
This first occurred on a nightly Linux 32-bit PGO build.  The failures did not occur on the non-nightly build from the same changeset:

https://tbpl.mozilla.org/php/getParsedLog.php?id=8014179&tree=Firefox
Rev3 Fedora 12 mozilla-central pgo test mochitests-1/5 on 2011-12-18 04:05:10 PST for push a5e63e00db27

5292 INFO TEST-PASS | /tests/content/base/test/test_XHR.html | should not be a File
5293 ERROR TEST-UNEXPECTED-FAIL | /tests/content/base/test/test_XHR.html | wrong blob size - got 19, expected 11
5294 INFO TEST-PASS | /tests/content/base/test/test_XHR.html | responseXML should be null. - null should equal null

https://tbpl.mozilla.org/php/getParsedLog.php?id=8014023&tree=Firefox
Rev3 Fedora 12 mozilla-central pgo test mochitests-2/5 on 2011-12-18 04:05:09 PST for push a5e63e00db27

6326 INFO TEST-START | /tests/dom/plugins/test/test_pluginstream_asfileonly.html
NPP_Destroy
NPP_NewStream
NPP_StreamAsFile, file=/tmp/tmpZEdB70/Cache/2/EC/3372Dd01
NPP_DestroyStream
6327 INFO TEST-PASS | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | bad XMLHttpRequest status - 200 should equal 200
6328 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | content doesn't match - got x^ÿffffedÿffff95Mnÿffffdb@%c…ÿfffff7%1r%7ÿffffc0ÿfffff0ÿffff82vtSÿfffff4, expected Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.

Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.

Nam liber tempor cum soluta nobis eleifend option congue nihil imperdiet doming id quod mazim placerat facer possim assum. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, At accusam aliquyam diam diam dolore dolores duo eirmod eos erat, et nonumy sed tempor et et invidunt justo labore Stet clita ea et gubergren, kasd magna no rebum. sanctus sea sed takimata ut vero voluptua. est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut laboreet dolore magna aliquyam erat.

Consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.
6329 INFO TEST-PASS | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | plugin reported error - pass should equal pass
6330 INFO TEST-END | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | finished in 196ms

https://tbpl.mozilla.org/php/getParsedLog.php?id=8014032&tree=Firefox
Rev3 Fedora 12 mozilla-central pgo test mochitests-3/5 on 2011-12-18 04:05:11 PST for push a5e63e00db27

9509 INFO TEST-START | /tests/dom/workers/test/test_xhr2.html
9510 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/workers/test/test_xhr2.html | an unexpected uncaught JS exception reported through window.onerror - 'blob' responseType failed at http://mochi.test:8888/tests/dom/workers/test/xhr2_worker.js:44
9511 INFO TEST-END | /tests/dom/workers/test/test_xhr2.html | finished in 311ms

https://tbpl.mozilla.org/php/getParsedLog.php?id=8014306&tree=Firefox
Rev3 Fedora 12 mozilla-central pgo test mochitest-other on 2011-12-18 04:05:07 PST for push a5e63e00db27

721 INFO TEST-START | /tests/dom/plugins/test/test_pluginstream_asfileonly.html
NPP_Destroy
NPP_NewStream
NPP_StreamAsFile, file=/tmp/tmpi5BKwG/Cache/2/EC/3372Dd01
NPP_DestroyStream
722 INFO TEST-PASS | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | bad XMLHttpRequest status - 200 should equal 200
723 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | content doesn't match - got x^ÿffffedÿffff95Mnÿffffdb@%c…ÿfffff7%1r%7ÿffffc0ÿfffff0ÿffff82vtSÿfffff4, expected Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.

Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.

Nam liber tempor cum soluta nobis eleifend option congue nihil imperdiet doming id quod mazim placerat facer possim assum. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, At accusam aliquyam diam diam dolore dolores duo eirmod eos erat, et nonumy sed tempor et et invidunt justo labore Stet clita ea et gubergren, kasd magna no rebum. sanctus sea sed takimata ut vero voluptua. est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut laboreet dolore magna aliquyam erat.

Consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.
724 INFO TEST-PASS | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | plugin reported error - pass should equal pass
725 INFO TEST-END | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | finished in 330ms
Is it reproducible on this build?
(Reporter)

Comment 2

5 years ago
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #1)
> Is it reproducible on this build?

Good question; I'll try retriggering to find out.

The following set occurred on a push that for some reason got four sets of M1/M2/M3 tests on a single Linux debug build.  Each of the suites failed once out of its four runs.  The mochitest-other suite ran only once and did not have the failure from comment 0:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=326455c9793d

https://tbpl.mozilla.org/php/getParsedLog.php?id=8025578&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-1/5 on 2011-12-18 21:02:51 PST for push 326455c9793d
5294 ERROR TEST-UNEXPECTED-FAIL | /tests/content/base/test/test_XHR.html | wrong blob size - got 19, expected 11

https://tbpl.mozilla.org/php/getParsedLog.php?id=8025429&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-2/5 on 2011-12-18 21:02:50 PST for push 326455c9793d
6318 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/plugins/test/test_pluginstream_asfileonly.html | content doesn't match - got x^ÿffffedÿffff95Mnÿffffdb@%c…ÿfffff7%1r%7ÿffffc0ÿfffff0ÿffff82vtSÿfffff4, expected Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

https://tbpl.mozilla.org/php/getParsedLog.php?id=8025521&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-3/5 on 2011-12-18 21:02:50 PST for push 326455c9793d
9516 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/workers/test/test_xhr2.html | an unexpected uncaught JS exception reported through window.onerror - 'blob' responseType failed at http://mochi.test:8888/tests/dom/workers/test/xhr2_worker.js:44
(Reporter)

Comment 3

5 years ago
On this Linux debug build, every test suite ran twice, and M1/2/3/other each failed the second time:

https://tbpl.mozilla.org/php/getParsedLog.php?id=8021184&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-1/5 on 2011-12-18 15:10:36 PST for push 7b4007e3cafa

https://tbpl.mozilla.org/php/getParsedLog.php?id=8020994&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-2/5 on 2011-12-18 15:10:33 PST for push 7b4007e3cafa

https://tbpl.mozilla.org/php/getParsedLog.php?id=8021089&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitests-3/5 on 2011-12-18 15:10:02 PST for push 7b4007e3cafa

https://tbpl.mozilla.org/php/getParsedLog.php?id=8021758&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitest-other on 2011-12-18 15:10:03 PST for push 7b4007e3cafa
Comment hidden (Treeherder Robot)
(Reporter)

Comment 5

5 years ago
And on this Linux PGO build, M1/2/3/other each ran once and failed:

https://tbpl.mozilla.org/php/getParsedLog.php?id=8033238&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound pgo test mochitests-1/5 on 2011-12-19 07:31:16 PST for push 2172873119db

https://tbpl.mozilla.org/php/getParsedLog.php?id=8032984&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound pgo test mochitests-2/5 on 2011-12-19 07:31:32 PST for push 2172873119db

https://tbpl.mozilla.org/php/getParsedLog.php?id=8033071&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound pgo test mochitests-3/5 on 2011-12-19 07:31:21 PST for push 2172873119db

http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux-pgo/1324305602/mozilla-inbound_fedora_test_pgo-mochitest-other-build141.txt.gz
builder: mozilla-inbound_fedora_test_pgo-mochitest-other
slave: talos-r3-fed-057
starttime: 1324308683.11
results: warnings (1)
buildid: 20111219064002
builduid: fd6f7d9ad52c42189104869f464fe1d0
revision: 2172873119db
Comment hidden (Treeherder Robot)
(Reporter)

Comment 7

5 years ago
comment 4 and comment 6 were from this Linux opt (not PGO) build where M1/2/3/other each ran once and failed: https://tbpl.mozilla.org/?tree=UX&rev=02fe687f8f52
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 10

5 years ago
In the cases where I retriggered, the retriggered tests were green.

The bug also doesn't seem associated with any particular slave(s).
(Reporter)

Comment 11

5 years ago
Update: On this push, the retriggers failed too:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=2172873119db

Anyone want to download that build and do any manual testing?  I've archived it at
http://people.mozilla.com/~mbrubeck/firefox-11.0a1.en-US.linux-i686.712032.tar.bz2
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
> The following set occurred on a push that for some reason got four sets of
> M1/M2/M3 tests on a single Linux debug build.

The reason being that "is it bad builds?" was my first question, and that was the first build I found where there didn't seem to be any funkiness about multiple builds on a push or any possible confusion about whether the retriggers would run on the same exact bits. Not reproducible on a given build, but the same set of failures happens in multiple suites on a give push? Touching a flaky network is the usual reason for that failure mode.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Related with bug 648429?
It seems that cacheAsFile may not be honored if the response is already cached.
(In reply to Masatoshi Kimura [:emk] from comment #38)
> It seems that cacheAsFile may not be honored if the response is already
> cached.
Bug 661115?

Comment 40

5 years ago
If this is a symptom of bug #661115 I would expect it to be permanent, not orange.

Note that #bug 648429 do not compress requests with STORE_ON_DISK_AS_FILE.

(In reply to Matt Brubeck (:mbrubeck) from comment #3)
> and M1/2/3/other each
> failed the second time:

Do the test clear the cache initially? If something is left in the cache from a previous run of the same test or some other test, the compression introduced in #bug 648429 may cause confusion. I.e. if the first request is marked STORE_ON_DISK_AS_FILE the cache-entry will *not* be compressed. If a subsequent request loads the same resource without STORE_ON_DISK_AS_FILE, it may find the cached (uncompressed) entry but expect it to be smaller.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
That set was talos-r3-fed-073, talos-r3-fed-073, talos-r3-fed-069 and talos-r3-fed-065.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
And those two were talos-r3-fed-004 and talos-r3-fed-050
Comment hidden (Treeherder Robot)
 talos-r3-fed-061
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
And those last two are the first time I've hit it while retriggering on a job that had failed the first time, on talos-r3-fed-046 and talos-r3-fed-054, neither of which were involved in the first round of failure there.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I started getting suspicious about the way that all of that explosion of retriggerable failure on inbound was on builds done by linux-ix-slave17, while in between builds done by linux-any-other-slave weren't failing, so catlee clobbered it. Of course it hasn't done a single build since, having taken up a life of building Android, where nobody even notices if none of your builds work right.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
And yes, now that you mention it, those failures *were* on the clobber build by linux-ix-slave17.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
That set, however, were on a build by linux-ix-slave33, and yesterday's debug were linux-ix-slave35.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
https://tbpl.mozilla.org/php/getParsedLog.php?id=8290264&tree=Mozilla-Inbound
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Josh, this is new and disturbing, it appears to indicate an error in the networking or cache system. Can you find the right person to investigate and fix this?

That the plugin "asfileonly" streams use the on-disk cache file and pin it, so that's why I expect something funky is going on with the cache.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Hmm, cache, you say? Looking like compressed data, you say? Be interesting to see whether this still happens in the post bug 715198 world.
Depends on: 715198
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)