The default bug view has changed. See this FAQ.
Last Comment Bug 1050242 - Intermittent Android 4.0 debug jsreftest/reftest failures with no error summary (timeout/crash/hang?)
: Intermittent Android 4.0 debug jsreftest/reftest failures with no error summa...
Status: RESOLVED FIXED
: intermittent-failure
Product: Release Engineering
Classification: Other
Component: Buildduty (show other bugs)
: unspecified
: ARM Android
-- normal (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
: Justin Wood (:Callek)
:
Mentors:
Depends on:
Blocks: 778688
  Show dependency treegraph
 
Reported: 2014-08-07 05:27 PDT by Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley]
Modified: 2014-09-08 08:19 PDT (History)
6 users (show)
See Also:
Crash Signature:
(edit)
Machine State: ---
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-07 05:27:38 PDT
Android 4.0 Panda mozilla-inbound debug test jsreftest-1 on 2014-08-07 00:37:05 PDT for push 1f01cae60e26

slave: panda-0587

https://tbpl.mozilla.org/php/getParsedLog.php?id=45400466&tree=Mozilla-Inbound

01:02:40     INFO -  REFTEST TEST-START | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-19.js
01:02:40     INFO -  REFTEST TEST-LOAD | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-19.js | 520 / 2114 (24%)
01:02:40     INFO -  REFTEST TEST-PASS | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-19.js | count = 0; result = ""; function f() { count++; return new Array("h","e","l","l","o"); }for ( p in f() ) { result += f()[p] }; count  item 1
01:02:40     INFO -  REFTEST TEST-PASS | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-19.js | result  item 2
01:02:40     INFO -  REFTEST INFO | Loading a blank page
01:02:40     INFO -  REFTEST TEST-END | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-19.js
01:02:40     INFO -  REFTEST TEST-START | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-2.js
01:02:40     INFO -  REFTEST TEST-LOAD | http://10.26.133.22:30587/jsreftest/tests/jsreftest.html?test=ecma/Statements/12.6.3-2.js | 521 / 2114 (24%)
01:02:40     INFO -  INFO | automation.py | Application ran for: 0:11:15.595872

It looks like 12.6.3-2.js starts, but we never get a TEST-END nor timeout or anything useful.
Comment 1 User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-11 08:39:50 PDT
Android 4.0 Panda mozilla-inbound opt test plain-reftest-2 on 2014-08-11 06:23:25 PDT for push 2e0e4d841f0f

https://tbpl.mozilla.org/php/getParsedLog.php?id=45663844&full=1&branch=mozilla-inbound

06:54:21     INFO -  REFTEST TEST-START | http://10.26.133.22:30587/tests/layout/reftests/bugs/403656-4.html
06:54:21     INFO -  REFTEST TEST-LOAD | http://10.26.133.22:30587/tests/layout/reftests/bugs/403656-4.html | 1186 / 1335 (88%)
06:54:21     INFO -  REFTEST TEST-LOAD | http://10.26.133.22:30587/tests/layout/reftests/bugs/403656-4-ref.html | 1186 / 1335 (88%)
06:54:21     INFO -  INFO | automation.py | Application ran for: 0:23:09.555395
Comment 2 User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-12 09:45:52 PDT
Slightly more useful:

https://tbpl.mozilla.org/php/getParsedLog.php?id=45706249&full=1&branch=fx-team

17:08:23     INFO -  REFTEST TEST-START | http://10.26.133.22:30584/jsreftest/tests/jsreftest.html?test=js1_5/extensions/regress-322957.js
17:08:23     INFO -  REFTEST TEST-LOAD | http://10.26.133.22:30584/jsreftest/tests/jsreftest.html?test=js1_5/extensions/regress-322957.js | 138 / 2375 (5%)
17:11:24     INFO -  mozdevice ERROR | DeviceManager: pull unsuccessful: no prompt found after file data--DeviceManager may be out of sync with agent
17:11:24     INFO -  INFO | automation.py | Application ran for: 0:04:02.888589
Comment 3 User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-12 10:01:55 PDT
Android 4.0 Panda mozilla-inbound debug test plain-reftest-7 on 2014-08-12 08:46:35 PDT for push f497c1d55fd0

slave: panda-0587

https://tbpl.mozilla.org/php/getParsedLog.php?id=45759429&full=1&branch=mozilla-inbound
Comment 4 User image Ryan VanderMeulen [:RyanVM] 2014-08-15 10:46:39 PDT
Geoff, these are pretty widespread at the moment (and due to the failure mode, we just kind of wave our hands at them and retrigger). Any ideas who can help with this?

https://tbpl.mozilla.org/php/getParsedLog.php?id=46046060&full=1&branch=mozilla-central
https://tbpl.mozilla.org/php/getParsedLog.php?id=46043820&tree=Mozilla-Inbound&full=1
Comment 7 User image Geoff Brown [:gbrown] 2014-08-15 14:43:29 PDT
:snorp - Do you have time / people to look at this? Pandas are rebooting mid-test in several jobs, on both debug and opt. I'll try to have a look through more logs on Monday.
Comment 9 User image Geoff Brown [:gbrown] 2014-08-18 09:18:15 PDT
These are not specific to one particular slave, but there are several duplicates. Let's track the affected slaves.

Comment 0: panda-0587
Comment 1: panda-0587
Comment 2: panda-0584
Comment 3: panda-0587
Comment 4: panda-0587, panda-0592
Comment 5: panda-0584
Comment 6: panda-0584, panda-0584
Comment 8: panda-0501

:kmoir -- Can you think of what might be causing these Pandas to reboot during tests?
Comment 10 User image Geoff Brown [:gbrown] 2014-08-18 10:21:56 PDT
There are 3 similar failures in bug 1052523 -- all are panda-0587.
Comment 11 User image Kim Moir [:kmoir] 2014-08-18 10:30:00 PDT
Pandas 0587-592 are all on foopy 79

panda-501 is on foopy72

I can disable panda-584 and panda-587, they seem to be causing problems and their reports in slave health show a large number of retries/failures.
Comment 12 User image Kim Moir [:kmoir] 2014-08-18 10:37:59 PDT
See bug 1055125 and bug 1055117 for panda slave tracking bugs
Comment 13 User image Geoff Brown [:gbrown] 2014-08-20 07:29:52 PDT
Any new failures since those pandas were disabled? Anything more to do here?
Comment 14 User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-20 07:31:11 PDT
Not sure, since I'd virtually reached the point of generically starring any occurrences - but from memory I don't think I've seen any in the last few days. Happy to call this fixed and reopen if needs be :-)
Comment 15 User image Ed Morley (Away 22nd-27th, 29th-2nd) [:emorley] 2014-08-20 09:16:35 PDT
https://tbpl.mozilla.org/php/getParsedLog.php?id=46364097&tree=Mozilla-Inbound
panda-0592
Comment 16 User image Justin Wood (:Callek) 2014-09-08 08:19:50 PDT
No new entries since 8-20, resolving again

Note You need to log in before you can comment on or make changes to this bug.