Closed Bug 796914 Opened 12 years ago Closed 11 years ago

Intermittent "talosError: 'stack found after process termination (org.mozilla.fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess)'"

Categories

(Testing :: Talos, defect)

Product:
Testing
Component:
Talos
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox17 affected, firefox18 affected, firefox19 affected)

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox17 --- affected
firefox18 --- affected
firefox19 --- affected

People

(Reporter: emorley, Unassigned)

References

Details

(Keywords: crash, intermittent-failure, Whiteboard: [red])

Android Tegra 250 mozilla-inbound talos remote-troboprovider on 2012-10-01 11:39:12 PDT for push 495a115ec2cc slave: tegra-121 https://tbpl.mozilla.org/php/getParsedLog.php?id=15710377&tree=Mozilla-Inbound { NOISE: __start_report373__end_report NOISE: __startTimestamp373__endTimestamp reconnecting socket FIRE PROC: 'am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec.tests.testBrowserProviderPerf org.mozilla.roboexample.test/org.mozilla.fennec.FennecInstrumentationTestRunner' getting files in '/mnt/sdcard/tests/profile/minidumps/' NOISE: Found crashdump: /tmp/tmpKDATmC/52ef2a3a-3e19-f0ea-06392cc9-1b297d98.dmp NOISE: __start_report373__end_report NOISE: __startTimestamp373__endTimestamp NOISE: __startBeforeLaunchTimestamp1349117633121__endBeforeLaunchTimestamp NOISE: __startAfterTerminationTimestamp1349117728362__endAfterTerminationTimestamp getting files in '/mnt/sdcard/tests/profile/minidumps/' NOISE: Found crashdump: /tmp/tmpSarrgZ/52ef2a3a-3e19-f0ea-06392cc9-1b297d98.dmp Traceback (most recent call last): File "run_tests.py", line 250, in run_tests talos_results.add(mytest.runTest(browser_config, test)) File "/builds/tegra-121/talos-data/talos/ttest.py", line 412, in runTest self.cleanupAndCheckForCrashes(browser_config, profile_dir) File "/builds/tegra-121/talos-data/talos/ttest.py", line 198, in cleanupAndCheckForCrashes Failed tprovider: Stopped Mon, 01 Oct 2012 11:56:32 raise talosError("stack found after process termination (" + cleanup_result+ ")") talosError: 'stack found after process termination (fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess)' Traceback (most recent call last): File "run_tests.py", line 298, in <module> FAIL: Busted: tprovider FAIL: stack found after process termination (fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess) main() File "run_tests.py", line 295, in main run_tests(parser) File "run_tests.py", line 259, in run_tests raise e utils.talosError: 'stack found after process termination (fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess)' program finished with exit code 1 }
Depends on: 797324
Assignee: nobody → gbrown
Depends on: 806371
Has become more frequent - please may someone take a look.
(In reply to Ed Morley [:edmorley UTC+0] from comment #64) > Has become more frequent - please may someone take a look. I am working as fast as I can! Bug 806371 will resolve this...I am closing in on a fix.
(In reply to Geoff Brown [:gbrown] from comment #66) > I am working as fast as I can! Bug 806371 will resolve this...I am closing > in on a fix. Ah sorry, missed the dependant bug - thank you :-)
Summary: Intermittent "talosError: 'stack found after process termination (fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess)'" → Intermittent "talosError: 'stack found after process termination (org.mozilla.fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAgent.killProcess)'"
The frequent failures in remote-troboprovider (rpr / testBrowserProviderPerf) seem to have ceased with the check-in for bug 785945. Of course, maybe it's just chance...
This is the #2 toporange across all platforms (even though it only occurs on Android). gbrown, will the work in bug 806371 help for all cases, or just rpr (I wasn't sure given comment 91).
(In reply to Ed Morley [:edmorley UTC+0] from comment #120) > This is the #2 toporange across all platforms (even though it only occurs on > Android). > > gbrown, will the work in bug 806371 help for all cases, or just rpr (I > wasn't sure given comment 91). The work in 806371 was specific to rpr, and that work seems irrelevant since comment 91...I'm very disappointed, and confused by the change in behavior. I am looking at the more general case now, but have little to show for it so far.
Thank you :-)
A try run with better logging: https://tbpl.mozilla.org/php/getParsedLog.php?id=16756602&tree=Try&full=1 11-05 10:03:09.742 I/GeckoDump( 1923): __end_tp_report 11-05 10:03:09.742 I/GeckoDump( 1923): __start_cc_report 11-05 10:03:09.742 I/GeckoDump( 1923): _x_x_mozilla_cycle_collect,4298 11-05 10:03:09.742 I/GeckoDump( 1923): __end_cc_report 11-05 10:03:09.742 I/GeckoDump( 1923): __startTimestamp1352109789751__endTimestamp 11-05 10:03:09.742 I/GeckoDump( 1923): 11-05 10:03:09.765 E/GeckoConsole( 1923): [JavaScript Warning: "Duplicate resource declaration for 'gre-resources' ignored." {file: "jar:jar:file:///data/app/org.mozilla.fennec-1.apk!/omni.ja!/chrome/nonlocalized.manifest" line: 8}] 11-05 10:03:09.774 E/GeckoConsole( 1923): Could not read chrome manifest 'file:///data/data/org.mozilla.fennec/chrome.manifest'. 11-05 10:03:09.795 W/SensorService( 1020): listener with binder android.os.BinderProxy@48673568, doesn't exist (sensor=tegra_accelerometer, id=1) 11-05 10:03:09.894 E/GeckoConsole( 1923): [JavaScript Error: "fallback.getPrompt is not a function" {file: "jar:jar:file:///data/app/org.mozilla.fennec-1.apk!/omni.ja!/components/PromptService.js" line: 41}] 11-05 10:03:10.172 I/SUTAgentAndroid( 1491): 10.250.48.218 : isdir /mnt/sdcard/tests 11-05 10:03:10.192 I/SUTAgentAndroid( 1491): 10.250.48.218 : cd /mnt/sdcard/tests 11-05 10:03:10.222 I/SUTAgentAndroid( 1491): 10.250.48.218 : ls 11-05 10:03:10.242 I/SUTAgentAndroid( 1491): 10.250.48.218 : pull /mnt/sdcard/tests/browser_output.txt 11-05 10:03:10.602 I/ActivityManager( 1020): Process org.mozilla.fennec (pid 1923) has died. 11-05 10:03:10.602 I/WindowManager( 1020): WIN DEATH: Window{4875d948 org.mozilla.fennec/org.mozilla.fennec.App paused=false} 11-05 10:03:10.602 I/WindowManager( 1020): WIN DEATH: Window{48763ae0 SurfaceView paused=false} 11-05 10:03:10.622 I/UsageStats( 1020): Unexpected resume of com.mozilla.SUTAgentAndroid while already resumed in org.mozilla.fennec 11-05 10:03:10.632 D/Zygote ( 939): Process 1923 terminated by signal (11) I interpret that as death-by-SIGSEGV in fennec -- but there's no stack. I wonder if: JavaScript Error: "fallback.getPrompt is not a function" is significant?
Depends on: 810471
I haven't made much progress here. Will focus on 810471 instead...
Assignee: gbrown → nobody
Whiteboard: [orange][red] → [red]
Blocks: 816584
The good: these now have stacks in the log. The bad: tbpl doesn't highlight the top frame. The ugly: they seem to all be the complex of nssCertificate_Destroy bugs that nobody is fixing.
(In reply to Phil Ringnalda (:philor) from comment #540) > The bad: tbpl doesn't highlight the top frame. Talos is using mozcrash which doesn't have the recent automationutils.py::checkForCrashes() improvements. Bug 813132 is filed to port them across.
Ignore comment 600, was bustage fixed by backout (da9cd7ac8713).
Blocks: 829371
Depends on: 829419
Summary of recent logs (since Jan 7): Comments 559, 562-584, 586, 588, 590, 591, 592, 593, 598, 599, 602, 605, 606, 607, 610, 611, 612, 613, 615-622: shutdown crash in nssCertificate_Destroy - bug 761987. Comments 561, 585, 587, 589, 594, 495, 596, 597, 603, 604, 608, 609, 614, 624: crash in nsDeleteDir - bug 829419. Comments 560, 600, 601: real comments, backout. Comment 623: unexplained crash, during test.
Resolving WFM keyword:intermittent-failure bugs last modified >3 months ago, whose whiteboard contains none of: {random,disabled,marked,fuzzy,todo,fails,failing,annotated,time-bomb,leave open} There will inevitably be some false positives; for that (and the bugspam) I apologise. Filter on orangewfm.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.