Closed Bug 1309088 Opened 8 years ago Closed 6 years ago

Intermittent remoteautomation.py | application crashed [@ libc.so + 0x1c3dc]

Categories

(Firefox for Android Graveyard :: Testing, defect, P3)

Product:
Firefox for Android Graveyard
Component:
Testing
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1426822

People

(Reporter: intermittent-bug-filer, Assigned: gbrown)

References

Details

(Keywords: bulk-close-intermittents, intermittent-failure) 

The crash was induced by the harness:

[task 2016-10-11T01:30:01.513217Z] 01:30:01     INFO -  mozdevice ERROR | Timeout exceeded for shell call 'adb shell dumpsys window windows; echo $?'
[task 2016-10-11T01:30:31.568908Z] 01:30:31     INFO -  mozdevice ERROR | Timeout exceeded for _runCmd call 'adb shell ls -a /storage/sdcard/tests/reftest/reftest.log'
[task 2016-10-11T01:30:40.009505Z] 01:30:40     INFO -  org.mozilla.fennec still alive after SIGABRT: waiting...

In the logcat, I see the process started at 10-10 18:28:41.741 and still starting up at 10-10 18:30:39.772, when it is killed.

The mozharness script was still able to communicate with the emulator and adb at shutdown time.

Did the adb timeouts cause the abort? What caused the adb timeouts?
Priority: -- → P3
Depends on: 1319196
Recent Android mochitest-32 failures reported here will be handled in bug 1345350.
Status: NEW → RESOLVED
Closed: 7 years ago
Keywords: bulk-close-intermittents
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 7 years ago → 6 years ago
Resolution: --- → INCOMPLETE
Recent failures appear to be no output time-outs; they are being mis-categorized because of bug 1464866.
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → INCOMPLETE
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=191485056&repo=autoland&lineNumber=1659

[task 2018-08-02T00:07:41.481Z] 00:07:41     INFO -  171 INFO Error: Unable to restore focus, expect failures and timeouts.
[task 2018-08-02T00:07:41.481Z] 00:07:41     INFO -  172 INFO TEST-START | dom/security/test/general/test_same_site_cookies_toplevel_set_cookie.html
[task 2018-08-02T00:08:02.125Z] 00:08:02     INFO -  173 INFO TEST-OK | dom/security/test/general/test_same_site_cookies_toplevel_set_cookie.html | took 23352ms
[task 2018-08-02T00:08:12.439Z] 00:08:12     INFO -  174 INFO Error: Unable to restore focus, expect failures and timeouts.
[task 2018-08-02T00:08:12.440Z] 00:08:12     INFO -  175 INFO TEST-START | Shutdown
[task 2018-08-02T00:08:12.440Z] 00:08:12     INFO -  176 INFO Passed:  155
[task 2018-08-02T00:08:12.440Z] 00:08:12     INFO -  177 INFO Failed:  0
[task 2018-08-02T00:08:12.440Z] 00:08:12     INFO -  178 INFO Todo:    0
[task 2018-08-02T00:08:12.440Z] 00:08:12     INFO -  179 INFO Mode:    non-e10s
[task 2018-08-02T00:08:12.441Z] 00:08:12     INFO -  180 INFO Slowest: 68195ms - /tests/dom/security/test/general/test_same_site_cookies_toplevel_nav.html
[task 2018-08-02T00:08:12.441Z] 00:08:12     INFO -  181 INFO SimpleTest FINISHED
[task 2018-08-02T00:08:14.575Z] 00:08:14     INFO -  INFO | automation.py | Application ran for: 0:06:39.067336
[task 2018-08-02T00:08:14.576Z] 00:08:14     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmpBdInGLpidlog
[task 2018-08-02T00:08:15.200Z] 00:08:15     INFO -  /data/tombstones does not exist; tombstone check skipped
[task 2018-08-02T00:08:16.021Z] 00:08:16     INFO -  Stopping web server
[task 2018-08-02T00:08:16.026Z] 00:08:16     INFO -  Stopping web socket server
[task 2018-08-02T00:08:16.046Z] 00:08:16     INFO -  Stopping ssltunnel
[task 2018-08-02T00:08:16.067Z] 00:08:16     INFO -  leakcheck | refcount logging is off, so leaks can't be detected!
[task 2018-08-02T00:08:16.067Z] 00:08:16     INFO -  runtests.py | Running tests: end.
[task 2018-08-02T00:08:16.901Z] 00:08:16     INFO -  Buffered messages finished
[task 2018-08-02T00:08:16.902Z] 00:08:16     INFO -  Running manifest: dom/security/test/mixedcontentblocker/mochitest.ini
[task 2018-08-02T00:08:16.903Z] 00:08:16     INFO -  The following extra prefs will be set:
[task 2018-08-02T00:08:16.903Z] 00:08:16     INFO -    security.mixed_content.upgrade_display_content=false
[task 2018-08-02T00:08:18.260Z] 00:08:18     INFO -  runtests.py | Failed to copy /builds/worker/workspace/build/tests/mochitest/hyphenation to profile
[task 2018-08-02T00:08:18.477Z] 00:08:18     INFO -  pk12util: PKCS12 IMPORT SUCCESSFUL
[task 2018-08-02T00:08:18.582Z] 00:08:18     INFO -  MochitestServer : launching [u'/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/xpcshell', '-g', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64', '-f', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpCR3X_w.mozrunner'; const _SERVER_PORT = '8854'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/builds/worker/workspace/build/tests/mochitest/server.js']
[task 2018-08-02T00:08:18.582Z] 00:08:18     INFO -  runtests.py | Server pid: 2291
[task 2018-08-02T00:08:18.587Z] 00:08:18     INFO -  runtests.py | Websocket server pid: 2294
[task 2018-08-02T00:08:18.595Z] 00:08:18     INFO -  runtests.py | SSL tunnel pid: 2297
[task 2018-08-02T00:08:22.136Z] 00:08:22     INFO -  runtests.py | Running with e10s: False
[task 2018-08-02T00:08:22.137Z] 00:08:22     INFO -  runtests.py | Running tests: start.
[task 2018-08-02T00:08:22.343Z] 00:08:22     INFO -  adb launch_application: am start -W -n org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es env9 MOZ_CRASHREPORTER_NO_REPORT=1 --es env8 R_LOG_DESTINATION=stderr --es args '-no-remote -profile /sdcard/tests/profile//' --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_LOG_FILE=/sdcard/tests/mozlog/moz.log --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env13 MOZ_HIDE_RESULTS_TABLE=1 --es env12 R_LOG_LEVEL=6 --es env11 MOZ_PROCESS_LOG=/tmp/tmpW_EzNBpidlog --es env10 NO_EM_RESTART=1 -d 'http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests'
[task 2018-08-02T00:08:31.669Z] 00:08:31     INFO -  INFO | automation.py | Application pid: 2086
[task 2018-08-02T00:15:13.245Z] 00:15:13     INFO -  Browser unexpectedly found running. Killing...
[task 2018-08-02T00:15:13.245Z] 00:15:13     INFO -  TEST-INFO | started process screentopng
[task 2018-08-02T00:15:13.582Z] 00:15:13     INFO -  TEST-INFO | screentopng: exit 0
[task 2018-08-02T00:15:28.031Z] 00:15:28  WARNING -  TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2018-08-02T00:15:28.031Z] 00:15:28     INFO -  INFO | automation.py | Application ran for: 0:07:05.893775
[task 2018-08-02T00:15:28.031Z] 00:15:28     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmpW_EzNBpidlog
[task 2018-08-02T00:15:28.344Z] 00:15:28     INFO -  Contents of /data/anr/traces.txt:
[task 2018-08-02T00:15:28.345Z] 00:15:28     INFO -  ----- pid 2086 at 2018-08-01 17:15:13 -----
[task 2018-08-02T00:15:28.345Z] 00:15:28     INFO -  Cmd line: org.mozilla.fennec_aurora
[task 2018-08-02T00:15:28.345Z] 00:15:28     INFO -  JNI: CheckJNI is on; workarounds are off; pins=0; globals=266
[task 2018-08-02T00:15:28.345Z] 00:15:28     INFO -  DALVIK THREADS:
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Comment 22 is "remoteautomation.py | application timed out after 370 seconds with no output": bug 1426822.

I think it would be better to just report this in the existing bug. Can we re-close this?
Flags: needinfo?(apavel)
Geoff, hello, after the timeout there is this: 

[task 2018-08-02T00:15:44.793Z] 00:15:44  WARNING -  PROCESS-CRASH | remoteautomation.py | application crashed [@ libc.so + 0x1c3dc]
[task 2018-08-02T00:15:44.795Z] 00:15:44     INFO -  Crash dump filename: /tmp/tmpmG6n5j/27c13dcb-ff98-2c7b-d8cd-9d4c94874966.dmp
[task 2018-08-02T00:15:44.795Z] 00:15:44     INFO -  Operating system: Android
[task 2018-08-02T00:15:44.795Z] 00:15:44     INFO -                    0.0.0 Linux 2.6.29-gea477bb #1 Wed Sep 26 11:04:45 PDT 2012 armv7l
[task 2018-08-02T00:15:44.795Z] 00:15:44     INFO -  CPU: arm
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -       ARMv7 ARM Cortex-A8 features: swp,half,thumb,fastmult,vfpv2,edsp,neon,vfpv3
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -       1 CPU
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -  GPU: UNKNOWN
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -  Crash reason:  SIGABRT
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -  Crash address: 0x967
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -  Process uptime: not available
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -  Thread 0 (crashed)
[task 2018-08-02T00:15:44.796Z] 00:15:44     INFO -   0  libc.so + 0x1c3dc
[task 2018-08-02T00:15:44.797Z] 00:15:44     INFO -       r0 = 0xfffffffc    r1 = 0xbe89b4a8    r2 = 0x00000010    r3 = 0xffffffff
[task 2018-08-02T00:15:44.797Z] 00:15:44     INFO -       r4 = 0x2a041ee8    r5 = 0x00000000    r6 = 0x2a041efc    r7 = 0x000000fc
[task 2018-08-02T00:15:44.798Z] 00:15:44     INFO -       r8 = 0x2a041f30    r9 = 0x00000014   r10 = 0x2a00d0a0   r12 = 0xbe89b4a8
[task 2018-08-02T00:15:44.798Z] 00:15:44     INFO -       fp = 0xbe89b61c    sp = 0xbe89b488    lr = 0x400b1fb5    pc = 0x400433dc
[task 2018-08-02T00:15:44.798Z] 00:15:44     INFO -      Found by: given as instruction pointer in context
[task 2018-08-02T00:15:44.799Z] 00:15:44     INFO -   1  libutils.so + 0x1785c
[task 2018-08-02T00:15:44.799Z] 00:15:44     INFO -       sp = 0xbe89b4a0    pc = 0x400b485e
[task 2018-08-02T00:15:44.799Z] 00:15:44     INFO -      Found by: stack scanning
[task 2018-08-02T00:15:44.800Z] 00:15:44     INFO -   2  libbinder.so + 0x1bc2d
[task 2018-08-02T00:15:44.800Z] 00:15:44     INFO -       sp = 0xbe89b4d8    pc = 0x400f7c2f
[task 2018-08-02T00:15:44.801Z] 00:15:44     INFO -      Found by: stack scanning

Still does not match to this bug? I see in orangefactor, other classifications have timeout before this crash and were marked against this bug.
Flags: needinfo?(apavel)
(In reply to Andreea Pavel [:apavel] from comment #24)
> Geoff, hello, after the timeout there is this: 
> 
> [task 2018-08-02T00:15:44.793Z] 00:15:44  WARNING -  PROCESS-CRASH |
> remoteautomation.py | application crashed [@ libc.so + 0x1c3dc]

> Still does not match to this bug? I see in orangefactor, other
> classifications have timeout before this crash and were marked against this
> bug.

I agree that this failure matches the title of this bug, and I see that other test failures like it have been reported in this bug in the past. However, the main cause of this crash is the time out that was reported earlier in the log:

  "remoteautomation.py | application timed out after 370 seconds with no output"

I think there is no point in reporting the same failure twice, in two separate bugs.
Assignee: nobody → gbrown
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → DUPLICATE
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.