Closed Bug 764147 Opened 13 years ago Closed 13 years ago

SUT agent: near 100% cpu during StartPrg2

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Platform:
x86
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla16

People

(Reporter: gbrown, Assigned: gbrown)

Details

Attachments

(1 file, 1 obsolete file)

sleep in between checks for output in RedirOutputThread
3.38 KB, patch
gbrown
: review+
Details | Diff | Splinter Review
Running robocop tests locally on a Galaxy S, if I run "top" on the device, I see cpu use for sutagent go to and remain at nearly 100% for the duration of each test. adb logcat shows: D/dalvikvm( 4765): GC_FOR_MALLOC freed 13086 objects / 524264 bytes in 22ms D/dalvikvm( 4765): GC_FOR_MALLOC freed 13086 objects / 524256 bytes in 23ms D/SUT ( 4765): StartPrg2 waited 10s for am D/dalvikvm( 4765): GC_FOR_MALLOC freed 13085 objects / 524304 bytes in 22ms D/dalvikvm( 4765): GC_FOR_MALLOC freed 13086 objects / 524256 bytes in 22ms D/dalvikvm( 4765): GC_FOR_MALLOC freed 13086 objects / 524264 bytes in 22ms D/dalvikvm( 4765): GC_FOR_MALLOC freed 13086 objects / 524256 bytes in 22ms D/dalvikvm( 4765): GC_FOR_MALLOC freed 13084 objects / 524200 bytes in 22ms I suspect the issue is a busy wait in RedirOutputThread.run(). I have no evidence that this is causing any particular problem, but I am concerned that sut may be starving Fennec, or at least running Fennec in an unduly and unnaturally harsh environment.
while it is in the 100% duration, does SUTAgent respond over telnet at all? I have seen an exhorbitant amount of nagios "tegra agent check" failures, more than I had a few months ago. sometimes it fails with connection refused, sometimes with socket timeout. Nagios checks it via a simple |check_command check_tcp!20701| This nagios alert/error would last >2 hours in general when it happens, and *sometimes* clear itself between 2.5 and 6 *hours* after it alerts. In [almost] all cases a manual PDU reboot has gotten it working again relatively fast. --- I can't speak to the code issue involved though.
Possibly helpful, we have a tegra that *right now* is not active with jobs, but I can still telnet in... top showed SUTAgent as 50% CPU (tegras are dual-core) so I did a top -t $> exec top -n 1 -t User 68%, System 10%, IOW 0%, IRQ 0% User 420 + Nice 0 + Sys 65 + Idle 126 + IOW 0 + IRQ 1 + SIRQ 1 = 613 PID TID CPU% S VSS RSS PCY UID Thread Proc 1404 1783 39% R 138052K 22788K fg app_31 Thread-24 com.mozilla.SUTAgentAndroid 1404 1837 38% R 138052K 22788K fg app_31 Thread-31 com.mozilla.SUTAgentAndroid 1836 1836 0% R 956K 476K fg app_31 top top 1782 1782 0% S 872K 392K fg app_31 top top 1404 1406 0% R 138052K 22788K fg app_31 HeapWorker com.mozilla.SUTAgentAndroid 1017 1033 0% S 187012K 39608K fg system er.ServerThread system_server 568 568 0% S 0K 0K fg root tegra_touch_thr 1404 1409 0% S 138052K 22788K fg app_31 Compiler com.mozilla.SUTAgentAndroid 18 18 0% S 0K 0K fg root suspend 42 42 0% S 0K 0K fg root NvOsKernelThrea 56 56 0% S 0K 0K unk root irq/136-dma_cha 57 57 0% S 0K 0K unk root irq/137-dma_cha 58 58 0% S 0K 0K unk root irq/138-dma_cha 59 59 0% S 0K 0K unk root irq/139-dma_cha 60 60 0% S 0K 0K unk root irq/140-dma_cha 61 61 0% S 0K 0K unk root irq/141-dma_cha 62 62 0% S 0K 0K unk root irq/142-dma_cha 63 63 0% S 0K 0K unk root irq/143-dma_cha 64 64 0% S 0K 0K unk root irq/144-dma_cha 65 65 0% S 0K 0K unk root irq/145-dma_cha 66 66 0% S 0K 0K unk root irq/146-dma_cha 67 67 0% S 0K 0K unk root irq/147-dma_cha 226 226 0% S 0K 0K fg root sync_supers 228 228 0% S 0K 0K fg root bdi-default 230 230 0% S 0K 0K fg root kblockd/0 238 238 0% S 0K 0K fg root ksuspend_usbd 242 242 0% S 0K 0K fg root khubd 245 245 0% S 0K 0K fg root kseriod 249 249 0% S 0K 0K fg root kmmcd 255 255 0% S 0K 0K fg root bluetooth 275 275 0% S 0K 0K unk root cpufreq-dvfsd 288 288 0% S 0K 0K fg root NvOsKernelThrea 289 289 0% S 0K 0K fg root NvOsKernelThrea 298 298 0% S 0K 0K fg root khungtaskd 299 299 0% S 0K 0K fg root kswapd0 348 348 0% S 0K 0K fg root aio/0 358 358 0% S 0K 0K fg root crypto/0 464 464 0% S 0K 0K fg root tegra_hsuart_rx 466 466 0% S 0K 0K fg root tegra_hsuart_rx 468 468 0% S 0K 0K fg root tegra_hsuart_rx 470 470 0% S 0K 0K fg root tegra_hsuart_rx 495 495 0% S 0K 0K fg root mtdblockd 520 520 0% S 0K 0K fg root tegra_spi.0 547 547 0% S 0K 0K fg root usb_mass_storag 560 560 0% S 0K 0K unk root NvOsIrqKern0192 561 561 0% S 0K 0K fg root nvec_keyboard_t 567 567 0% S 0K 0K unk root NvOsIrqKern0218 580 580 0% S 0K 0K fg root intr_work 594 594 0% S 0K 0K fg root NvOsKernelThrea 595 595 0% S 0K 0K fg root NvOsKernelThrea 596 596 0% S 0K 0K unk root NvOsIrqKern0371 602 602 0% S 0K 0K fg root kstriped 622 622 0% S 0K 0K fg root usbhid_resumer 625 625 0% S 0K 0K fg root binder 642 642 0% S 0K 0K fg root krfcommd 648 648 0% S 0K 0K fg root mmcqd 932 932 0% S 740K 256K fg system servicemanager /system/bin/servicemanager 933 933 0% S 3684K 564K fg root vold /system/bin/vold 933 956 0% S 3684K 564K fg root vold /system/bin/vold 933 976 0% S 3684K 564K fg root vold /system/bin/vold 935 935 0% S 3668K 544K fg root netd /system/bin/netd 935 962 0% S 3668K 544K fg root netd /system/bin/netd 935 963 0% S 3668K 544K fg root netd /system/bin/netd 936 936 0% S 604K 256K fg root debuggerd /system/bin/debuggerd 937 937 0% S 4300K 688K fg radio rild /system/bin/rild 937 983 0% S 4300K 688K fg radio rild /system/bin/rild 937 984 0% S 4300K 688K fg radio rild /system/bin/rild 938 938 0% S 90924K 26956K fg root zygote zygote 939 939 0% S 33012K 4420K fg media mediaserver /system/bin/mediaserver 939 1008 0% S 33012K 4420K fg media mediaserver /system/bin/mediaserver 939 1009 0% S 33012K 4420K fg media mediaserver /system/bin/mediaserver 939 1010 0% S 33012K 4420K fg media mediaserver /system/bin/mediaserver 939 1011 0% S 33012K 4420K fg media AudioCommandThr /system/bin/mediaserver 939 1012 0% S 33012K 4420K fg media ApmCommandThrea /system/bin/mediaserver 939 1013 0% S 33012K 4420K fg media mediaserver /system/bin/mediaserver 939 1014 0% S 33012K 4420K fg media Playback Thread /system/bin/mediaserver 939 1016 0% S 33012K 4420K fg media Binder Thread # /system/bin/mediaserver 939 1161 0% S 33012K 4420K fg media Binder Thread # /system/bin/mediaserver 941 941 0% S 1216K 720K fg bluetoot dbus-daemon /system/bin/dbus-daemon 943 943 0% S 744K 312K fg root installd /system/bin/installd 944 944 0% S 1560K 396K fg keystore keystore /system/bin/keystore 946 946 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 993 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 994 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 995 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 996 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 997 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 998 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 999 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1000 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1001 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1002 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1003 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1004 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1005 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1006 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1007 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1015 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1029 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 1 1 0% S 236K 224K fg root init /init 946 1128 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1130 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1136 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 946 1152 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 948 948 0% S 4240K 672K fg root abtfilt /system/bin/abtfilt 948 977 0% S 4240K 672K fg root abtfilt /system/bin/abtfilt 948 978 0% S 4240K 672K fg root abtfilt /system/bin/abtfilt 948 981 0% S 4240K 672K fg root abtfilt /system/bin/abtfilt 950 950 0% S 3296K 168K fg root adbd /sbin/adbd 950 951 0% S 3296K 168K fg root adbd /sbin/adbd 950 952 0% S 3296K 168K fg root adbd /sbin/adbd 950 953 0% S 3296K 168K fg root adbd /sbin/adbd 955 955 0% S 0K 0K unk root NvOsIrqKern0302 1017 1017 0% S 187012K 39608K fg system system_server system_server 1017 1018 0% S 187012K 39608K fg system HeapWorker system_server 1017 1019 0% S 187012K 39608K fg system Signal Catcher system_server 1017 1020 0% S 187012K 39608K fg system JDWP system_server 1017 1021 0% S 187012K 39608K fg system Compiler system_server 1017 1022 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1023 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1024 0% S 187012K 39608K fg system SurfaceFlinger system_server 1017 1025 0% S 187012K 39608K fg system DisplayEventThr system_server 1017 1028 0% S 187012K 39608K fg system DisplayEventThr system_server 1017 1039 0% S 187012K 39608K fg system ActivityManager system_server 1017 1043 0% S 187012K 39608K fg system ProcessStats system_server 1017 1044 0% S 187012K 39608K bg system PackageManager system_server 1017 1045 0% S 187012K 39608K fg system FileObserver system_server 1017 1121 0% S 187012K 39608K fg system AccountManagerS system_server 1017 1126 0% S 187012K 39608K bg system SyncHandlerThre system_server 1017 1127 0% S 187012K 39608K fg system UEventObserver system_server 1017 1129 0% S 187012K 39608K fg system PowerManagerSer system_server 1017 1131 0% S 187012K 39608K fg system AlarmManager system_server 1017 1135 0% S 187012K 39608K fg system WindowManager system_server 1017 1137 0% S 187012K 39608K fg system InputDeviceRead system_server 1017 1138 0% S 187012K 39608K fg system WindowManagerPo system_server 1017 1139 0% S 187012K 39608K fg system InputDispatcher system_server 1017 1140 0% S 187012K 39608K fg system DaemonConnector system_server 1017 1142 0% S 187012K 39608K fg system ConnectivityThr system_server 1017 1146 0% S 187012K 39608K fg system WifiService system_server 1017 1147 0% S 187012K 39608K fg system Tethering system_server 1017 1148 0% S 187012K 39608K fg system MountService system_server 1017 1149 0% S 187012K 39608K fg system DaemonConnector system_server 1017 1151 0% S 187012K 39608K fg system AudioService system_server 1017 1156 0% S 187012K 39608K fg system SoundPool system_server 1017 1157 0% S 187012K 39608K fg system SoundPoolThread system_server 1017 1159 0% S 187012K 39608K bg system backup system_server 1017 1194 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1195 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1196 0% S 187012K 39608K fg system watchdog system_server 1017 1214 0% S 187012K 39608K bg system LocationManager system_server 1017 1215 0% S 187012K 39608K fg system ThrottleService system_server 1017 1219 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1241 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1278 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1523 0% S 187012K 39608K fg system Binder Thread # system_server 1017 1688 0% S 187012K 39608K fg system WifiMonitor system_server 1017 1690 0% S 187012K 39608K fg system WifiWatchdogThr system_server 1017 1695 0% S 187012K 39608K fg system DHCP Handler Th system_server 1206 1206 0% S 120492K 21768K fg app_18 putmethod.latin com.android.inputmethod.latin 1206 1208 0% S 120492K 21768K fg app_18 HeapWorker com.android.inputmethod.latin 1206 1209 0% S 120492K 21768K fg app_18 Signal Catcher com.android.inputmethod.latin 1206 1210 0% S 120492K 21768K fg app_18 JDWP com.android.inputmethod.latin 1206 1211 0% S 120492K 21768K fg app_18 Compiler com.android.inputmethod.latin 1206 1212 0% S 120492K 21768K fg app_18 Binder Thread # com.android.inputmethod.latin 1206 1213 0% S 120492K 21768K fg app_18 Binder Thread # com.android.inputmethod.latin 1206 1261 0% S 120492K 21768K bg app_18 AsyncTask #1 com.android.inputmethod.latin 1206 1267 0% S 120492K 21768K bg app_18 AsyncTask #2 com.android.inputmethod.latin 1206 1268 0% S 120492K 21768K bg app_18 AsyncTask #3 com.android.inputmethod.latin 1217 1217 0% S 131016K 23540K fg radio m.android.phone com.android.phone 1217 1220 0% S 131016K 23540K fg radio HeapWorker com.android.phone 1217 1222 0% S 131016K 23540K fg radio Signal Catcher com.android.phone 1217 1223 0% S 131016K 23540K fg radio JDWP com.android.phone 1217 1224 0% S 131016K 23540K fg radio Compiler com.android.phone 1217 1225 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1217 1226 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1217 1286 0% S 131016K 23540K fg radio RILSender com.android.phone 1217 1287 0% S 131016K 23540K fg radio RILReceiver com.android.phone 1217 1293 0% S 131016K 23540K fg radio GsmDataConnecti com.android.phone 1217 1294 0% S 131016K 23540K fg radio Stk Telephony s com.android.phone 1217 1295 0% S 131016K 23540K fg radio RilMessageDecod com.android.phone 1217 1296 0% S 131016K 23540K fg radio Stk Icon Loader com.android.phone 1217 1298 0% S 131016K 23540K fg radio AsyncQueryWorke com.android.phone 1217 1307 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1217 1376 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1217 1377 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1217 1378 0% S 131016K 23540K fg radio Binder Thread # com.android.phone 1227 1227 0% S 127400K 24628K bg app_17 ndroid.launcher com.android.launcher 1227 1229 0% S 127400K 24628K bg app_17 HeapWorker com.android.launcher 1227 1230 0% S 127400K 24628K bg app_17 Signal Catcher com.android.launcher 1227 1231 0% S 127400K 24628K bg app_17 JDWP com.android.launcher 1227 1232 0% S 127400K 24628K bg app_17 Compiler com.android.launcher 1227 1238 0% S 127400K 24628K fg app_17 Binder Thread # com.android.launcher 1227 1239 0% S 127400K 24628K fg app_17 Binder Thread # com.android.launcher 1233 1233 0% S 121516K 21172K bg system ndroid.settings com.android.settings 1233 1234 0% S 121516K 21172K bg system HeapWorker com.android.settings 1233 1240 0% S 121516K 21172K bg system Signal Catcher com.android.settings 1233 1242 0% S 121516K 21172K bg system JDWP com.android.settings 1233 1243 0% S 121516K 21172K bg system Compiler com.android.settings 1233 1244 0% S 121516K 21172K fg system Binder Thread # com.android.settings 1233 1245 0% S 121516K 21172K bg system Binder Thread # com.android.settings 1262 1262 0% S 132816K 26508K bg app_1 d.process.acore android.process.acore 1262 1263 0% S 132816K 26508K bg app_1 HeapWorker android.process.acore 1262 1264 0% S 132816K 26508K bg app_1 Signal Catcher android.process.acore 1262 1265 0% S 132816K 26508K bg app_1 JDWP android.process.acore 1262 1266 0% S 132816K 26508K bg app_1 Compiler android.process.acore 1262 1272 0% S 132816K 26508K bg app_1 Binder Thread # android.process.acore 1262 1273 0% S 132816K 26508K bg app_1 Binder Thread # android.process.acore 1262 1285 0% S 132816K 26508K bg app_1 ApplicationsPro android.process.acore 1262 1303 0% S 132816K 26508K bg app_1 Binder Thread # android.process.acore 1262 1304 0% S 132816K 26508K fg app_1 Binder Thread # android.process.acore 1262 1305 0% S 132816K 26508K bg app_1 Binder Thread # android.process.acore 1326 1326 0% S 114352K 19984K bg app_23 droid.deskclock com.android.deskclock 1326 1328 0% S 114352K 19984K bg app_23 HeapWorker com.android.deskclock 1326 1329 0% S 114352K 19984K bg app_23 Signal Catcher com.android.deskclock 1326 1330 0% S 114352K 19984K bg app_23 JDWP com.android.deskclock 1326 1331 0% S 114352K 19984K bg app_23 Compiler com.android.deskclock 1326 1332 0% S 114352K 19984K fg app_23 Binder Thread # com.android.deskclock 1326 1333 0% S 114352K 19984K bg app_23 Binder Thread # com.android.deskclock 1335 1335 0% S 118280K 21208K bg app_13 d.process.media android.process.media 1335 1337 0% S 118280K 21208K bg app_13 HeapWorker android.process.media 1335 1338 0% S 118280K 21208K bg app_13 Signal Catcher android.process.media 1335 1339 0% S 118280K 21208K bg app_13 JDWP android.process.media 1335 1340 0% S 118280K 21208K bg app_13 Compiler android.process.media 1335 1341 0% S 118280K 21208K fg app_13 Binder Thread # android.process.media 1335 1342 0% S 118280K 21208K fg app_13 Binder Thread # android.process.media 1335 1343 0% S 118280K 21208K bg app_13 thumbs thread android.process.media 1335 1345 0% S 118280K 21208K fg app_13 Binder Thread # android.process.media 1335 1346 0% S 118280K 21208K fg app_13 Binder Thread # android.process.media 1347 1347 0% S 118196K 21208K bg app_21 m.android.email com.android.email 1347 1349 0% S 118196K 21208K bg app_21 HeapWorker com.android.email 1347 1350 0% S 118196K 21208K bg app_21 Signal Catcher com.android.email 1347 1351 0% S 118196K 21208K bg app_21 JDWP com.android.email 1347 1352 0% S 118196K 21208K bg app_21 Compiler com.android.email 1347 1353 0% S 118196K 21208K fg app_21 Binder Thread # com.android.email 1347 1354 0% S 118196K 21208K fg app_21 Binder Thread # com.android.email 1347 1357 0% S 118196K 21208K bg app_21 Thread-9 com.android.email 1361 1361 0% S 127716K 20744K bg app_11 com.android.mms com.android.mms 1361 1363 0% S 127716K 20744K bg app_11 HeapWorker com.android.mms 1361 1364 0% S 127716K 20744K bg app_11 Signal Catcher com.android.mms 1361 1365 0% S 127716K 20744K bg app_11 JDWP com.android.mms 1361 1366 0% S 127716K 20744K bg app_11 Compiler com.android.mms 1361 1368 0% S 127716K 20744K bg app_11 Binder Thread # com.android.mms 1361 1369 0% S 127716K 20744K fg app_11 Binder Thread # com.android.mms 1361 1372 0% S 127716K 20744K bg app_11 Thread-9 com.android.mms 1381 1381 0% S 115452K 19888K bg app_30 droid.bluetooth com.android.bluetooth 1381 1383 0% S 115452K 19888K bg app_30 HeapWorker com.android.bluetooth 1381 1385 0% S 115452K 19888K bg app_30 Signal Catcher com.android.bluetooth 1381 1386 0% S 115452K 19888K bg app_30 JDWP com.android.bluetooth 1381 1387 0% S 115452K 19888K bg app_30 Compiler com.android.bluetooth 1381 1388 0% S 115452K 19888K fg app_30 Binder Thread # com.android.bluetooth 1381 1389 0% S 115452K 19888K fg app_30 Binder Thread # com.android.bluetooth 1381 1541 0% S 115452K 19888K fg app_30 Binder Thread # com.android.bluetooth 1390 1390 0% S 115360K 19996K bg app_27 viders.calendar com.android.providers.calendar 1390 1392 0% S 115360K 19996K bg app_27 HeapWorker com.android.providers.calendar 1390 1393 0% S 115360K 19996K bg app_27 Signal Catcher com.android.providers.calendar 1390 1394 0% S 115360K 19996K bg app_27 JDWP com.android.providers.calendar 1390 1395 0% S 115360K 19996K bg app_27 Compiler com.android.providers.calendar 1390 1396 0% S 115360K 19996K bg app_27 Binder Thread # com.android.providers.calendar 1390 1397 0% S 115360K 19996K fg app_27 Binder Thread # com.android.providers.calendar 1404 1404 0% S 138052K 22788K fg app_31 SUTAgentAndroid com.mozilla.SUTAgentAndroid 1404 1407 0% S 138052K 22788K fg app_31 Signal Catcher com.mozilla.SUTAgentAndroid 1404 1408 0% S 138052K 22788K fg app_31 JDWP com.mozilla.SUTAgentAndroid 1404 1410 0% S 138052K 22788K fg app_31 Binder Thread # com.mozilla.SUTAgentAndroid 1404 1411 0% S 138052K 22788K fg app_31 Binder Thread # com.mozilla.SUTAgentAndroid 1404 1709 0% S 138052K 22788K fg app_31 RunCmdThread com.mozilla.SUTAgentAndroid 1404 1710 0% S 138052K 22788K fg app_31 RunDataThread com.mozilla.SUTAgentAndroid 1404 1759 0% S 138052K 22788K fg app_31 .ProcessManager com.mozilla.SUTAgentAndroid 1404 1776 0% S 138052K 22788K fg app_31 DataWorkerThrea com.mozilla.SUTAgentAndroid 1404 1780 0% S 138052K 22788K fg app_31 CmdWorkerThread com.mozilla.SUTAgentAndroid 1404 1828 0% S 138052K 22788K fg app_31 CmdWorkerThread com.mozilla.SUTAgentAndroid 1414 1414 0% S 115672K 20392K bg app_32 mozilla.watcher com.mozilla.watcher 1414 1416 0% S 115672K 20392K bg app_32 HeapWorker com.mozilla.watcher 1414 1417 0% S 115672K 20392K bg app_32 Signal Catcher com.mozilla.watcher 1414 1418 0% S 115672K 20392K bg app_32 JDWP com.mozilla.watcher 1414 1419 0% S 115672K 20392K bg app_32 Compiler com.mozilla.watcher 1414 1420 0% S 115672K 20392K fg app_32 Binder Thread # com.mozilla.watcher 1414 1421 0% S 115672K 20392K fg app_32 Binder Thread # com.mozilla.watcher 1414 1431 0% S 115672K 20392K bg app_32 Timer-0 com.mozilla.watcher 1414 1496 0% S 115672K 20392K bg app_32 .ProcessManager com.mozilla.watcher 1423 1423 0% S 114880K 19368K bg app_10 m.android.music com.android.music 1423 1425 0% S 114880K 19368K bg app_10 HeapWorker com.android.music 1423 1426 0% S 114880K 19368K bg app_10 Signal Catcher com.android.music 1423 1427 0% S 114880K 19368K bg app_10 JDWP com.android.music 1423 1428 0% S 114880K 19368K bg app_10 Compiler com.android.music 1423 1429 0% S 114880K 19368K bg app_10 Binder Thread # com.android.music 1423 1430 0% S 114880K 19368K bg app_10 Binder Thread # com.android.music 1434 1434 0% S 116392K 19812K bg app_6 .quicksearchbox com.android.quicksearchbox 1434 1436 0% S 116392K 19812K bg app_6 HeapWorker com.android.quicksearchbox 1434 1437 0% S 116392K 19812K bg app_6 Signal Catcher com.android.quicksearchbox 1434 1438 0% S 116392K 19812K bg app_6 JDWP com.android.quicksearchbox 1434 1439 0% S 116392K 19812K bg app_6 Compiler com.android.quicksearchbox 1434 1440 0% S 116392K 19812K bg app_6 Binder Thread # com.android.quicksearchbox 1434 1441 0% S 116392K 19812K bg app_6 Binder Thread # com.android.quicksearchbox 1434 1442 0% S 116392K 19812K bg app_6 RefQueueWorker@ com.android.quicksearchbox 1443 1443 0% S 116184K 20508K bg app_20 .cooliris.media com.cooliris.media 1443 1445 0% S 116184K 20508K bg app_20 HeapWorker com.cooliris.media 1443 1446 0% S 116184K 20508K bg app_20 Signal Catcher com.cooliris.media 1443 1447 0% S 116184K 20508K bg app_20 JDWP com.cooliris.media 1443 1448 0% S 116184K 20508K bg app_20 Compiler com.cooliris.media 1443 1449 0% S 116184K 20508K bg app_20 Binder Thread # com.cooliris.media 1443 1450 0% S 116184K 20508K bg app_20 Binder Thread # com.cooliris.media 1443 1451 0% S 116184K 20508K bg app_20 RefQueueWorker@ com.cooliris.media 1477 1477 0% S 788K 340K fg dhcp dhcpcd /system/bin/dhcpcd 1680 1680 0% S 0K 0K fg root AR6K Async 1684 1684 0% S 0K 0K unk root ksdioirqd/mmc0 1686 1686 0% S 1888K 992K fg wifi wpa_supplicant /system/bin/wpa_supplicant 1784 1784 0% S 0K 0K unk root migration/1 1785 1785 0% S 0K 0K fg root ksoftirqd/1 1786 1786 0% S 0K 0K unk root watchdog/1 1787 1787 0% S 0K 0K fg root crypto/1 1788 1788 0% S 0K 0K fg root aio/1 1789 1789 0% S 0K 0K fg root kblockd/1 1790 1790 0% S 0K 0K fg root events/1 1806 1806 0% S 121604K 18584K bg app_33 g.mozilla.fencp org.mozilla.fencp 1806 1808 0% S 121604K 18584K bg app_33 HeapWorker org.mozilla.fencp 1806 1809 0% S 121604K 18584K bg app_33 Signal Catcher org.mozilla.fencp 1806 1810 0% S 121604K 18584K bg app_33 JDWP org.mozilla.fencp 1806 1811 0% S 121604K 18584K bg app_33 Compiler org.mozilla.fencp 1806 1812 0% S 121604K 18584K bg app_33 Binder Thread # org.mozilla.fencp 1806 1813 0% S 121604K 18584K fg app_33 Binder Thread # org.mozilla.fencp 1806 1814 0% S 121604K 18584K bg app_33 Binder Thread # org.mozilla.fencp 1806 1815 0% S 121604K 18584K bg app_33 Binder Thread # org.mozilla.fencp 1816 1816 0% S 117472K 18540K bg app_34 g.mozilla.ffxcp org.mozilla.ffxcp 1816 1818 0% S 117472K 18540K bg app_34 HeapWorker org.mozilla.ffxcp 1816 1819 0% S 117472K 18540K bg app_34 Signal Catcher org.mozilla.ffxcp 1816 1820 0% S 117472K 18540K bg app_34 JDWP org.mozilla.ffxcp 1816 1821 0% S 117472K 18540K bg app_34 Compiler org.mozilla.ffxcp 1816 1822 0% S 117472K 18540K bg app_34 Binder Thread # org.mozilla.ffxcp 1816 1823 0% S 117472K 18540K fg app_34 Binder Thread # org.mozilla.ffxcp 1824 1824 0% S 0K 0K fg root flush-179:8 946 1042 0% S 24544K 1156K fg root nvrm_daemon /system/bin/nvrm_daemon 2 2 0% S 0K 0K fg root kthreadd 3 3 0% S 0K 0K unk root migration/0 4 4 0% S 0K 0K fg root ksoftirqd/0 5 5 0% S 0K 0K unk root watchdog/0 9 9 0% S 0K 0K fg root events/0 11 11 0% S 0K 0K fg root khelper 15 15 0% S 0K 0K fg root async/mgr return code [0] $>
(In reply to Justin Wood (:Callek) from comment #2) > so I did a top -t Good idea! So your 2 worst offenders are anonymous threads - not CmdWorkerThread, not DataWorkerThread, etc. RedirOutputThreads are anonymous, so this is consistent with the issue that I am observing. (And I have updated my patch to also name RedirOutputThreads so that we can observe them with more confidence in future.)
Nothing fancy here: Just sleep in between checks for more output. SUT agent already does something very similar in the FindProcThread. This brings the cpu down to approx 0% while waiting for a process, according to top.
Attachment #632699 - Flags: review?(jmaher)
Comment on attachment 632699 [details] [diff] [review] sleep in between checks for output in RedirOutputThread Review of attachment 632699 [details] [diff] [review]: ----------------------------------------------------------------- simple idea and great solution! ::: build/mobile/sutagent/android/FindProcThread.java @@ +16,5 @@ > boolean bFoundIt = false; > boolean bStillRunning = true; > > public FindProcThread(ContextWrapper ctx, String sProcessName) { > + super("FindProcThread"); why the need for super() here?
Attachment #632699 - Flags: review?(jmaher) → review+
(In reply to Joel Maher (:jmaher) from comment #5) > > + super("FindProcThread"); > > why the need for super() here? Based on Comment 2, I am thinking that it would be nice to have as many named threads as possible, so that we can track down this sort of problem easier in future.
makes good sense! Do we have other sutagent low hanging fruit in the works? This will require a sutagent upgrade on the whole tegra farm.
Changed sleep time to 50 ms as per irc chat to avoid possible latency issues. I re-tested and sutagent's cpu was still < 1%.
Attachment #632699 - Attachment is obsolete: true
Attachment #632818 - Flags: review+
(In reply to Brad Lassey [:blassey] from comment #9) > can we use a FileObserver instead? No, not really. The RedirOutputThread is collecting a process' stdout and stderr output and redirecting it to another stream. FileObserver just works with files. (We could perhaps redirect the process output to a file, observe that file, and read it...but then we need to manage these temporary files, etc -- I don't want to get into that.) I suppose the standard way of handling this would be to block in a stream.read() call, rather than spinning and constantly checking for data. The complication here is that we are reading from 2 streams (out/err) at the same time and coalescing the output of those 2 streams. We could do it with 2 threads...not sure if that change is worth making, given that we already have a (inelegant but) working solution.
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/6daf5f7edac8
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla16
Blocks: 759792
(In reply to Geoff Brown [:gbrown] from comment #10) > (In reply to Brad Lassey [:blassey] from comment #9) > > can we use a FileObserver instead? > > No, not really. The RedirOutputThread is collecting a process' stdout and > stderr output and redirecting it to another stream. FileObserver just works > with files. (We could perhaps redirect the process output to a file, observe > that file, and read it...but then we need to manage these temporary files, > etc -- I don't want to get into that.) > > I suppose the standard way of handling this would be to block in a > stream.read() call, rather than spinning and constantly checking for data. > The complication here is that we are reading from 2 streams (out/err) at the > same time and coalescing the output of those 2 streams. We could do it with > 2 threads...not sure if that change is worth making, given that we already > have a (inelegant but) working solution. this seems worth exploring to me. please file a follow up bug
(In reply to Brad Lassey [:blassey] from comment #13) > this seems worth exploring to me. please file a follow up bug Done - bug 768184.
No longer blocks: 759792
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: