1340001 - Local 'mach robocop' timeouts on x86 emulator

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: Testing, defect, P5)

Description

[Tom Klein]

Attachment: x86 emulator single test output

I had some issues running './mach robocop' locally on an x86 emulator (api 23) - in some regards it seemed to work normally, with some tests passing, but there also seemed to be more timeouts and more fails than I'm used to seeing.  I stopped it after four hours before it completed, so I'm not entirely sure if there really were more failures or not.  In the past (a couple months ago?) it has worked for me and completed in ~one hour.

The attached log is the output of running just one test: './mach robocop testPictureLinkContextMenu'.

Geoff suggested trying
  adb shell pm grant org.mozilla.fennec_klein android.permission.CAMERA
on irc to fix the 'WARNING: Unable to grant runtime permissions to org.mozilla.fennec_klein' error, but that didn't have any effect.

Comment 1

[Geoff Brown [:gbrown]]

(In reply to Tom Klein from comment #0)
> Geoff suggested trying
>   adb shell pm grant org.mozilla.fennec_klein android.permission.CAMERA
> on irc to fix the 'WARNING: Unable to grant runtime permissions to
> org.mozilla.fennec_klein' error, but that didn't have any effect.

Actually, I suggested that not to fix that warning, but to diagnose it: That warning suggests that when the test harness issues commands like "adb shell pm grant org.mozilla.fennec_klein android.permission...." something fails, but the log doesn't show what is going wrong. I thought a manual command like "adb shell pm grant org.mozilla.fennec_klein android.permission.CAMERA" might tell us what is going wrong.

Comment 2

[Jan Henning [:JanH]]

Just wanted to add that I've been seeing the "Unable to grant runtime permissions" when running mochitests locally as well, possibly ever since my old phone died and I upgraded to Android 6.

Comment 3

[Tom Klein]

Attachment: 'mach robocop' output on x86 emulator (zipped)

I let 'mach robocop' run overnight.  It finished in 3 hours 52 minutes.  The final summary listed 14 "Unexpected Results", all except one of which are on my list of tests that have basically always been permafails for me locally, so those aren't surprising.

But the final summary doesn't mention 20 other tests that failed with a crash and an ANR report.  The output on those are all basically the same, like:

 2:39.83 LOG: None INFO SimpleTest START
 2:39.83 TEST_START: None testAboutPage
 8:22.82 LOG: MainThread INFO EventExpecter: no longer listening for Gecko:Ready
 8:22.82 LOG: MainThread INFO EventExpecter: no longer listening for Content:DOMContentLoaded
 8:22.82 LOG: MainThread INFO PaintExpecter: no longer listening for events
11:39.24 LOG: MainThread INFO Failed to retrieve MOZ_UPLOAD_DIR env var
org.mozilla.fennec_klein still alive after SIGABRT: waiting...
TEST-UNEXPECTED-FAIL | testAboutPage | application timed out after 180 seconds with no output
INFO | automation.py | Application ran for: 0:08:46.211150
INFO | zombiecheck | Reading PID log: /tmp/tmpL0F7Ispidlog
Contents of /data/anr/traces.txt:
...[anr report output]...
MOZ_UPLOAD_DIR not defined; tombstone check skipped
11:49.82 CRASH: MainThread pid:26458. Test:testAboutPage. Minidump anaylsed:False. Signature:[None]
Crash dump filename: /tmp/tmpoIKPYF/13aec17b-3188-0fab-093eb43d-272d9644.dmp
MINIDUMP_STACKWALK not set, can't process dump.

11:49.82 LOG: MainThread ERROR runApp() exited with code 1
11:50.10 LOG: MainThread INFO PROCESS-CRASH | Automation Error: Missing end of test marker (process crashed?)

Then I reran five of those anr tests at semi-random, with the following results:
testFormHistory --> pass
testLoad --> failed with an anr trace (I didn't capture output)
testPanCorrectness --> failed with an anr trace (I'll say more in a followup)
testPrefs --> pass
testPromptGridInput --> regular old unexpected fail, no anr or crash

Comment 4

[Tom Klein]

Attachment: Logcat and test output for testPictureLinkContextMenu with anr and crash

I reran testPanCorrectness again and watched logcat and the emulator - a test file loads fairly quickly but then there's no more logcat or onscreen action until logcat reports some anr stuff, shortly after which the test finishes.  There's never an onscreen ANR warning.

I get the same results with testPictureLinkContextMenu (from comment 0), for which I'm attaching the end of the logcat (with the anr stuff) and the test output.  That test I've seen run successfully, so I know in this failing version it's not opening menus etc like it does normally; after the test page loads I don't see any change on screen.  There's nothing suspicious, aside from the anr writing, in the logcat output.

Comment 5

[Geoff Brown [:gbrown]]

(In reply to Tom Klein from comment #4)
> Created attachment 8838108 [details]
> Logcat and test output for testPictureLinkContextMenu with anr and crash

I see:

 5:40.57 LOG: MainThread INFO EventExpecter: no longer listening for Gecko:Ready
 5:40.58 LOG: MainThread INFO EventExpecter: no longer listening for Content:DOMContentLoaded
 5:40.58 LOG: MainThread INFO PaintExpecter: no longer listening for events
 8:54.99 LOG: MainThread INFO Failed to retrieve MOZ_UPLOAD_DIR env var
org.mozilla.fennec_klein still alive after SIGABRT: waiting...
TEST-UNEXPECTED-FAIL | testPictureLinkContextMenu | application timed out after 180 seconds with no output

After  5:40.58, it looks like there's a lot of non-activity. After 180 seconds of no new log output, at 8:54.99, the test harness gives up and dumps "application timed out after 180 seconds with no output" and then kills Fennec, initially with SIGABRT.

You can see the SIGABRT received in the logcat:

02-16 09:34:59.400 9511-9516/org.mozilla.fennec_klein I/art: Thread[2,tid=9516,WaitingInMainSignalCatcherLoop,Thread*=0xed891d00,peer=0x12c450a0,"Signal Catcher"]: reacting to signal 3


I think the interesting issue is, why is there no test output for 180 seconds -- what is the test waiting for?

Comment 6

[Tom Klein]

Attachment: Full logcat of testPictureLinkContextMenu failed run

Ugh, I had only been looking in logcat for failures once the test was up and running and didn't see anything (didn't even realize robocop updated its progress in logcat), but looking further back there are actually a number of warnings and errors that look test related.  That's lines 215-273  in the attachment, since it's hard to read here.

02-16 12:06:09.290 22916-22940/org.mozilla.fennec_klein W/FileUtils: Failed to chmod(/mnt/sdcard/tests/profile/browser.db): android.system.ErrnoException: chmod failed: EPERM (Operation not permitted)
02-16 12:06:09.320 22916-22916/org.mozilla.fennec_klein D/GeckoToolbar: onTabChanged: RESTORED
02-16 12:06:09.320 22916-22916/org.mozilla.fennec_klein D/Telemetry: SendUIEvent: event = launch.1 method = homescreen timestamp = 4763460 extras = launcher
02-16 12:06:09.420 22916-22940/org.mozilla.fennec_klein E/SQLiteLog: (1) no such table: bookmarks
02-16 12:06:09.650 22916-22942/org.mozilla.fennec_klein W/GeckoLinker: /data/app/org.mozilla.fennec_klein-1/base.apk!/assets/x86/libsoftokn3.so: unhandled flags #8 not handled
02-16 12:06:09.660 22916-22942/org.mozilla.fennec_klein W/GeckoLinker: /data/app/org.mozilla.fennec_klein-1/base.apk!/assets/x86/libfreebl3.so: unhandled flags #8 not handled
02-16 12:06:09.760 22916-22916/org.mozilla.fennec_klein I/Choreographer: Skipped 37 frames!  The application may be doing too much work on its main thread.
02-16 12:06:09.780 22916-22942/org.mozilla.fennec_klein W/GeckoLinker: /data/app/org.mozilla.fennec_klein-1/base.apk!/assets/x86/libnssckbi.so: unhandled flags #8 not handled
02-16 12:06:09.900 22916-22942/org.mozilla.fennec_klein I/GeckoConsole: 1487268369890	addons.xpi	WARN	Failed to call uninstall for [e-mail]: Error: Unknown add-on ID [e-mail] (resource://gre/modules/addons/XPIProvider.jsm:8053:11) JS Stack trace: [e-mail]:8053:11 < [e-mail]:3361:31 < [e-mail]:3681:19 < [e-mail]:2793:25 < [e-mail]:272:12 < [e-mail]:756:5 < [e-mail]:938:9 < [e-mail]:3129:5 < [e-mail]:65:9
02-16 12:06:09.900 22916-22942/org.mozilla.fennec_klein I/Gecko: 1487268369890	addons.xpi	WARN	Failed to call uninstall for [e-mail]: Error: Unknown add-on ID [e-mail] (resource://gre/modules/addons/XPIProvider.jsm:8053:11) JS Stack trace: [e-mail]:8053:11 < [e-mail]:3361:31 < [e-mail]:3681:19 < [e-mail]:2793:25 < [e-mail]:272:12 < [e-mail]:756:5 < [e-mail]:938:9 < [e-mail]:3129:5 < [e-mail]:65:9
02-16 12:06:09.900 22916-22942/org.mozilla.fennec_klein I/GeckoConsole: 1487268369900	addons.xpi	WARN	Attempted to remove [e-mail] from app-profile but it was already gone
02-16 12:06:09.900 22916-22942/org.mozilla.fennec_klein I/Gecko: 1487268369900	addons.xpi	WARN	Attempted to remove [e-mail] from app-profile but it was already gone
02-16 12:06:09.900 22916-22942/org.mozilla.fennec_klein I/GeckoConsole: 1487268369900	addons.xpi	WARN	Failed to set permissions 755 on /storage/emulated/0/tests/profile/extensions/staged/[e-mail]: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.permissions]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: setFilePermissions :: line 413"  data: no] Stack trace: setFilePermissions()@resource://gre/modules/addons/XPIProvider.jsm:413 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1971 < cleanStagingDir()@resource://gre/modules/addons/XPIProvider.jsm:8140 < processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:3493 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3681 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2793 < callProvider()@resource://gre/modules/AddonManager.jsm:272 < _startProvider()@resource://gre/modules/AddonManager.jsm:756 < startup()@resource://gre/modules/
02-16 12:06:09.910 22916-22942/org.mozilla.fennec_klein I/Gecko: 1487268369900	addons.xpi	WARN	Failed to set permissions 755 on /storage/emulated/0/tests/profile/extensions/staged/[e-mail]: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.permissions]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: setFilePermissions :: line 413"  data: no] Stack trace: setFilePermissions()@resource://gre/modules/addons/XPIProvider.jsm:413 < recursiveRemove()@resource://gre/modules/addons/XPIProvider.jsm:1971 < cleanStagingDir()@resource://gre/modules/addons/XPIProvider.jsm:8140 < processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:3493 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3681 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2793 < callProvider()@resource://gre/modules/AddonManager.jsm:272 < _startProvider()@resource://gre/modules/AddonManager.jsm:756 < startup()@resource://gre/modules/
02-16 12:06:09.910 22916-22942/org.mozilla.fennec_klein I/GeckoConsole: 1487268369910	addons.xpi	WARN	Failed to set permissions 755 on /storage/emulated/0/tests/profile/extensions/staged: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.permissions]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: setFilePermissions :: line 413"  data: no] Stack trace: setFilePermissions()@resource://gre/modules/addons/XPIProvider.jsm:413 < cleanStagingDir()@resource://gre/modules/addons/XPIProvider.jsm:8155 < processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:3493 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3681 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2793 < callProvider()@resource://gre/modules/AddonManager.jsm:272 < _startProvider()@resource://gre/modules/AddonManager.jsm:756 < startup()@resource://gre/modules/AddonManager.jsm:938 < startup()@resource://gre/modules/AddonManager.jsm:3129 < observe()@jar:ja
02-16 12:06:09.910 22916-22942/org.mozilla.fennec_klein I/Gecko: 1487268369910	addons.xpi	WARN	Failed to set permissions 755 on /storage/emulated/0/tests/profile/extensions/staged: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.permissions]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: setFilePermissions :: line 413"  data: no] Stack trace: setFilePermissions()@resource://gre/modules/addons/XPIProvider.jsm:413 < cleanStagingDir()@resource://gre/modules/addons/XPIProvider.jsm:8155 < processPendingFileChanges()@resource://gre/modules/addons/XPIProvider.jsm:3493 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3681 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2793 < callProvider()@resource://gre/modules/AddonManager.jsm:272 < _startProvider()@resource://gre/modules/AddonManager.jsm:756 < startup()@resource://gre/modules/AddonManager.jsm:938 < startup()@resource://gre/modules/AddonManager.jsm:3129 < observe()@jar:ja
02-16 12:06:10.040 22916-22916/org.mozilla.fennec_klein W/FragmentManager: moveToState: Fragment state for BookmarksPanel{f9be882 #1 id=0x7f090110} not updated inline; expected state 3 found 2
02-16 12:06:10.070 22916-22916/org.mozilla.fennec_klein W/FragmentManager: moveToState: Fragment state for CombinedHistoryPanel{6689bfb #2 id=0x7f090110} not updated inline; expected state 3 found 2
02-16 12:06:10.090 22916-22916/org.mozilla.fennec_klein D/GeckoToolbar: onTabChanged: MENU_UPDATED
02-16 12:06:10.090 22916-22916/org.mozilla.fennec_klein D/GeckoBrowserApp: BrowserApp.onTabChanged: 0: MENU_UPDATED
02-16 12:06:10.090 22916-22916/org.mozilla.fennec_klein D/GeckoToolbar: onTabChanged: FAVICON
02-16 12:06:10.090 22916-22916/org.mozilla.fennec_klein D/GeckoBrowserApp: BrowserApp.onTabChanged: 0: FAVICON
02-16 12:06:10.180 22916-22942/org.mozilla.fennec_klein I/nsScreenManagerAndroid: nsWindow[0xd883c440]::Create 0x0 [0 0 1 1]
02-16 12:06:10.180 22916-22942/org.mozilla.fennec_klein I/nsScreenManagerAndroid: nsScreenManagerAndroid: add PRIMARY screen
02-16 12:06:10.190 22916-22942/org.mozilla.fennec_klein W/ResourceType: Too many attribute references, stopped at: 0x01010099
02-16 12:06:10.290 22916-22916/org.mozilla.fennec_klein W/InputMethodManager: Ignoring onBind: cur seq=16, given seq=15
02-16 12:06:10.300 22916-22916/org.mozilla.fennec_klein D/GeckoNetworkManager: Incoming event enableNotifications for state OnNoListeners -> OnWithListeners
02-16 12:06:10.300 22916-22916/org.mozilla.fennec_klein D/GeckoNetworkManager: New network state: UP, CELLULAR, CELL_3G
02-16 12:06:10.300 22916-22916/org.mozilla.fennec_klein D/GeckoNetworkManager: Incoming event receivedUpdate for state OnWithListeners -> OnWithListeners
02-16 12:06:10.300 22916-22916/org.mozilla.fennec_klein D/GeckoNetworkManager: New network state: UP, CELLULAR, CELL_3G
02-16 12:06:10.380 22916-22939/org.mozilla.fennec_klein I/Robocop: {"thread":null,"source":"robocop","pid":null,"status":"PASS","time":1487268370380,"action":"test_status","test":"testPictureLinkContextMenu","subtest":"Robocop tests need the test device screen to be powered on.","message":""}
02-16 12:06:10.380 22916-22939/org.mozilla.fennec_klein D/Robocop: waiting for Gecko:Ready
02-16 12:06:10.380 22916-22986/org.mozilla.fennec_klein D/GeckoSuggestedSites: No cached sites, refreshing.
02-16 12:06:10.380 22916-22986/org.mozilla.fennec_klein D/GeckoSuggestedSites: Refreshing suggested sites from file
02-16 12:06:10.390 22916-22986/org.mozilla.fennec_klein D/GeckoSuggestedSites: Loading blacklisted suggested sites from SharedPreferences.
02-16 12:06:10.390 22916-22986/org.mozilla.fennec_klein D/GeckoSuggestedSites: Number of suggested sites: 8
02-16 12:06:10.410 22916-22916/org.mozilla.fennec_klein W/Settings: Setting airplane_mode_on has moved from android.provider.Settings.System to android.provider.Settings.Global, returning read-only value.
02-16 12:06:10.500 22916-22942/org.mozilla.fennec_klein I/Gecko: -*- nsDNSServiceDiscovery.js : nsDNSServiceDiscovery
02-16 12:06:10.570 22916-22942/org.mozilla.fennec_klein I/Gecko: Attempting load of libEGL.so
                                                                 
                                                                 
                                                                 [ 02-16 12:06:10.570 22916:22942 D/         ]
                                                                 HostConnection::get() New Host Connection established 0xeb8b0750, tid 22942
02-16 12:06:10.700 22916-22942/org.mozilla.fennec_klein I/nsScreenManagerAndroid: nsWindow[0xd883e740]::Create 0x0 [0 0 100 100]
02-16 12:06:10.710 22916-22942/org.mozilla.fennec_klein D/GeckoLayerClient: Screen-size changed to (1080,1800)
02-16 12:06:10.710 22916-22942/org.mozilla.fennec_klein D/GeckoLayerClient: Window-size changed to (1080,1620)
                                                                            
                                                                            [ 02-16 12:06:10.720 22916:23001 D/         ]
                                                                            HostConnection::get() New Host Connection established 0xd7608b60, tid 23001
02-16 12:06:10.820 22916-23001/org.mozilla.fennec_klein I/GeckoConsole: OpenGL compositor Initialized Succesfully.
                                                                        Version: OpenGL ES 2.0 (3.0 Mesa 12.0.3)
                                                                        Vendor: Google (X.Org)
                                                                        Renderer: Android Emulator OpenGL ES Translator (Gallium 0.4 on AMD ARUBA (DRM 2.48.0 / 4.9.6-100.fc24.x86_64, LLVM 3.8.0))
                                                                        FBO Texture Target: TEXTURE_2D
02-16 12:06:10.850 22916-22942/org.mozilla.fennec_klein E/GeckoLinker: /storage/emulated/0/tests/profile/plugins/libnpsecondtest.so: Failed to mmap
02-16 12:06:10.850 22916-22942/org.mozilla.fennec_klein E/GeckoLinker: /storage/emulated/0/tests/profile/plugins/libnptest.so: Failed to mmap
02-16 12:06:10.850 22916-22942/org.mozilla.fennec_klein E/GeckoLinker: /storage/emulated/0/tests/profile/plugins/libnpswftest.so: Failed to mmap
02-16 12:06:10.850 22916-22942/org.mozilla.fennec_klein E/GeckoLinker: /storage/emulated/0/tests/profile/plugins/libnptestjava.so: Failed to mmap
02-16 12:06:10.850 22916-22942/org.mozilla.fennec_klein E/GeckoLinker: /storage/emulated/0/tests/profile/plugins/libnpthirdtest.so: Failed to mmap

Comment 7

[Tom Klein]

I added some extra logging to the test (mine start with "*** " in case they show up in the logs anywhere), which shows that we never get to line 27 here: https://dxr.mozilla.org/mozilla-central/source/mobile/android/tests/browser/robocop/src/org/mozilla/gecko/tests/testPictureLinkContextMenu.java#26 Don't know why not yet, but at this point the errors in the previous comment seem like the place to start, at least to me (and I don't know anything about those issues :).

Also I think the timestamps on the test output are somewhat misleading in this case: those
 5:40.57 LOG: MainThread INFO EventExpecter: no longer listening for Gecko:Ready
 5:40.58 LOG: MainThread INFO EventExpecter: no longer listening for Content:DOMContentLoaded
 5:40.58 LOG: MainThread INFO PaintExpecter: no longer listening for events
lines in the test output actually take place as soon as the test page is loaded, which I can see in logcat (and on the emulator) happens just shortly after the browser starts up, less than a minute into the test.

Comment 8

[Tom Klein]

I don't know if it's worth investigating or not given the hard shutdowns and other things going on here, but I'm also seeing session store errors in logcat during startup (lines 36-53 in the full logcat):

02-16 12:06:08.970 22916-22940/org.mozilla.fennec_klein E/GeckoApp: An error occurred during restore, switching to backup file
                                                                    org.mozilla.gecko.GeckoApp$SessionRestoreException: Could not read from session file
                                                                        at org.mozilla.gecko.GeckoApp.restoreSessionTabs(GeckoApp.java:1770)
                                                                        at org.mozilla.gecko.GeckoApp.access$200(GeckoApp.java:131)
                                                                        at org.mozilla.gecko.GeckoApp$15.run(GeckoApp.java:1352)
                                                                        at android.os.Handler.handleCallback(Handler.java:739)
                                                                        at android.os.Handler.dispatchMessage(Handler.java:95)
                                                                        at android.os.Looper.loop(Looper.java:148)
                                                                        at org.mozilla.gecko.util.GeckoBackgroundThread.run(GeckoBackgroundThread.java:43)
02-16 12:06:08.970 22916-22940/org.mozilla.fennec_klein E/GeckoApp: An error occurred during restore
                                                                    org.mozilla.gecko.GeckoApp$SessionRestoreException: Could not read from session file
                                                                        at org.mozilla.gecko.GeckoApp.restoreSessionTabs(GeckoApp.java:1770)
                                                                        at org.mozilla.gecko.GeckoApp.access$200(GeckoApp.java:131)
                                                                        at org.mozilla.gecko.GeckoApp$15.run(GeckoApp.java:1370)
                                                                        at android.os.Handler.handleCallback(Handler.java:739)
                                                                        at android.os.Handler.dispatchMessage(Handler.java:95)
                                                                        at android.os.Looper.loop(Looper.java:148)
                                                                        at org.mozilla.gecko.util.GeckoBackgroundThread.run(GeckoBackgroundThread.java:43)

Comment 9

[Jan Henning [:JanH]]

The session restore error logging is perhaps a bit unfortunate, but expected when starting with a fresh profile.

Comment 10

[Tom Klein]

This has improved somewhat, though it's possible it's simply because I've changed emulators.  Previous tests were run on a Nexus 5x API 23 emulator, but then I changed SDKs around and that emulator's (basically) gone. Now running 'mach robocop' on an x86 Nexus 5x API 24 emulator takes 107 minutes (compare with 232 minutes in comment 3).  In comment 3 there were 20 tests that reported timing out, now there are only 4, still including testPictureLinkContextMenu.

Comment 11

[Tom Klein]

A couple more related notes:

* 'mach install' with no devices attached or emulators running works again now (i.e. an emulator starts when I say 'Y'), and running 'mach robocop' on the 'mach android-emulator --version x86' emulator finishes cleanly in 74 minutes \o/

* Running 'mach robocop' on the 'mach android-emulator --version x86-6.0' emulator, on the other hand, ran for 314 minutes without finishing before I stopped it, and had 32 'application timed out after 180 seconds with no output' entries at that point.

Comment 12

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 13

[Geoff Brown [:gbrown]]

robocop is obsolete now.