mochitest-robocop logs are very frequently missing parts, including tests

RESOLVED INCOMPLETE

Status

Testing
Mochitest
RESOLVED INCOMPLETE
6 years ago
3 years ago

People

(Reporter: philor, Unassigned)

Tracking

Trunk
ARM
Android
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
If you look at the pass/fail counts on tbpl, you'll see that green mochitest-robocop runs vary between around 200 passes and 206. Looking at the logs, say comparing https://tbpl.mozilla.org/php/getParsedLog.php?id=10161889&tree=Mozilla-Inbound (200 passes) with https://tbpl.mozilla.org/php/getParsedLog.php?id=10161953&tree=Mozilla-Inbound (206 passes), it looks like a pretty hefty percentage of the individual test runs have their logs cut off before the finish/pass/fail part, and quite often something is cut off without showing all of the test bits.

That 200 pass is missing 3 things from testWebContentContextMenu, and I'm not sure what else - probably bits of testBrowserProvider, but so far I haven't seen a single log that shows the end of it.

I'm assuming just logs getting cut off, but I guess it's equally likely that it's actually something worse, like a crash not getting logged and not marking the run as a failure.
For testBrowserProvider, it looks like we are crashing.  The reason why we don't report it is right now we don't have a solid solution for reporting.  I haven't ran into a case where a log is missing lines that exist on the tegra.  

My bugzilla skills are not very good today, but there is a bug which has some information about this general problem with some proposed solutions.
(Reporter)

Comment 2

3 years ago
Wonder whether that bug ever fixed it?
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.