it looks as if we can sometimes get a screenshot of the 'next test' instead of the failure when it happens (in browser chrome at least)

ASSIGNED
Assigned to

Status

Testing
Mochitest
ASSIGNED
5 months ago
4 months ago

People

(Reporter: jmaher, Assigned: ahal)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 months ago
I was pointed to an interesting problem, given this failure:
https://treeherder.mozilla.org/logviewer.html#?job_id=105189307&repo=try&lineNumber=3378

we see this screenshot:
https://public-artifacts.taskcluster.net/Qx2DVtFjSHCbTwlO97yYgQ/0/public/test_info//mozilla-test-fail-screenshot_ayDru2.png

the screenshot is for printpreview which is the test immediately after the failing test.

I don't see any other reason why that would happen other than we preload or do something in parallel/out of sequence.

We should look in more detail at the screenshot on failure code for browser-chrome and ensure that it is taking a screenshot before cleaning up and before moving onto the next test.
(Reporter)

Comment 1

5 months ago
:ahal, is this something you could pick up?
Flags: needinfo?(ahalberstadt)
(Assignee)

Comment 2

5 months ago
Yep! I'll take a look.
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
(Assignee)

Comment 3

4 months ago
The screenshot handling code is all in python, which means that there's no way it can block the JS side from moving on to the next test. I think it's a race condition where the screenshot may or may not be captured before the next test is loaded. As far as I can tell, this has always happened. I guess it is mostly useful when the browser hangs and doesn't progress to the next test (or gets killed immediately after).

I also think the log is a bit confusing, some things may be out of order due to buffering. Yesterday I discovered that mochitest does a whole bunch of wrong things related to structured logging (bug 1372567) which contributes further to the log confusion.
Flags: needinfo?(ahalberstadt)
You need to log in before you can comment on or make changes to this bug.