I think the recent oranges are simply due to the timeout ("1257700.html | load failed: timed out after 300000 ms waiting for 'load' event for http://10.0.2.2:8854/tests/dom/media/test/crashtests/1257700.html").
The log-parser thinks it should show all of the TEST-UNEXPECTED-FAIL/PASS lines for the unexpected assertion counts as well, but in fact those are not what caused the log to get assessed as orange.
So maybe there's a known issue with assertion count mismatches on android (e.g. due to mis-grouping as is my current working theory), and we intentionally ignore them, but the log parser isn't smart enough to know that we're ignoring them, so it still highlights them when the testrun is classified as a failure for other reasons?
Bug 1585652 Comment 5 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
I think the recent oranges are simply due to the unrelated test timeout ("1257700.html | load failed: timed out after 300000 ms waiting for 'load' event for http://10.0.2.2:8854/tests/dom/media/test/crashtests/1257700.html").
The log-parser thinks it should show all of the TEST-UNEXPECTED-FAIL/PASS lines for the unexpected assertion counts as well, but in fact those are not what caused the log to get assessed as orange.
So maybe there's a known issue with assertion count mismatches on android (e.g. due to mis-grouping as is my current working theory), and we intentionally ignore them, but the log parser isn't smart enough to know that we're ignoring them, so it still highlights them when the testrun is classified as a failure for other test-failure reasons? (e.g. timeouts/crashes)
I think the recent oranges are simply due to the unrelated test timeout ("1257700.html | load failed: timed out after 300000 ms waiting for 'load' event for http://10.0.2.2:8854/tests/dom/media/test/crashtests/1257700.html").
The log-parser thinks it should show all of the TEST-UNEXPECTED-FAIL/PASS lines for the unexpected assertion counts as well, but in fact those are not what caused the log to get assessed as orange.
So maybe there's a known issue with assertion count mismatches on android (e.g. due to mis-grouping as is my current working theory), and so we intentionally ignore them, but the log parser isn't smart enough to know that we're ignoring them, so it still highlights them when the testrun is classified as a failure for other test-failure reasons? (e.g. timeouts/crashes)