1706016 - logs for many wpt tasks contain more than 100 lines picked up as failure lines by Treeherder, prevents real test failures from being shown

Status: RESOLVED WORKSFORME

(Testing :: web-platform-tests, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Last week, 23 wpt chunks hit the limit of failure lines extracted from the log (100). Open https://sql.telemetry.mozilla.org/queries/65547/source and sort by line_count_new. This prevents the real failures from being shown and sheriffs have to open the log and search for it. Please reduce this verbose logging.

Comment 1

[James Graham [:jgraham]]

OK, so it seems like what happened here is we suddenly started logging the failures in handler functions. Those need to be fixed, but also the way the logging was happening was broken. So I've filed some upstream PRs to both fix the logging and some of the specific problems:

• https://github.com/web-platform-tests/wpt/pull/28621 (fixes the logging setup)
• https://github.com/web-platform-tests/wpt/pull/28625
• https://github.com/web-platform-tests/wpt/pull/28626

Comment 2

[James Graham [:jgraham]]

Given the above PRs landed, it seems like this problem has got a lot better, so I'm closing as WFM. Feel free to reopen if you disagree with that assessment.