1140452 - Failures from bug 1131287 have to be classified manually since "test.html" returns too many bug suggestions

Status: RESOLVED WORKSFORME

(Testing :: web-platform-tests, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

This means it's a manual star. I'm not sure offhand what's different about these failures.

Comment 1

[Ed Morley [:emorley]]

The search term is presumably "test.html", awful name for the test.
So this is working as expected, since there are 98 bugs that match it:
https://treeherder.mozilla.org/api/bugscache/?search=test.html

Solutions:
a) Rename the test
b) Manually star this one and shrug it off as an edge case
c) Come up with some even more convoluted algorithm for selecting a better search term (eg: if too many results, try using more of the filepath than just the filename and search again using that)

I'm not adverse to (c) per se, but it would be good to know how many other cases this would help (eg the webgl ones?) or if it's just a one off, and so (a) is simpler.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

I like the idea of renaming the test.

Comment 3

[Ed Morley [:emorley]]

Obviously in an ideal world we'd use the mozlog json output, get path and test names from there & have a proper mapping of intermittent failures to tests (would avoid the mis-stars we see occasionally when two tests have the same filename but different path). But that vision is a fair way down the road.

Comment 4

[James Graham [:jgraham]]

The test can be renamed in m-c and we'll push that change upstream next time we sync.

Comment 5

[Ed Morley [:emorley]]

This was renamed somewhere yeah?

Comment 6

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

https://hg.mozilla.org/mozilla-central/file/4b9b12c248dc/testing/web-platform/tests/webaudio/the-audio-api/the-gainnode-interface/test.html is still here

Comment 7

[Ryan VanderMeulen [:RyanVM]]

I still see various wpt test.html files in the tree, but I'm wondering if this is still causing pain for sheriffs?

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Not that I know. For test.html, it's not an issue anymore.