1660896 - wpt6 failures with clang 11

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect)

Description

[(Away)]

Baseline with clang 9: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&tier=1%2C2%2C3&revision=597af269d03dfd2c2926134b0a48b9929a330711

With clang 11 rc2: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&tier=1%2C2%2C3&revision=5cb8a3deeb92dfa83ad3f91d90cd1fa773b609ad

/editing/run/bold.html?1-1000 | [["stylewithcss","false"],["bold",""]] "<span>[foo</span> <span>bar]</span>" compare innerHTML - assert_equals: Unexpected innerHTML (after normalizing inline style) expected "<b><span>foo</span> <span>bar</span></b>" but got "<b>foo bar</b>"

Only seen on Linux x64 debug builds.

Comment 1

[James Graham [:jgraham]]

Hmm, so the way to run these tests locally is something like mach wpt /editing/run/bold.html?1-1000. It takes command line options for things like running Firefox under a debugger, similar to other test suites.

Looking at the ini files [1] it seems we have some failures, but those aren't platform-specific or anything so it's not like this is something that fails on other platforms but previously passed on Linux 64 debug for mysterious reasons. Given that the unexpected fails are in innerHTML comparisons, it could be worth talking to hsivonen in case this is some weirdness in the HTML parser. I don't really know who, if anyone, owns editing these days.

If you can't make any progress and want me to dig into the tests more just let me know; from a quick glance they're pretty complex so it's not trivial to say what's going on here.

[1] https://searchfox.org/mozilla-central/source/testing/web-platform/meta/editing/run/bold.html.ini

Comment 2

[(Away)]

I attempted to debug this and didn't get anywhere.

This bug came up in my team's weekly meeting and the general consensus was that we can't afford to block on this. Our engineering capacity is really limited at the moment, and this test failure is only seen on debug builds on one OS. As much as I want to solve the technical puzzle at the bottom of this, that's just not a practical reality right now, so unfortunately let's disable the test.

Comment 3

[(Away)]

Attachment: Adjust test expectations for editing WPT on Linux debug

A few test cases fail under clang-11 on Linux debug builds only. As described in the bug, we unfortunately don't have the bandwidth to investigate, so this patch accepts the failures.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9c8388c30032
Adjust test expectations for editing WPT on Linux debug r=jgraham

Comment 5

[Razvan Maries]

Backed out for perma failures. See https://bugzilla.mozilla.org/show_bug.cgi?id=1660340#c3

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/94c19736f73e
Adjust test expectations for editing WPT on Linux debug r=jgraham

Comment 7

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/94c19736f73e

Comment 8

[Narcis Beleuzu [:NarcisB]]

Backed out for causing Btime failures on Android 7.0 (bug 1660340)

Backout link: https://hg.mozilla.org/integration/autoland/rev/592e556af3ecee3700c80dac778585e93cbe755f
Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=314317947&repo=autoland&lineNumber=1864

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backout merged: https://hg.mozilla.org/mozilla-central/rev/592e556af3ec

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c7ec33304139
Adjust test expectations for editing WPT on Linux debug r=jgraham

Comment 11

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/c7ec33304139