1577093 - Perma fail Tier 2 /css/css-text/animations/<something>-interpolation.html | CSS Transitions/Animations: <text - see log>

So the fundamental issue seems to be that these tests aren't running on try[1], despite the web-platform-tests.yml suggesting they should. This means the sync bot is unable to pick up the aarch64-specific failures, which means we end up with bogus metadata.

[1] c.f. https://treeherder.mozilla.org/#/jobs?repo=try&revision=abdb3ebb089a248edb3521f4c52660785a04cfb3&searchStr=aarch64

Flags: needinfo?(jmaher)

Joel Maher ( :jmaher ) (UTC -8)

Comment 4

•

5 years ago

we don't have capacity to have these running by default on try- do you push with --full when selecting jobs on try?

Flags: needinfo?(jmaher)

James Graham [:jgraham]

Comment 5

•

5 years ago

No. I can although I assume that's evern slower. In general we can't have wpt tests in Tier-2+ configurations that we can't run on try. If the advice is that we need the bot to push with --full and start picking jobs that aren't run by default I guess that's possible (but also non-trivial since it presumably means we have to start filtering out ccov and similar jobs that aren't sensible to run for each sync).

Joel Maher ( :jmaher ) (UTC -8)

Comment 6

•

5 years ago

we are moving more jobs to tier-2 that have less relevant value or are on more unique configurations. Sheriffs will get notified with ccov tests are perma failing even though those are not run by default. Are there other options besides not running wpt at all on win/aarch64 ?

James Graham [:jgraham]

Comment 7

•

5 years ago

It's not about Tier-2 specifically, the point is just about "we need to be able to update the metadata for any configuration that's either Tier-1 or Tier-2". That happens through a try push, so we need to be able to run these things on try (the fact that it doesn't run on inbound can also be a problem since there are unfortunately some cases where we need post landing fixup, but that's really an issue for Tier-1 not Tier-2).

If the answer is "well in your case you need to run with --full" I guess that's acceptable. But we still don't want to run ccov jobs because we assume they aren't meaningfully different from non-ccov jobs on the same platforms, except perhaps for being slower. aarch64 is unlike that in that we are seeing a bunch of platform-specific issues, and expect to on general grounds (it's an actual different platform rather than just a different use of an existing platform). So running with --full means having a list of things we don't want to be run that has to be maintained. That maintainance has been tricky in the past so obviously if we can avoid it that's better.

James Graham [:jgraham]

Comment 8

•

5 years ago

As a concrete example of a problem, if we run with --full then we currently want to disable wpt fisson tests (presumably) since they aren't actually run at any tier level on any platform, but are enabled on try. But I assume at some point we will enable those as T2 like the mochitest fisson tests and then the try push configuration will need to be updated (but presumably won't actually get updated as part of turning them on since the code lives elsewhere and no one is aware it's a thing to consider).

Cosmin Sabou [:CosminS]

Comment 10

•

5 years ago

•

Edited

Had a go at these failures with a try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=23f8b66c8380624fc35aa313b4b06eaa8f3409cc
It definitely needs some adjustments but it's a start I hope.

Comment hidden (Intermittent Failures Robot)

Andreea Pavel [:apavel]

Updated

•

5 years ago

Summary: Perma fail Tier 2 /css/css-text/animations/letter-spacing-interpolation.html | CSS Transitions/Animations: <text - see log> → Perma fail Tier 2 /css/css-text/animations/<something>-interpolation.html | CSS Transitions/Animations: <text - see log>

Cosmin Sabou [:CosminS]

Updated

•

5 years ago

Keywords: intermittent-failure

Comment hidden (Intermittent Failures Robot)

Cosmin Sabou [:CosminS]

Comment 21

•

5 years ago

Jobs are green with the latest wpt-sync: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&group_state=expanded&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=windows%2Caarch64%2Cwpt&revision=fd891e83a7cd54038b0bcebb3ab85b6818e2550e&selectedJob=264857316

Flags: needinfo?(james)

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 24

•

5 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Status: NEW → RESOLVED

Closed: 5 years ago

Resolution: --- → INCOMPLETE

Bugzilla

Quick Search

Perma fail Tier 2 /css/css-text/animations/<something>-interpolation.html | CSS Transitions/Animations: <text - see log>

Categories

(Core :: CSS Parsing and Computation, defect, P5)

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, regression)

Crash Data

Security

(public)

User Story

Description

Comment 1

Comment 2

Comment 3

Comment 4

Comment 5

Comment 6

Comment 7

Comment 8

Comment 10

Comment 11

Updated

Updated

Comment 19

Comment 20

Comment 21

Comment 22

Comment 23

Comment 24