1885370 - Perma [Tier 2] AttributeError: 'NoneType' object has no attribute 'default_expected'

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=450729409&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YG_Wk1pIQN2bWG859ESybg/runs/0/artifacts/public/logs/live_backing.log

---

{"action": "log", "time": 1710406795369, "thread": "MainThread", "pid": 19, "source": "web-platform-tests", "message": "  /css/filter-effects/backdrop-filter-basic-opacity-2.html ('PASS', []) ('PASS', ['ERROR']) True", "level": "DEBUG"}
[task 2024-03-14T08:59:55.378Z] Traceback (most recent call last):
[task 2024-03-14T08:59:55.378Z]   File "/builds/worker/checkouts/gecko/testing/web-platform/mach_commands.py", line 397, in run
[task 2024-03-14T08:59:55.378Z]     return fissionregressions.run(logger, src_root, obj_root, **kwargs)
[task 2024-03-14T08:59:55.378Z]   File "/builds/worker/checkouts/gecko/testing/web-platform/fissionregressions.py", line 463, in run
[task 2024-03-14T08:59:55.378Z]     baseline_results = allowed_results(baseline_test, name)
[task 2024-03-14T08:59:55.378Z]   File "/builds/worker/checkouts/gecko/testing/web-platform/fissionregressions.py", line 204, in allowed_results
[task 2024-03-14T08:59:55.378Z]     return test.expected(subtest), test.known_intermittent(subtest)
[task 2024-03-14T08:59:55.378Z]   File "/builds/worker/checkouts/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/wpttest.py", line 435, in expected
[task 2024-03-14T08:59:55.378Z]     default = self.subtest_result_cls.default_expected
[task 2024-03-14T08:59:55.378Z] AttributeError: 'NoneType' object has no attribute 'default_expected'
[task 2024-03-14T08:59:55.378Z] > /builds/worker/checkouts/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/wpttest.py(435)expected()
[task 2024-03-14T08:59:55.378Z] -> default = self.subtest_result_cls.default_expected
[taskcluster:error] Task timeout after 2700 seconds. Force killing container.
[taskcluster 2024-03-14 09:37:03.906Z] === Task Finished ===
[taskcluster 2024-03-14 09:37:03.907Z] Unsuccessful task run with exit code: -1 completed in 2701.244 seconds

Comment 1

[Sandor Molnar[:smolnar]]

@Sasha, could you check out this perma wpt lint failure?
started to fail after the latest wpt-sync

Comment 2

[Alexandra Borovova [:Sasha]]

Attachment: Bug 1885370 - Add ReftestSubtestResult class with default_expected.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1885040

Comment 4

[Intermittent Failures Robot]

28 failures in 3849 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #35 most frequent failure this week.

Repository breakdown:

• autoland: 23
• mozilla-central: 5

Platform and build breakdown:

• lint: 28
  • opt: 28

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1885370&startday=2024-03-11&endday=2024-03-17&tree=all

Comment 5

[James Graham [:jgraham]]

But reftests shouldn't have subtests. So I think the patch is kind of wallpapering over the problem, which I suspect is really an error in the data.

Maybe https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/wpttest.py#435 should raise a ValueError in this case with a message that makes the error more obvious? And perhaps we should just ignore the error in fissionregressions.py in that case, since it seems to be the only tool that actually errors out here, rather than just not trying to access the problematic data.

But in any case I'm going to put up a patch to fix the underlying issue, which seems to be caused by rewriting a testharness test as a reftest, which is a scenario the sync doesn't fully handle (should remove all subtest expectation data in that case).

Comment 6

[James Graham [:jgraham]]

Attachment: Bug 1885370 - Remove subtest metadata from reftest,

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ce6e019086c8
Remove subtest metadata from reftest, r=Sasha

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1885040

Comment 9

[Sandor Molnar[:smolnar]]

Backed out for causing perma wpt canvas failures @ /html/canvas/element/filters/2d.filter.canvasFilterObject.<...>

Backout link: https://hg.mozilla.org/integration/autoland/rev/450538ee1ca2a3ef8685321eee6e2f8a14e47aa6

Push with failures

Failure log ->

Comment 10

[James Graham [:jgraham]]

Updated the patch.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a960e8bc0601
Remove subtest metadata from reftest, r=Sasha

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/a960e8bc0601

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:Sasha, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox125 to wontfix.

For more information, please visit BugBot documentation.

Comment 14

[Intermittent Failures Robot]

27 failures in 3791 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 19
• mozilla-beta: 1
• mozilla-central: 7

Platform and build breakdown:

• lint: 27
  • opt: 27

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1885370&startday=2024-03-18&endday=2024-03-24&tree=all

Comment 15

[Intermittent Failures Robot]

1 failures in 3235 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• lint: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1885370&startday=2024-04-08&endday=2024-04-14&tree=all

Comment 16

[Intermittent Failures Robot]

1 failures in 4252 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• lint: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1885370&startday=2024-04-15&endday=2024-04-21&tree=all

Comment 17

[Intermittent Failures Robot]

1 failures in 4336 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• lint: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1885370&startday=2024-04-22&endday=2024-04-28&tree=all