1886114 - Perma [Tier 2 a11y only] TEST-UNEXPECTED-PASS | browser/components/firefoxview/tests/browser/browser_opentabs_more.js | We expect at least one assertion to fail because this test file is marked as fail-if in the manifest. -

Status: RESOLVED FIXED

(Firefox :: Firefox View, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=451230967&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/a6ftMF9BT-mJluVJBgVklg/runs/0/artifacts/public/logs/live_backing.log

---

TEST-PASS | browser/components/firefoxview/tests/browser/browser_opentabs_more.js | One about:blank tab open at the end up test cleanup - ["data:,Window0-Tab0"] deepEqual ["data:,Window0-Tab0"] - 
[task 2024-03-19T06:40:27.491Z] 06:40:27     INFO - Leaving test bound toggle_show_more_link
[task 2024-03-19T06:40:27.491Z] 06:40:27     INFO - Console message: 1710830427159	FirefoxView.TabsSetup	DEBUG	updateViewVisibility for instance: 9393884, visibility: hidden
[task 2024-03-19T06:40:27.492Z] 06:40:27     INFO - Console message: 1710830427159	FirefoxView.TabsSetup	DEBUG	Resetting timestamp and tabs pending flags as there are no visible views
[task 2024-03-19T06:40:27.493Z] 06:40:27     INFO - Console message: 1710830427159	FirefoxView.TabsSetup	DEBUG	updateViewVisibility for instance: 7696908, visibility: hidden
[task 2024-03-19T06:40:27.494Z] 06:40:27     INFO - Console message: 1710830427159	FirefoxView.TabsSetup	DEBUG	Resetting timestamp and tabs pending flags as there are no visible views
[task 2024-03-19T06:40:27.494Z] 06:40:27     INFO - Buffered messages finished
[task 2024-03-19T06:40:27.495Z] 06:40:27     INFO - TEST-UNEXPECTED-PASS | browser/components/firefoxview/tests/browser/browser_opentabs_more.js | We expect at least one assertion to fail because this test file is marked as fail-if in the manifest. - 
[task 2024-03-19T06:40:27.496Z] 06:40:27     INFO - TEST-INFO | expected FAIL
[task 2024-03-19T06:40:27.497Z] 06:40:27     INFO - GECKO(1062) | MEMORY STAT | vsize 20169MB | residentFast 615MB | heapAllocated 428MB
[task 2024-03-19T06:40:27.498Z] 06:40:27     INFO - TEST-OK | browser/components/firefoxview/tests/browser/browser_opentabs_more.js | took 3408ms

Comment 1

[Sandor Molnar[:smolnar]]

@Sam, hi could you take a look at these perma failures?
I think they are caused by latest patches in bug 1877678

Comment 2

[Sam Foster [:sfoster] (he/him)]

Ah yeah I think the test that fails a11y got moved to browser_opentabs_more.js in the patch on bug 1877678. I did add the fail-if = ["a11y_checks"] for the new file and I just need to remove it from the old entry.

It looks like bug 1877678 hasn't merged yet. I can update that patch or do a fast follow on this bug :smolnar?

Comment 3

[Sandor Molnar[:smolnar]]

Unfortunately, bug 1877678 has just been merged so you can craft a patch on this bug

Comment 4

[Sam Foster [:sfoster] (he/him)]

Actually, :ayeddi do you know what is going on here? AFAICT the fail-if = ["a11y_checks"] we had on browser_opentabs_cards.js was for the task that clicks the "view more" button. And in bug 1877678 I moved that test to a new file called browser_opentabs_more.js and took the fail-if with it, and removed the directive from the browser_opentabs_cards.js. There was no functional changes to the test itself, just moving it and giving it a longer timeout. So I'm confused now that I'm getting:

browser/components/firefoxview/tests/browser/browser_opentabs_more.js
  UNEXPECTED-PASS We expect at least one assertion to fail because this test file is marked as fail-if in the manifest

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1877678

Comment 6

[Anna Yeddi [:ayeddi]]

(In reply to Sam Foster [:sfoster] (he/him) from comment #4)

Actually, :ayeddi do you know what is going on here? AFAICT the fail-if = ["a11y_checks"] we had on browser_opentabs_cards.js was for the task that clicks the "view more" button. And in bug 1877678 I moved that test to a new file called browser_opentabs_more.js and took the fail-if with it, and removed the directive from the browser_opentabs_cards.js. There was no functional changes to the test itself, just moving it and giving it a longer timeout. So I'm confused now that I'm getting:

browser/components/firefoxview/tests/browser/browser_opentabs_more.js
  UNEXPECTED-PASS We expect at least one assertion to fail because this test file is marked as fail-if in the manifest

Interestingly, the updated version of the Show More link activation has resolved the issue that was there before: the link used to be clicked when it still had hidden attribute on - probably, right before it was rendered as shown/un-hidden, since the test harness is moving too fast. In this updated case, all the instances of the Show More links being clicked while they are visible and, therefore, accessible for the Accessibility API. We could safely remove the fail-if since it's working now as expected!

Comment 7

[Anna Yeddi [:ayeddi]]

Attachment: Bug 1886114 - Remove fail-if notation from the Firefox View test passing a11y-checks. r=sfoster!

the updated version of the Show More link activation has resolved the issue that was there before: the link used to be clicked when it still had hidden attribute on - probably, right before it was rendered as shown/un-hidden, since the test harness is moving too fast. In this updated case, all the instances of the Show More links being clicked while they are visible and, therefore, accessible for the Accessibility API. We could safely remove the fail-if since it's working now as expected

Comment 8

[Anna Yeddi [:ayeddi]]

sent a patch for that :) I like when a11y checks are fixed

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4b4f12e6b4af
Remove fail-if notation from the Firefox View test passing a11y-checks. r=sfoster,fxview-reviewers

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/4b4f12e6b4af

Comment 11

[Intermittent Failures Robot]

29 failures in 444 pushes (0.065 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 26
• mozilla-central: 3

Platform and build breakdown:

• linux1804-64-qr: 20
  • opt: 20
• linux1804-64-shippable-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1886114&startday=2024-03-19&endday=2024-03-19&tree=all

Comment 12

[Intermittent Failures Robot]

30 failures in 3791 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 26
• mozilla-central: 3
• try: 1

Platform and build breakdown:

• linux1804-64-qr: 21
  • opt: 21
• linux1804-64-shippable-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1886114&startday=2024-03-18&endday=2024-03-24&tree=all