1806262 - Perma Mac headless TEST-UNEXPECTED-PASS | /webdriver/tests/<test>.py | test_fully_exit_fullscreen - expected FAIL

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=399979169&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WMrrXThHQ5aMX1-PTacFNw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-12-17T17:30:16.441Z] 17:30:16     INFO - TEST-PASS | /webdriver/tests/minimize_window/minimize.py | test_no_browsing_context 
[task 2022-12-17T17:30:16.441Z] 17:30:16     INFO - TEST-UNEXPECTED-PASS | /webdriver/tests/minimize_window/minimize.py | test_fully_exit_fullscreen - expected FAIL
[task 2022-12-17T17:30:16.441Z] 17:30:16     INFO - TEST-INFO | expected FAIL
[task 2022-12-17T17:30:16.443Z] 17:30:16     INFO - ...
[task 2022-12-17T17:30:16.443Z] 17:30:16     INFO - TEST-OK | /webdriver/tests/minimize_window/minimize.py | took 18896ms

Comment 1

[Natalia Csoregi [:nataliaCs]]

Another log example:

• TEST-UNEXPECTED-PASS | /webdriver/tests/set_window_rect/set.py | test_fully_exit_fullscreen - expected FAIL

They all have in common: test_fully_exit_fullscreen - expected FAIL

Only fails on tier 2 on headless.

Regressed by Bug 1631735.

Brad, can you take a look please?
Thank you.

Comment 2

[Brad Werth [:bradwerth]]

Attachment: Bug 1806262: Mark macOS minimize fullscreen tests as intermittent.

These tests fail on macOS and pass on macOS headless. Until Bug 1802192
resolves the fundamental cause of the failure, this change will mark these
tests as intermittent (either pass or fail).

Comment 3

[Intermittent Failures Robot]

29 failures in 142 pushes (0.204 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 26
• mozilla-central: 3

Platform and build breakdown:

• macosx1015-64-qr: 27
  • opt: 22
  • debug: 5
• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806262&startday=2022-12-17&endday=2022-12-17&tree=all

Comment 4

[Intermittent Failures Robot]

65 failures in 3558 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 33
• mozilla-central: 27
• try: 5

Platform and build breakdown:

• macosx1015-64-qr: 56
  • debug: 29
  • opt: 27
• macosx1015-64-shippable-qr: 7
  • opt: 7
• macosx1015-64-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806262&startday=2022-12-12&endday=2022-12-18&tree=all

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is only for Mac headless. Lets make it clearer in the bug's summary.

Comment 6

[Intermittent Failures Robot]

45 failures in 631 pushes (0.071 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 28
• mozilla-central: 8
• try: 9

Platform and build breakdown:

• macosx1015-64-qr: 37
  • debug: 23
  • opt: 14
• macosx1015-64-shippable-qr: 6
  • opt: 6
• macosx1015-64-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806262&startday=2022-12-19&endday=2022-12-19&tree=all

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1806262 - [wdspec] Mark MacOS headful tests for minimize and set window rect after fullscreen as failing.

These tests fail on macOS and pass on macOS headless. Until Bug 1802192
resolves the fundamental cause of the failure, this change will mark these
tests as fail for headful..

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be14f7e8712b
[wdspec] Mark MacOS headful tests for minimize and set window rect after fullscreen as failing. r=webdriver-reviewers,jdescottes

Comment 9

[Sandor Molnar[:smolnar]]

Backed out because bug 1631735 was backed out.

Backout link: https://hg.mozilla.org/integration/autoland/rev/5417477f71ca5891b9ec13d1c63fb406983d368b

LE. Fixed by backing out bug 1631735

Comment 10

[Intermittent Failures Robot]

54 failures in 2999 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #9 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 37
• mozilla-central: 8
• try: 9

Platform and build breakdown:

• macosx1015-64-qr: 46
  • debug: 27
  • opt: 19
• macosx1015-64-shippable-qr: 6
  • opt: 6
• macosx1015-64-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1806262&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 11

[BMO Automation]

Moving bug to Remote Protocol::Marionette component per bug 1815831.