1898598 - Intermittent browser/components/translations/tests/browser/browser_translations_select_telemetry_translation_failure_ui.js | single tracking bug

Status: RESOLVED FIXED

(Firefox :: Translations, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=459398288&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/F9Y1JXkcTjql7SdQ4C_6zw/runs/0/artifacts/public/logs/live_backing.log

---

TEST-PASS | browser/components/translations/tests/browser/browser_translations_select_telemetry_translation_failure_ui.js | Expected select-translations-panel-to selection to match 'en' - 
[task 2024-05-23T18:41:27.984Z] 18:41:27     INFO - Buffered messages finished
[task 2024-05-23T18:41:27.985Z] 18:41:27     INFO - TEST-UNEXPECTED-FAIL | browser/components/translations/tests/browser/browser_translations_select_telemetry_translation_failure_ui.js | Uncaught exception in test bound test_select_translations_panel_telemetry_translation_failure_ui - undefined - timed out after 200 tries.
[task 2024-05-23T18:41:27.985Z] 18:41:27     INFO - Leaving test bound test_select_translations_panel_telemetry_translation_failure_ui
[task 2024-05-23T18:41:27.986Z] 18:41:27     INFO - GECKO(3433) | MEMORY STAT | vsize 3700MB | residentFast 515MB | heapAllocated 304MB
[task 2024-05-23T18:41:27.986Z] 18:41:27     INFO - TEST-OK | browser/components/translations/tests/browser/browser_translations_select_telemetry_translation_failure_ui.js | took 24764ms
[task 2024-05-23T18:41:27.987Z] 18:41:27     INFO - Not taking screenshot here: see the one that was previously logged
[task 2024-05-23T18:41:27.988Z] 18:41:27     INFO - TEST-UNEXPECTED-FAIL | browser/components/translations/tests/browser/browser_translations_select_telemetry_translation_failure_ui.js | Found an unexpected tab at the end of test run: https://example.com/browser/toolkit/components/translations/tests/browser/translations-tester-select.html - 
[task 2024-05-23T18:41:27.988Z] 18:41:27     INFO - GECKO(3433) | [Child 9665, Main Thread] WARNING: NS_ENSURE_TRUE(mDoneSetup) failed: file /builds/worker/checkouts/gecko/editor/composer/nsEditingSession.cpp:1163
[task 2024-05-23T18:41:27.989Z] 18:41:27     INFO - GECKO(3433) | [Child 3693: Main Thread]: I/DocShellAndDOMWindowLeak ++DOCSHELL 7f4610d5c400 == 1 [pid = 3693] [id = 83]
[task 2024-05-23T18:41:27.989Z] 18:41:27     INFO - GECKO(3433) | [Child 3693: Main Thread]: I/DocShellAndDOMWindowLeak ++DOMWINDOW == 165 (7f460fb2b200) [pid = 3693] [serial = 250] [outer = 0]
[task 2024-05-23T18:41:27.990Z] 18:41:27     INFO - GECKO(3433) | [Child 3693: Main Thread]: I/DocShellAndDOMWindowLeak ++DOMWINDOW == 166 (7f4610d5c800) [pid = 3693] [serial = 251] [outer = 7f460fb2b200]
[task 2024-05-23T18:41:28.020Z] 18:41:28     INFO - checking window state
[task 2024-05-23T18:41:28.120Z] 18:41:28     INFO - TEST-START | browser/components/translations/tests/browser/browser_translations_select_telemetry_unsupported_language_ui.js

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:nordzilla, since you are the author of the regressor, bug 1870368, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1870368

Comment 3

[Erik Nordin [:nordzilla]]

This intermittent seems to be fairly regular in CI, but it doesn't produce as well locally.

Given how often it's happening, and that it's a new test I'm going to try to prioritize resolving this one.

Comment 4

[Intermittent Failures Robot]

25 failures in 829 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 4
• autoland: 15
• try: 6

Platform and build breakdown:

• windows11-64-2009-ccov-qr: 1
  • opt: 1
• windows11-64-2009-qr: 9
  • debug: 9
• linux1804-64-qr: 4
  • debug: 4
• windows11-64-2009-asan-qr: 3
  • opt: 3
• windows11-32-2009-qr: 5
  • debug: 5
• macosx1015-64-qr: 2
  • debug: 2
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-24&endday=2024-05-24&tree=all

Comment 5

[Intermittent Failures Robot]

16 failures in 132 pushes (0.121 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 4
• try: 4

Platform and build breakdown:

• windows11-64-2009-qr: 8
  • debug: 8
• windows11-64-2009-asan-qr: 1
  • opt: 1
• macosx1015-64-qr: 1
  • debug: 1
• linux1804-64-tsan-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 3
  • opt: 3
• macosx1015-64-devedition-qr: 1
  • opt: 1
• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-25&endday=2024-05-25&tree=all

Comment 6

[Intermittent Failures Robot]

64 failures in 3995 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #17 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 39
• try: 13
• mozilla-central: 12

Platform and build breakdown:

• windows11-64-2009-qr: 24
  • debug: 23
  • opt: 1
• linux1804-64-qr: 13
  • debug: 13
• windows11-64-2009-ccov-qr: 1
  • opt: 1
• windows11-32-2009-qr: 8
  • debug: 7
  • opt: 1
• windows11-64-2009-asan-qr: 7
  • opt: 7
• macosx1015-64-qr: 5
  • opt: 2
  • debug: 3
• linux1804-64-tsan-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 4
  • opt: 4
• macosx1015-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-20&endday=2024-05-26&tree=all

Comment 7

[Intermittent Failures Robot]

15 failures in 651 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• try: 1
• mozilla-central: 5

Platform and build breakdown:

• macosx1015-64-qr: 3
  • opt: 1
  • debug: 2
• windows11-32-2009-qr: 3
  • debug: 3
• windows11-64-2009-asan-qr: 1
  • opt: 1
• linux1804-64-qr: 1
  • debug: 1
• windows11-64-2009-qr: 7
  • debug: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-27&endday=2024-05-27&tree=all

Comment 8

[Erik Nordin [:nordzilla]]

Attachment: Bug 1898598 - Split up translation_failure_ui test r=#translations-reviewers!

Splits up the test case in browser_translations_select_telemetry_translation_failure_ui.js
in hopes that it cuts down on the intermittent failure in CI.

Comment 9

[Intermittent Failures Robot]

36 failures in 851 pushes (0.042 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 5
• autoland: 22
• mozilla-central: 9

Platform and build breakdown:

• linux1804-64-qr: 7
  • debug: 7
• windows11-32-2009-qr: 5
  • debug: 5
• macosx1015-64-qr: 7
  • debug: 2
  • opt: 5
• windows11-64-2009-asan-qr: 6
  • opt: 6
• windows11-64-2009-qr: 10
  • debug: 10
• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-28&endday=2024-05-28&tree=all

Comment 10

[Intermittent Failures Robot]

27 failures in 861 pushes (0.031 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 16
• mozilla-central: 8
• try: 3

Platform and build breakdown:

• linux1804-64-qr: 8
  • debug: 8
• windows11-64-2009-qr: 9
  • debug: 9
• macosx1015-64-qr: 7
  • debug: 6
  • opt: 1
• windows11-64-2009-asan-qr: 1
  • opt: 1
• macosx1100-64-shippable-qr: 1
  • opt: 1
• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-29&endday=2024-05-29&tree=all

Comment 11

[Intermittent Failures Robot]

27 failures in 810 pushes (0.033 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 11
• try: 5
• mozilla-central: 11

Platform and build breakdown:

• macosx1015-64-qr: 4
  • debug: 2
  • opt: 2
• windows11-64-2009-qr: 10
  • debug: 10
• linux1804-64-qr: 5
  • debug: 5
• windows11-32-2009-qr: 2
  • debug: 2
• windows11-64-2009-asan-qr: 3
  • opt: 3
• macosx1015-64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-30&endday=2024-05-30&tree=all

Comment 12

[Erik Nordin [:nordzilla]]

I'm going to land the patch to split up this test, but I don't think it fixes it entirely, so I'm adding leave-open.

Comment 13

[Chris H-C :chutten]

FOG's IPC and test methods are very synchronous and deterministic, so intermittents like this should be avoidable.

Current hypothesis: Maybe the "events" ping is being submitted during the test run, gobbling the recorded event out of storage before it is asserted. FOG Logging being on during a failed run will tell us if there's any merit to that, and also give us clues to follow if it isn't.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/729e400f7b0b
Split up translation_failure_ui test r=translations-reviewers,gregtatum

Comment 15

[Chris H-C :chutten]

Thank you, :nordzilla, for catching a failure in action with logging enabled.

This log shows evidence consistent with my hypothesis: the event ping is sent on line 15323 due to a user-interaction-inactive being received after more than 2mins of "user activity" due to previous tests. This happens after the open event is recorded but before its presence is asserted, which is some wickedly bad luck.

This also explains why it doesn't happen locally: running locally we don't give it ten minutes per run. Not enough time to grow idle, or if it becomes idle it's after too short of an active period to trigger the "events" ping (which shares the "baseline" pings schedule).

We can paper over this failure immediately with a Services.obs.notifyObservers("user-interaction-inactive"); at the beginning of this test (triggers the "events" ping early, or avoids triggering it at all. Both are acceptable), or by doing what we did in bug 1690728 and giving dom.events.user_interaction_interval a value in excess of the length the app is open (meaning EventStateManager won't ever claim the user's gone active or idle for the duration). Either will ensure the "events" ping will not be sent during the test.

A more appropriate (less hacky) solution is being pursued in bug 1900085, so if we go one of those above routes in the meantime please do leave a note there (and reference the bug number in a comment next to the hack) so we can undo it later.

Comment 16

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/729e400f7b0b

Comment 17

[Intermittent Failures Robot]

17 failures in 768 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• try: 3
• mozilla-central: 4

Platform and build breakdown:

• linux1804-64-qr: 3
  • debug: 3
• windows11-64-2009-asan-qr: 3
  • opt: 3
• windows11-64-2009-qr: 4
  • debug: 4
• macosx1015-64-shippable-qr: 3
  • opt: 3
• macosx1015-64-qr: 3
  • debug: 2
  • opt: 1
• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-31&endday=2024-05-31&tree=all

Comment 18

[Intermittent Failures Robot]

125 failures in 4202 pushes (0.03 failures/push) were associated with this bug in the last 7 days.

This is the #4 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• try: 20
• autoland: 68
• mozilla-central: 37

Platform and build breakdown:

• linux1804-64-qr: 25
  • debug: 25
• windows11-32-2009-qr: 11
  • debug: 11
• macosx1015-64-qr: 24
  • opt: 10
  • debug: 14
• windows11-64-2009-qr: 41
  • debug: 41
• windows11-64-2009-asan-qr: 15
  • opt: 15
• macosx1015-64-shippable-qr: 6
  • opt: 6
• macosx1100-64-shippable-qr: 2
  • opt: 2
• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-05-27&endday=2024-06-02&tree=all

Comment 19

[Intermittent Failures Robot]

2 failures in 3649 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform and build breakdown:

• windows11-64-2009-qr: 1
  • debug: 1
• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1898598&startday=2024-06-03&endday=2024-06-09&tree=all

Comment 20

[Erik Nordin [:nordzilla]]

The two pushes in comment 19 must be based on an old branch, because I split up this test file into two separate files, and a test file with this exact name no longer exists in tree.