1775917 - Intermittent browser/components/search/test/browser/browser_search_telemetry_sources.js | single tracking bug

Status: RESOLVED FIXED

(Firefox :: Search, defect, P5)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Additional information about this bug failures and frequency patterns can be found by running: ./mach test-info failure-report --bug 1775917

Comment 2

[Intermittent Failures Robot]

1 failures in 3312 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr91: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-06-27&endday=2022-07-03&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 3497 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-07-18&endday=2022-07-24&tree=all

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 6

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=387317528&repo=autoland

Comment 7

[Intermittent Failures Robot]

1 failures in 2916 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows7-32-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-08-15&endday=2022-08-21&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 3293 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-2004-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-08-22&endday=2022-08-28&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 3537 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-32-2004-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-08-29&endday=2022-09-04&tree=all

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 11

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=396026497&repo=autoland

Comment 12

[Intermittent Failures Robot]

1 failures in 3445 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2022-11-07&endday=2022-11-13&tree=all

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 14

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=402532757&repo=autoland

Comment 15

[Intermittent Failures Robot]

1 failures in 3717 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-2004-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-01-16&endday=2023-01-22&tree=all

Comment 16

[Intermittent Failures Robot]

2 failures in 3766 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-02-06&endday=2023-02-12&tree=all

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 18

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=416823612&repo=autoland

Comment 19

[Intermittent Failures Robot]

23 failures in 860 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 17
• mozilla-central: 6

Platform and build breakdown:

• macosx1015-64-qr: 2
  • debug: 2
• windows11-32-2009-qr: 2
  • debug: 2
• windows11-64-2009-qr: 9
  • debug: 9
• linux1804-64-qr: 5
  • debug: 5
• linux1804-64-asan-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-25&endday=2023-05-25&tree=all

Comment 20

[Intermittent Failures Robot]

37 failures in 699 pushes (0.053 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 27
• mozilla-central: 8
• try: 2

Platform and build breakdown:

• linux1804-64-qr: 12
  • debug: 12
• windows11-32-2009-qr: 8
  • debug: 8
• windows11-64-2009-qr: 11
  • debug: 11
• linux1804-64-asan-qr: 2
  • opt: 2
• macosx1015-64-qr: 3
  • opt: 3
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-26&endday=2023-05-26&tree=all

Comment 21

[Intermittent Failures Robot]

79 failures in 3946 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #4 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 54
• mozilla-central: 17
• try: 8

Platform and build breakdown:

• macosx1015-64-qr: 7
  • opt: 4
  • debug: 3
• windows11-32-2009-qr: 16
  • debug: 16
• windows11-64-2009-qr: 27
  • debug: 27
• linux1804-64-qr: 19
  • debug: 19
• linux1804-64-asan-qr: 9
  • opt: 9
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-22&endday=2023-05-28&tree=all

Comment 22

[Mark Banner (:standard8)]

James, please can you take a look?

Comment 23

[James Teow [:jteow]]

I think it's because the Persist Search tip is appearing. Kind of strange that it only popped up recently.

Here is the log of the results of the urlbar from getFirstSuggestionIndex when I get a test failure on TH:

[task 2023-05-29T12:26:31.940Z] 12:26:31     INFO - GECKO(20387) | console.log: Result: {"result":{"type":7,"source":5,"rowIndex":0,"heuristic":false,"exposureResultType":"","exposureResultHidden":false,"payload":{"type":"searchTip_persist","buttons":[{"l10n":{"id":"urlbar-search-tips-confirm-short"}}],"icon":"chrome://global/skin/icons/lightbulb.svg","titleL10n":{"id":"urlbar-search-tips-persist"}},"payloadHighlights":{"type":[],"buttons":[],"icon":[]},"providerName":"UrlbarProviderSearchTips","providerType":2},"url":null,"postData":null,"type":7,"source":5,"heuristic":false,"autofill":false,"image":"chrome://global/skin/icons/lightbulb.svg","title":"","tags":[],"displayed":{"title":"Searching just got simpler. Try making your search more specific here in the address bar. To show the URL instead, visit Search, in settings.","action":"","url":"","typeIcon":"none"},"element":{"action":{},"row":{"_elements":{},"_buttons":{},"result":{"type":7,"source":5,"rowIndex":0,"heuristic":false,"exposureResultType":"","exposureResultHidden":false,"payload":{"type":"searchTip_persist","buttons":[{"l10n":{"id":"urlbar-search-tips-confirm-short"}}],"icon":"chrome://global/skin/icons/lightbulb.svg","titleL10n":{"id":"urlbar-search-tips-persist"}},"payloadHighlights":{"type":[],"buttons":[],"icon":[]},"providerName":"UrlbarProviderSearchTips","providerType":2},"_content":{}},"separator":{},"title":{"_tooltip":""},"url":{"_tooltip":""}}}

It's strange that it's appearing in this situation when we haven't hit a search page yet.

Will continue investigating.

Comment 24

[James Teow [:jteow]]

The tests enter "searchSuggestion" in the Urlbar and wait for the Urlbar to open, at which point it queries the list of results and looks for a search suggestion. If no search results are available, it'll return -1.

For reference, here is a commit here where I add some logging to help understand the failures.

The reason why it's failing is because the Persist Search Tip is the result in the Urlbar: https://treeherder.mozilla.org/logviewer?job_id=417409193&repo=try&lineNumber=42488 . This is strange because we haven't landed on a SERP and this is the first test in this file. I wonder if it's caused from another test?

I added a log right before we commit to showing a search tip. In the same job, the search tip is schedule to show in browser/components/search/test/browser/browser_rich_suggestions.js. Locally, I am able to confirm it'll pass the checks to generate a search tip. That test gets run several tests before browser_search_telemetry_sources.js, I not expecting it to remain in the Urlbar so far after.

I believe that test is causing the issue because I haven't touched the persist search tip in months and I don't think anything in browser_search_telemetry_sources.js would trigger this issue.

I noticed browser_rich_suggestions.js had a commit on May 25th at 10:50 and the surge of failures started on May 25th at 14:14 onward. The change indeed does a search with a term suggested in the Urlbar, and then immediately asserts telemetry is recorded properly.

I'll try testing disabling the persist search tip in in the rich suggestions test to see if it alleviates the issue, since I can regularly hit this error in running these tests on Try. It might be the case that we should avoid enabling the persist search tip on all search tests since it can cause issues such as these, but I'd like to know why one test is affecting another, that seems iffy.

Comment 25

[Intermittent Failures Robot]

19 failures in 455 pushes (0.042 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 16
• try: 3

Platform and build breakdown:

• windows11-32-2009-qr: 2
  • debug: 2
• windows11-64-2009-qr: 6
  • debug: 6
• linux1804-64-qr: 7
  • debug: 7
• linux1804-64-asan-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-29&endday=2023-05-29&tree=all

Comment 26

[James Teow [:jteow]]

Attachment: Bug 1775917 - Disable the persist search tip from appearing in browser_rich_suggestions.js - r?daleharvey

Comment 27

[Intermittent Failures Robot]

26 failures in 707 pushes (0.037 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 21
• mozilla-central: 4
• try: 1

Platform and build breakdown:

• windows11-32-2009-qr: 3
  • debug: 3
• linux1804-64-qr: 6
  • debug: 6
• linux1804-64-asan-qr: 4
  • opt: 4
• windows11-64-2009-qr: 10
  • debug: 10
• macosx1015-64-qr: 2
  • debug: 2
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-30&endday=2023-05-30&tree=all

Comment 28

[Intermittent Failures Robot]

38 failures in 854 pushes (0.044 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 31
• mozilla-central: 6
• try: 1

Platform and build breakdown:

• windows11-32-2009-qr: 9
  • debug: 9
• windows11-64-2009-asan-qr: 1
  • opt: 1
• windows11-64-2009-qr: 17
  • debug: 17
• linux1804-64-asan-qr: 4
  • opt: 4
• linux1804-64-qr: 7
  • debug: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-31&endday=2023-05-31&tree=all

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9718fca142ae
Disable the persist search tip from appearing in browser_rich_suggestions.js - r=daleharvey

Comment 30

[Intermittent Failures Robot]

27 failures in 657 pushes (0.041 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 17
• mozilla-central: 8
• try: 2

Platform and build breakdown:

• linux1804-64-qr: 10
  • debug: 10
• windows11-64-2009-qr: 10
  • debug: 10
• macosx1015-64-qr: 2
  • opt: 1
  • debug: 1
• windows11-32-2009-qr: 1
  • debug: 1
• linux1804-64-asan-qr: 3
  • opt: 3
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-06-01&endday=2023-06-01&tree=all

Comment 31

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/9718fca142ae

Comment 32

[Intermittent Failures Robot]

111 failures in 3471 pushes (0.032 failures/push) were associated with this bug in the last 7 days.

This is the #6 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 85
• try: 8
• mozilla-central: 18

Platform and build breakdown:

• windows11-32-2009-qr: 16
  • debug: 16
• windows11-64-2009-qr: 43
  • debug: 43
• linux1804-64-qr: 30
  • debug: 30
• linux1804-64-asan-qr: 15
  • opt: 15
• macosx1015-64-qr: 4
  • debug: 3
  • opt: 1
• windows11-64-2009-asan-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1775917&startday=2023-05-29&endday=2023-06-04&tree=all