Intermittent test_toolbars.py TestAutoCompleteResults.test_matching_text | TimeoutException: Timed out after 5.0 seconds

NEW
Unassigned

Status

Testing
Marionette
8 months ago
6 months ago

People

(Reporter: Treeherder Bug Filer, Unassigned, NeedInfo)

Tracking

({intermittent-failure, leave-open})

Version 3
intermittent-failure, leave-open
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [stockwell disabled])

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

[task 2017-06-01T20:39:43.554992Z] 20:39:43     INFO -  1496349583552	Marionette	TRACE	14 -> [0,208,"executeScript",{"scriptTimeout":null,"newSandbox":true,"args":[],"filename":"toolbars.py","script":"\n          Components.utils.import(\"resource://gre/modules/Services.jsm\");\n\n          let win = Services.focus.activeWindow;\n          if (win) {\n            return win.gURLBar.controller.searchStatus >=\n                   Components.interfaces.nsIAutoCompleteController.STATUS_COMPLETE_NO_MATCH;\n          }\n\n          return null;\n        ","sandbox":"default","line":391}]
[task 2017-06-01T20:39:43.557087Z] 20:39:43     INFO -  1496349583556	Marionette	TRACE	14 <- [1,208,null,{"value":false}]

Looks like the search via the urlbar runs a very long time, or not sure what's not causing the status to change.

Comment 2

7 months ago
1 failures in 892 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-06-19&endday=2017-06-25&tree=all

Comment 3

7 months ago
2 failures in 718 pushes (0.003 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2

Platform breakdown:
* osx-10-10: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 4

6 months ago
3 failures in 656 pushes (0.005 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* try: 1
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* windows7-32-vm: 1
* windows10-64-vm: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-07-03&endday=2017-07-09&tree=all

Comment 5

6 months ago
17 failures in 720 pushes (0.024 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 7
* autoland: 4
* mozilla-central: 3
* mozilla-beta: 2
* try: 1

Platform breakdown:
* linux32: 6
* windows10-64: 4
* windows7-32: 3
* linux64: 3
* windows10-64-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-07-10&endday=2017-07-16&tree=all
this picked up frequency on July 12th across the board.
Marco, did we change something for the autocomplete recently?
Flags: needinfo?(mak77)

Comment 8

6 months ago
The Location bar code changes almost every week in preparation for Photon, so the question is a bit generic :)
I don't know of a specific change to the location bar that happened on 12th (some happened on 14th).
Something that recently happened is work to delay most startup code to after the first paint, looks like this code relies on default bookmarks being imported, maybe they were not yet at that point? I'm just guessing.
For example bug 1371677 landed on 12th.

Can you give me more details about the point where the test is failing?
Flags: needinfo?(mak77) → needinfo?(hskupin)

Comment 9

6 months ago
I'd probably suggest waiting for places-browser-init-complete after startup, if you're not doing that yet. It should ensure you default bookmarks are there.
:whimboo is on pto, redirecting to :automatedtester since this is a high frequency failure and we want to see this resolved in the next 2 weeks.
Flags: needinfo?(dburns)
Whiteboard: [stockwell needswork]

Comment 11

6 months ago
50 failures in 822 pushes (0.061 failures/push) were associated with this bug in the last 7 days. 

This is the #29 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 27
* mozilla-inbound: 10
* try: 6
* mozilla-beta: 3
* pine: 2
* mozilla-central: 2

Platform breakdown:
* windows10-64: 17
* linux64: 15
* windows7-32: 9
* linux32: 8
* windows10-64-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-07-17&endday=2017-07-23&tree=all
:mak

Is this test covered by other test suites? I am wondering if this is a duplicate or if this test is actually worthwhile to the awesomebar folk.
Flags: needinfo?(dburns) → needinfo?(mak77)

Comment 13

6 months ago
17 failures in 152 pushes (0.112 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 5
* autoland: 5
* try: 3
* pine: 3
* mozilla-beta: 1

Platform breakdown:
* windows10-64: 5
* windows7-32: 4
* linux32: 3
* linux64-stylo: 2
* linux64: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-07-24&endday=2017-07-24&tree=all
From what I see the test searches for a string and then it checks that results match that string either in title or url.

We surely have tests in toolkit/components/places/tests/unifiedcomplete covering the fact that adding N visits/bookmarks and then searching for a string that appears only in M < N results, we only get the expected M results and not the nonmatching ones.
In particular toolkit/components/places/tests/unifiedcomplete/test_word_boundary_search.js that checks we match on word boundaries.
Those make me pretty confident that we return things that are matching.

The only thing this test is adding we don't seem to have is checking those matching strings are shown in the UI. The value isn't that high, since the presentation may also change in a way to not directly show what the urlbar is matching, and in the future we may match umlauts and diacritics differently and this may break again.
I also assume a regression in returning totally unrelated results would be easily catched by Nightly population.

To sum up, I'd not complain loudly if this would be removed. If instead you prefer to fix it to retain that small amount of added testing, I see 2 ways:
1. Wait for places-browser-init-complete before proceeding with the harness
2. Manually add your own bookmarks before trying to match them.
Flags: needinfo?(mak77)
Comment hidden (mozreview-request)
Attachment #8889844 - Flags: review?(dburns)

Comment 16

6 months ago
mozreview-review
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160926/#review166214

Comment 17

6 months ago
mozreview-review
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160926/#review166216
Test disabled for now while we work out if its worth fixing in the future.
Keywords: leave-open
Whiteboard: [stockwell needswork] → [stockwell disabled]
Comment hidden (mozreview-request)
Attachment #8889844 - Flags: review?(dburns) → review?(aryx.bugmail)
Note that I limited my analysis to "test_matching_text", I didn't look at all the other sub tests in test_toolbars

Is there a reason to not just disable the single sub test?
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160928/#review166218
Attachment #8889844 - Flags: review?(aryx.bugmail) → review+

Comment 22

6 months ago
Pushed by dburns@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ff6edfdd8b6c
Disable test_toolbars for high intermittence. r=aryx

Comment 24

6 months ago
40 failures in 1008 pushes (0.04 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* pine: 9
* mozilla-inbound: 9
* autoland: 9
* try: 4
* mozilla-central: 4
* mozilla-beta: 4
* cedar: 1

Platform breakdown:
* linux32: 11
* windows7-32: 9
* windows10-64: 8
* linux64: 7
* linux64-stylo: 3
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1369556&startday=2017-07-24&endday=2017-07-30&tree=all
You need to log in before you can comment on or make changes to this bug.