Closed Bug 1369556 Opened 7 years ago Closed 4 years ago

Intermittent test_toolbars.py TestAutoCompleteResults.test_matching_text | TimeoutException: Timed out after 5.0 seconds

Categories

(Testing :: Firefox UI Tests, defect, P5)

Version 3
defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, test-disabled, Whiteboard: [stockwell disabled])

Attachments

(1 file)

[task 2017-06-01T20:39:43.554992Z] 20:39:43     INFO -  1496349583552	Marionette	TRACE	14 -> [0,208,"executeScript",{"scriptTimeout":null,"newSandbox":true,"args":[],"filename":"toolbars.py","script":"\n          Components.utils.import(\"resource://gre/modules/Services.jsm\");\n\n          let win = Services.focus.activeWindow;\n          if (win) {\n            return win.gURLBar.controller.searchStatus >=\n                   Components.interfaces.nsIAutoCompleteController.STATUS_COMPLETE_NO_MATCH;\n          }\n\n          return null;\n        ","sandbox":"default","line":391}]
[task 2017-06-01T20:39:43.557087Z] 20:39:43     INFO -  1496349583556	Marionette	TRACE	14 <- [1,208,null,{"value":false}]

Looks like the search via the urlbar runs a very long time, or not sure what's not causing the status to change.
this picked up frequency on July 12th across the board.
Marco, did we change something for the autocomplete recently?
Flags: needinfo?(mak77)
The Location bar code changes almost every week in preparation for Photon, so the question is a bit generic :)
I don't know of a specific change to the location bar that happened on 12th (some happened on 14th).
Something that recently happened is work to delay most startup code to after the first paint, looks like this code relies on default bookmarks being imported, maybe they were not yet at that point? I'm just guessing.
For example bug 1371677 landed on 12th.

Can you give me more details about the point where the test is failing?
Flags: needinfo?(mak77) → needinfo?(hskupin)
I'd probably suggest waiting for places-browser-init-complete after startup, if you're not doing that yet. It should ensure you default bookmarks are there.
:whimboo is on pto, redirecting to :automatedtester since this is a high frequency failure and we want to see this resolved in the next 2 weeks.
Flags: needinfo?(dburns)
Whiteboard: [stockwell needswork]
:mak

Is this test covered by other test suites? I am wondering if this is a duplicate or if this test is actually worthwhile to the awesomebar folk.
Flags: needinfo?(dburns) → needinfo?(mak77)
From what I see the test searches for a string and then it checks that results match that string either in title or url.

We surely have tests in toolkit/components/places/tests/unifiedcomplete covering the fact that adding N visits/bookmarks and then searching for a string that appears only in M < N results, we only get the expected M results and not the nonmatching ones.
In particular toolkit/components/places/tests/unifiedcomplete/test_word_boundary_search.js that checks we match on word boundaries.
Those make me pretty confident that we return things that are matching.

The only thing this test is adding we don't seem to have is checking those matching strings are shown in the UI. The value isn't that high, since the presentation may also change in a way to not directly show what the urlbar is matching, and in the future we may match umlauts and diacritics differently and this may break again.
I also assume a regression in returning totally unrelated results would be easily catched by Nightly population.

To sum up, I'd not complain loudly if this would be removed. If instead you prefer to fix it to retain that small amount of added testing, I see 2 ways:
1. Wait for places-browser-init-complete before proceeding with the harness
2. Manually add your own bookmarks before trying to match them.
Flags: needinfo?(mak77)
Attachment #8889844 - Flags: review?(dburns)
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160926/#review166214
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160926/#review166216
Test disabled for now while we work out if its worth fixing in the future.
Keywords: leave-open
Whiteboard: [stockwell needswork] → [stockwell disabled]
Attachment #8889844 - Flags: review?(dburns) → review?(aryx.bugmail)
Note that I limited my analysis to "test_matching_text", I didn't look at all the other sub tests in test_toolbars

Is there a reason to not just disable the single sub test?
Comment on attachment 8889844 [details]
Bug 1369556 - Disable test_toolbars for high intermittence.

https://reviewboard.mozilla.org/r/160928/#review166218
Attachment #8889844 - Flags: review?(aryx.bugmail) → review+
Pushed by dburns@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ff6edfdd8b6c
Disable test_toolbars for high intermittence. r=aryx
Priority: -- → P4
Priority: P4 → P5

Given that we want to get rid of the firefox-puppeteer Python package, I would say we shouldn't spend any time in getting this re-enabled.

Blocks: 1573383
Component: Marionette → Firefox UI Tests
Flags: needinfo?(hskupin)
Keywords: test-disabled
QA Contact: hskupin

We are going to remove the firefox-puppeteer package. So this becomes a wontfix.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.