939749 - Test failure "The auto-complete result is a bookmark - 'favicon' should contain 'bookmark'" in tests/functional/testAwesomeBar/testSuggestBookmarks.js

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, defect, P4)

Description

[Mihaela Velimiroviciu (:mihaelav)]

The testSuggestBookmarks from the functional testrun fails with "The auto-complete result is a bookmark - 'favicon' should contain 'bookmark'" on Australis.

Report:
http://mozmill-crowd.blargon7.com/#/functional/report/2f1ca72938985fc4b989b7efcc45e3c8

The failure was seen only on Linux (Ubuntu 13.10).

Comment 1

[Mihaela Velimiroviciu (:mihaelav)]

> The failure was seen only on Linux (Ubuntu 13.10).
The Ubuntu version was 13.04, not 13.10.

I reproduced the failure on non-australis build, as well, so it's not australis-related:
http://mozmill-crowd.blargon7.com/#/functional/report/2f1ca72938985fc4b989b7efcc98201a

The issue reproduces constantly on my 13.04 and on a 12.04, but it doesn't reproduce at all on other machines (12.04 and 13.10).

Comment 2

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: minimizedTestCase.js

I attached the minimized test case for this, this reproduces with all firefox versions.
It doesn't reproduce manually.
The steps are:
1  Opens: "http://mozqa.com/data/firefox/layout/mozilla_grants.html"
2  Opens: "about:blank"
3  Types: "grants" in url-bar
4  Clears all history.
5  Opens: "http://mozqa.com/data/firefox/layout/mozilla_grants.html"
6  Bookmarks the page
7  Clears all history.
8  Types: "grants" in url-bar
  
  Expected result:
    The suggestion should be a bookmark (have the yellow star in the right)
  Actual result:
    The suggestion is an previous search result (without the yellow star in the right)

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8336169 [details]
minimizedTestCase.js

Not sure for what this feedback request is for. I assume you want needinfo on what? Also ensure to attach plain/text attachments. Thanks.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Also I thought Mihaela is working on this bug? Or are you joining and helping her?

Comment 5

[Andrei Eftimie]

If this is reproducible, please use the testcase to finding a regression range.

Comment 6

[Cosmin Malutan, [:cosmin-malutan]]

Hi Henrik, I didn't take the bug, I was just helping since it reproduced on my machine (Ubuntu 12.04) and Mihaela had a 13.04.

Comment 7

[Mihaela Velimiroviciu (:mihaelav)]

Cosmin can take this, since it is not Australis related (see comment #1).

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, so how often does it fail? Is it something we should focus on? What's the priority of this bug?

Comment 9

[Andreea Matei [:AndreeaMatei]]

I don't think we've ever seen this on our ci testruns, only locally by Mihaela and Cosmin.

Comment 10

[Cosmin Malutan, [:cosmin-malutan]]

Hi, here is what I found out:
This failure happens because nsAutoCompleteController caches the previous suggestions so if we search for the same word we will get the previous found suggestion.
In order to see this manually we have to paste in the whole "grants" word instead of typing letter by letter in steps 3 and 8.

Comment 11

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: patch_v1.0.patch

I couldn't find how to don't cache the suggestions, so I propose to search for full title in the failing tests so we won't get the cached suggestion from the previous search.  

Reports:
http://mozmill-crowd.blargon7.com/#/functional/report/456bebe92845279408c15c03e8fc6474
http://mozmill-crowd.blargon7.com/#/functional/report/456bebe92845279408c15c03e8fc5abe
http://mozmill-crowd.blargon7.com/#/functional/report/456bebe92845279408c15c03e8fc5cfa

Comment 12

[Andrei Eftimie]

Comment on attachment 8338611 [details] [diff] [review]
patch_v1.0.patch

Review of attachment 8338611 [details] [diff] [review]:
-----------------------------------------------------------------

This appears likely to be a bug in Firefox, lets not hastily fix our code to accommodate it.

On OSX the testcase passes with Firefox 25 and up.
But it fails on ESR17

From this limited testing this looks like it might have been fixed on OSX somewhere between 17 and 25.
We should delve deeper into this issue.

Comment 13

[Cosmin Malutan, [:cosmin-malutan]]

Ok so this reproduces manually, I still have no idea if its an expected behavior or a bug.

The steps to reproduce this manually are:

1  Open: "http://mozqa.com/data/firefox/layout/mozilla_grants.html"
2  If page is bookmarked, remove the bookmark
3  Open: "about:blank"
4  Paste in "grants" in url-bar (It's very important to paste the whole word and not typing letter by letter)
5  Clear all history.
6  Open: "http://mozqa.com/data/firefox/layout/mozilla_grants.html"
7  Bookmarks the page
8  Open: "about:blank"
9  Clear all history.
10 Paste in "grants" in url-bar (It's very important to paste the whole word and not typing letter by letter)
  
  Expected result:
    The suggestion should be a bookmark (have the yellow star in the right)
  Actual result:
    The suggestion is an previous search result (without the yellow star in the right)

Marco, do you know what would be the expected behavior here, when we search for the same word we should get the previous suggestion?

Comment 14

[Marco Bonardo [:mak]]

I tried a couple times on current nightly using steps in comment 13 but I couldn't reproduce... It's possible that autocomplete optimizes results this way, and it's hard to tell if that'd be wanted or not, the perf/nonflicker benefit could be better than the downside (it's sort of an edge case).
Are you sure the problem is due to caching and not a timing issue? clear history, search and other history operations are asynchronous and may confuse the search. If you can still reproduce the issue by waiting some seconds between each steps then it's very likely a caching problem and we may look at it, otherwise it's a timing issue, very unlikely to hit a user, and we wouldn't care much about it.
Regardless, even we fall in the first case (caching issue), due to the complexity of the involved steps to reproduce, I'd suggest to workaround it for this test and file a very specific bug (requesting a test) for that edge case.

Comment 15

[Cosmin Malutan, [:cosmin-malutan]]

Thanks Marco.

A sleep doesn't help here, but this doesn't reproduce with mozmill-2.0(because type method will synthesize an keypress event for each letter) so we won't see this failure in our tests.
This still reproduces if I follow the steps manually, or with mozmill 1.5.
Also as I said in steps from comment 13 it reproduces only if we paste in the whole word and not if we type in each letter(as user does). With that in mind, and because this wont have any impact on user I would close this bug as it "works for me".
Henrik what do you think?

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As Marco suggested I would try to reproduce it manually with some delays between those steps. Typically should be around 4s. If you can still reproduce it, it might be worth to get fixed at some point. If not, we might want to close it as WONTFIX but not as WFM, because the latter is clearly not the case.

Comment 17

[Cosmin Malutan, [:cosmin-malutan]]

If this wont be fixed I close the bug.
Thanks

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Cosmin, why is this wontfix? Can you please give the reasoning for your judgement? I don't see any indication for it.

What kind of workaround can we do to at least make it not failing anymore?

Comment 19

[Cosmin Malutan, [:cosmin-malutan]]

(In reply to Henrik Skupin (:whimboo) from comment #18)
> Cosmin, why is this wontfix? Can you please give the reasoning for your
> judgement? I don't see any indication for it.
> 
> What kind of workaround can we do to at least make it not failing anymore?

Henrik, as I said in comment 15 this won't reproduce anymore with mozmill 2.0+, because we will type in each letter instead of a whole word so the suggestions won't be cached. This is an edge case which is almost impossible for user to walk in, in our tests it won't reproduce anymore.

About my reasoning for closing this as wontfix you said so in comment 16 to close it so.

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Well, that's a better explanation as comment 17. Given the long time of no activity I missed those points. In that case lets close it, yes.