729037 - Mozmill test failure /testAwesomeBar/testEscapeAutocomplete.js | Current URL should be identical to the target URL

Status: VERIFIED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Maniac Vlad Florin (:vladmaniac)]

Firefox version: Firefox 13.0a1 (13.0a1, en-US, 20120218031156)
-----------------------------------------------------------------------------
Platform: All 
-----------------------------------------------------------------------------
Mozmill version: 1.5.7 
-----------------------------------------------------------------------------
First started failling: 2012/02/18
-----------------------------------------------------------------------------
Currently failling on Nightly/default branch 
-----------------------------------------------------------------------------
Report: http://mozmill-release.blargon7.com/#/functional/report/55d601cc2aabfac28f59060a84a11ad4
-----------------------------------------------------------------------------
Error: TimeoutError("Current URL should be identical to the target URL - got localhost:43336/layout/mozilla_community.html, expected mozilla")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Current URL should be identical to the target URL - got localhost:43336/layout/mozilla_community.html, expected mozilla",5000,100,(void 0))@resource://mozmill/modules/utils.js:467 ([object Proxy],"Current URL should be identical to the target URL - got localhost:43336/layout/mozilla_community.html, expected mozilla")@resource://mozmill/modules/controller.js:648

Comment 1

[Maniac Vlad Florin (:vladmaniac)]

This failure was due to a firefox change/bug in Nightly - 

Before: (Aurora/Beta/Release) 
1. first ESC key press dismisses the awesome bar suggestion list
2. 2nd ESC key dismisses the search string in the awesome bar 

Now: 
1. first ESC key press dismisses the awesome bar suggestion list
2. 2nd ESC key press - does nothing 

If this is intended, we'll change the test for the default branch considering that 
2nd ESC key press is useless now. 
However, if this is not intended, then a Firefox bug should be in order. 

Link with investigation results 
http://pastebin.mozilla.org/1489478

Comment 2

[Maniac Vlad Florin (:vladmaniac)]

Attachment: skip patch v1.0

Skip the test on default until we know 'which is which' - intended change or bug

Comment 3

[Alex Lakatos[:AlexLakatos]]

Comment on attachment 599114 [details] [diff] [review]
skip patch v1.0

># HG changeset patch
># User Vlad Florin Maniac <[e-mail]>
># Date 1329829376 -7200
># Node ID 6fee298555262f09504d4b35e57a163eceb27a3a
># Parent  8286c25c8978d7b193d3b35cc75d8c83a645ef6e
>Bug 729037 - Mozmill test failure /testAwesomeBar/testEscapeAutocomplete.js | Current URL should be identical to the target URL. r=alexlakatos
Commit message should be "Bug 729037 - disable testAwesomeBar/testEscapeAutocomplete.js due to failure. r=alexlakatos"

Comment 4

[Maniac Vlad Florin (:vladmaniac)]

Attachment: skip patch v2.0 [landed]

fixed

Comment 5

[u279076]

Comment on attachment 599115 [details] [diff] [review]
skip patch v2.0 [landed]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/750c91e821a4 (default)

Does this need to land on any other branches?

Comment 6

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> Comment on attachment 599115 [details] [diff] [review]
> skip patch v2.0 [landed]
> 
> Landed:
> http://hg.mozilla.org/qa/mozmill-tests/rev/750c91e821a4 (default)
> 
> Does this need to land on any other branches?

Nope - only on default. 

Anthony, can you help with my question in comment 1? Do you know whether this change is intended or not? If yes, I have to update this test and the litmus test as well. If not, there's a Firefox bug. Thanks

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Vlad, you should simply check when the failure has been appeared the first time and then check the pushlog of mozilla-central to figure out the change in Firefox. It's similar to the other regression tests we already did a couple of times.

Comment 8

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #7)
> Vlad, you should simply check when the failure has been appeared the first
> time and then check the pushlog of mozilla-central to figure out the change
> in Firefox. It's similar to the other regression tests we already did a
> couple of times.

The failure started happening from 2012/02/18, as stated in the bug description info. 

pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2012-02-17&enddate=2012-02-19 

the only bug here that has some relation to the awesomebar is bug 725200, which landed only on nightly. It's a small patch but I cannot decide for sure if the changes caused our test to fail.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Instead of using dates you should make use of the changeset of the builds which give a 100% accurate information. The above pushlog could miss some changesets.

Comment 10

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #9)
> Instead of using dates you should make use of the changeset of the builds
> which give a 100% accurate information. The above pushlog could miss some
> changesets.

I've downloaded the builds and used about: page to get the changeset information. 

http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=2271cb92cc05

http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=550779e6bab4

http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=4d47329bb02e

Sorry, its my first time with this. Is this the information we want?

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So which of those builds regressed? Create the link similar to:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b8dd6f6f4207&tochange=3491b2f021bf

Comment 12

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #11)
> So which of those builds regressed? Create the link similar to:
> 
> http://hg.mozilla.org/mozilla-central/
> pushloghtml?fromchange=b8dd6f6f4207&tochange=3491b2f021bf

So this is the link we want 
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=2271cb92cc05&tochange=4d47329bb02e

Again, I see no explicit changes so we might have a Firefox regression.

Comment 13

[Maniac Vlad Florin (:vladmaniac)]

Build works 
http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=2271cb92cc05

Build broken
http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=550779e6bab4

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So what's now the broken build? Changeset 4d47329bb02e or 550779e6bab4? You mixed up those data.

Comment 15

[Maniac Vlad Florin (:vladmaniac)]

Build broken is http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=550779e6bab4, I went further a bit in that link. 

Comment 13 is the regression range we want

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So the actual pushlog is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=2271cb92cc05&tochange=550779e6bab4

Does it happen across platforms and can you reproduce it locally with a fresh profile? A good start would be to build Firefox on your own to nail down the exact change. Sadly no tinderbox builds are available during this time range.

Comment 17

[u279076]

(In reply to Maniac Vlad Florin (:vladmaniac) from comment #6)
> Anthony, can you help with my question in comment 1? Do you know whether
> this change is intended or not? If yes, I have to update this test and the
> litmus test as well. If not, there's a Firefox bug. Thanks

My suggestion would be to file a new Firefox bug. Before we go to all the trouble of this "regression" investigation, first find out if this is expected behaviour.

If this behaviour is intended, the bug will be RESOLVED INVALID and we can then proceed with fixing the test.

If this behaviour is not intended, then it is a real Firefox bug and we can continue regression investigation there.

Comment 18

[Maniac Vlad Florin (:vladmaniac)]

Strange, the latest Nightly works 
Build identifier: Mozilla/5.0 (X11; Linux i686; rv:13.0) Gecko/20120227 Firefox/13.0a1

I will run the mozmill test locally see if the failure is still reproducible, but I doubt it. 
Something fixed our issue overnight.

Comment 19

[Maniac Vlad Florin (:vladmaniac)]

Attachment: enable v1.0 [landed]

Yes, we can enable back the test on the default branch, starting today, the problem is somehow fixed. 

Tested locally both manual and with a Mozmill profile.

Comment 20

[u279076]

Comment on attachment 601195 [details] [diff] [review]
enable v1.0 [landed]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/b10a21241078 (default)

Comment 21

[u279076]

Please verify with tomorrow's test results.

Comment 22

[Maniac Vlad Florin (:vladmaniac)]

No failure 

http://mozmill-release.blargon7.com/#/functional/top?branch=13.0&platform=All&from=2012-03-05&to=2012-03-06

Verified fixed.