900911 - Test failure 'Active tab url should equal the page url' in /testAwesomeBar/testSwitchToTab.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Andrei Eftimie]

Test:      testSwitchToTab    
Failure:   Active tab url should equal the page url
Branches:  default
Platforms: All

Does not fail if test is run independently.
Either one of testCheckItemHighlight.js or testEscapeAutocomplete.js if run before the mentioned test make it fail.

Possible regression. Does not fail on older Nightly versions.
Working on a regression range now.

The failure also mentions a distorted test name:
> "message": "Active tab url should equal the page url", 
> "lineNumber": 82, 
> "name": "testSwitchToTab/</switchToTab<", 
> "fileName": "file:///Users/andrei.eftimie/work/mozilla/temp/mozmill-tests/tests/functional/testAwesomeBar/testSwitchToTab.js"

Comment 1

[Andrei Eftimie]

Something regressed in the Nightly build from 2013-06-28, which was build from: http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=8e3a124c9c1a

Looking into tinderbox builds to further narrow it down.

Comment 2

[Andrei Eftimie]

Responsible pushlog: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=8f7ef2198996

Responsible bug 791776

I'll make a simplified testcase before tagging Marko for more info.

Comment 3

[Andrei Eftimie]

Attachment: testcase.js

Attached is a testcase.
This doesn't look like a genuine regression to me.

I'll have shortly a fix attached and explained.

Comment 4

[Andrei Eftimie]

Attachment: 1.patch

Now to explain the problem:
We are typing multiple strings in the urlbar, for each of them waiting for the switchToTab item to appear.
After we type something in we wait for the Autocomplete popup to 1) be open and 2) have a different number of items than 0.

The problem would arise at the next iteration.
Typing (or some parts of the update to the UI) are now async and no longer block, we still have the old Autocomplete results open. Our checks pass, and we click again on the previous link.

The fix is to simply clear the urlbar before doing any typing, the Autocomplete closes, and we start the second iteration fresh. (We could wait for a few ms before searching for the switchToTab item).

Comment 5

[Andrei Eftimie]

Comment on attachment 785027 [details] [diff] [review]
1.patch

Since this is a mozmill-tests issue, added Andreea as reviewer aswell.

Comment 6

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 785027 [details] [diff] [review]
1.patch

Review of attachment 785027 [details] [diff] [review]:
-----------------------------------------------------------------

Please add a commit message to the patch.

::: tests/functional/testAwesomeBar/testSwitchToTab.js
@@ +43,5 @@
>    // Wait for 4 seconds to work around Firefox LAZY ADD of items to the DB
>    controller.sleep(PLACES_DB_TIMEOUT);
>  
>    TEST_DATA.forEach(function (aPage) {
> +    locationBar.clear();

Hm, interesting that this is only happening on 2.0. I'd look more into that. 
But it would make sense to have this change anyway, looks like an improvement of the code.
The thing is, clear() already focuses before actually clearing, so I would remove the line bellow.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

[mozmill-2.0?] is an entry for the mozmill component only. It will not help when set for mozmill-tests. As of now I don't see what has to be fixed for Mozmill 2 here.

Comment 8

[Andrei Eftimie]

This only fails under mozmill 2.0, hence the whiteboard entry.
Didn't thought it applied only to mozmill proper.

I'll look a bit more into this, since it still seems a bit strange.
Clearing the locationbar before each change does fix it, but it seems I am missing something.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Well, than lets indicate that. I thought you wanted to bring this up as blocker for 2.0.

Comment 10

[Andrei Eftimie]

Attachment: 2.patch

> The thing is, clear() already focuses before actually clearing, so I would
> remove the line bellow.
Thanks Andreea for noticing this.
Have removed the focus call now.

> Hm, interesting that this is only happening on 2.0. I'd look more into that. 
> But it would make sense to have this change anyway, looks like an improvement 
> of the code.
I am not sure what we do differently in Mozmill 1.5... I don't think it's worth 
much more time
This change does improve the stability of the test, and fixes the problem we see
in Mozmill 2.0

Here are testruns in 1.5:
http://mozmill-crowd.blargon7.com/#/functional/report/6d53b7d749a60e65551b4b1d19d61c86
http://mozmill-crowd.blargon7.com/#/functional/report/6d53b7d749a60e65551b4b1d19d7175c

Here is a testrun under 2.0:
http://mozmill-crowd.blargon7.com/#/functional/report/6d53b7d749a60e65551b4b1d19d574d1
(have bug 886811 applied)

Comment 11

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 796048 [details] [diff] [review]
2.patch

Review of attachment 796048 [details] [diff] [review]:
-----------------------------------------------------------------

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/8290d510a1cc (default)

Are there any other branches affected here?

Comment 12

[Andrei Eftimie]

Yes we should backport this.
Will run some tests and come back.

Comment 13

[Andrei Eftimie]

Only Aurora is affected.

Comment 14

[Andrei Eftimie]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/b6577c665f7d (aurora)

Testruns:
http://mozmill-crowd.blargon7.com/#/functional/report/3c3cd991de01b704f3f65783a14dc08a
http://mozmill-crowd.blargon7.com/#/functional/report/3c3cd991de01b704f3f65783a14ca727