Closed Bug 709063 Opened 13 years ago Closed 7 years ago

Our /endurance/testTabbedBrowsing_PinAndUnpinAppTab/ test is not reliable when using more than 50 entities

Categories

(Mozilla QA Graveyard :: Mozmill Tests, enhancement)

Product:
Mozilla QA Graveyard
Component:
Mozmill Tests
Platform:
x86
Linux
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: vladmaniac, Unassigned)

References

(
URL
)

Details

(Whiteboard: [mozmill-endurance])

Test location: tests/endurance/testTabbedBrowsing_PinAndUnpinAppTab/test1.js Branch: any Mozmill version: 1.5.7 Failure links: http://mozmill-crowd.brasstacks.mozilla.com/#/endurance/report/d0d111e12f8df4dcec337749863da064 http://mozmill-crowd.brasstacks.mozilla.com/#/endurance/report/d0d111e12f8df4dcec337749863d9c0c Reason: Investigated and the reason why it fails is that tabs location is fully populated. The test just opens the context menu but cannot access the right page to pin and just freezes. We should re-think this because IMO we cannot rely on this if someone wants to use many entities when testing Firefox.
We have a backend API - we could replace the context menu IMO it does not matter whether we use the UI - context menu or not, it's just the pinning and unpinning which we are interested in and need some metrics on.
Whiteboard: [mozmill-test-failure][mozmill-endurance]
Is there an access key we can use instead?
Yes but you have to have the context menu triggered and just press 'P' for pinning tab and 'b' for unpinning. This only changes click with keyboard event. More, keypress in mozmill will be deprecated so...i do not recommend it Unfortunately, the backend API idea is all I have under the sleeve at the moment
Okay, lets wait for Dave to provide his thoughts on this.
Sorry, but I don't understand the definition of the problem here. I'm not keen on moving to a backend API as this does not simulate a user. I'd much prefer a click or keyboard shortcut. Under what circumstances does this fail? On what platforms? Can we replicate it reliably? Also, can you provide a reference for key presses being deprecated in Mozmill?
(In reply to Dave Hunt (:davehunt) from comment #5) > Sorry, but I don't understand the definition of the problem here. I'm not > keen on moving to a backend API as this does not simulate a user. I'd much > prefer a click or keyboard shortcut. > > Under what circumstances does this fail? On what platforms? Can we replicate > it reliably? > > Also, can you provide a reference for key presses being deprecated in > Mozmill? All the details are in Description. It's reproducible always, regardless of the platform As for key pressing please take a look at https://developer.mozilla.org/en/Mozmill/Mozmill_Controller_Object
Sorry, but I still don't understand the description of the problem. "The reason why it fails is that tabs location is fully populated. The test just opens the context menu but cannot access the right page to pin and just freezes." What do you mean by "the tabs location is fully populated"? Or "cannot access the right page to pin"? You are saying this can be replicated reliably, so please provide details of how you run this to replicate the issue. The reference you link to states that controller.keyPress is deprecated, and to use the mozmill element object instead. See https://developer.mozilla.org/en/Mozmill/Mozmill_Element_Object#keypress.28.29
our test does not take into consideration the case when all our tab bar is fulled with pinned tabs - this is the failure reason just run then test and give it 50 iterations. it will fails, at least it does here locally
Please see https://bugzilla.mozilla.org/show_bug.cgi?id=670721#c31. This should not affect our daily testruns.
(In reply to Dave Hunt (:davehunt) from comment #9) > Please see https://bugzilla.mozilla.org/show_bug.cgi?id=670721#c31. This > should not affect our daily testruns. I see. Nevertheless we should keep this opened because it makes our test not robust. In time we should fix it
But this should not be the cause of the intermittent failures that we are seeing, that still needs to be investigated and identified.
(In reply to Dave Hunt (:davehunt) from comment #11) > But this should not be the cause of the intermittent failures that we are > seeing, that still needs to be investigated and identified. Those failures are not intermittent - those two mozmill crowd db links show the error we get, we can always reproduce the error in the given conditions of having many pinned tabs However, for 10 entities we do not fail
In a case of hidden app tabs, are the overflow arrows visible - or at least the properties of the tabbar set? That way we could probably limit the number of entities to a max number as long as the Firefox bug hasn't been fixed yet. I would say bug 654604 is blocking us in some way here.
Vlad, please write a patch to disable this test immediately; this way we get a clean testrun again.
Anthony: This is not failing in our daily testrun and doesn't need to be disabled.
(In reply to Dave Hunt (:davehunt) from comment #15) > Anthony: This is not failing in our daily testrun and doesn't need to be > disabled. Actually, it is: http://mozmill-release.brasstacks.mozilla.com/#/endurance/report/bc010f9f935bacb82b110d15da08143e
Sorry, I was going by the fact that this bug has been changed to being unreliable "with more than 50 entities", the only reports previously linked being Mozmill Crowd ones, and the statement: "for 10 entities we do not fail".
Given that this is on our live results, and per discussion via IRC, please disable the test Vlad, thanks. We can continue to debug with it disabled.
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #18) > Given that this is on our live results, and per discussion via IRC, please > disable the test Vlad, thanks. We can continue to debug with it disabled. Remus, please take over this and investigate also bug 707663 We might have a fast fix for it
Assignee: nobody → remus.pop
Status: NEW → ASSIGNED
See Also: → 707663
We should continue on bug 707663 - this is the same issue which needs handling
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → DUPLICATE
Reopening due to comment 40 and 42 from bug 707663.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Assignee: remus.pop → nobody
We will never run this amount of entities. But it would be good for an enhancement.
Severity: normal → enhancement
Status: REOPENED → NEW
Whiteboard: [mozmill-test-failure][mozmill-endurance] → [mozmill-endurance]
Mozmill is dead, WONTFIX the remaining bugs.
Status: NEW → RESOLVED
Closed: 13 years ago → 7 years ago
Resolution: --- → WONTFIX
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.