Closed Bug 688146 Opened 9 years ago Closed 9 years ago

Backout mozmill tests for deleted collection panels from the Discovery Pane

Categories

(Mozilla QA Graveyard :: Mozmill Tests, defect)

defect
Not set
normal

Tracking

(Not tracked)

VERIFIED WONTFIX

People

(Reporter: vladmaniac, Unassigned)

References

Details

Discovery Pane has changed lately and some of the panes from the Collection Panel were removed 

Those related mozmill tests are failing due to lack of data to test on, so they should be urgently removed from hg. 

We are currently working on a new testing strategy for the Collections Panel automation tests

The tests I am talking about are 

* bug 664019
* bug 657492
Please feel free to invalidate this bug if considering this is not a proper place for this request
Requesting feedback from Henrik. Can we simply backout the changesets from those respective bugs?
Where are the bugs which cover the removal of those panels? I would like to see those first in the dependency list before I give any statement regarding their removal.
(In reply to Henrik Skupin (:whimboo) from comment #3)
> Where are the bugs which cover the removal of those panels? I would like to
> see those first in the dependency list before I give any statement regarding
> their removal.

Well, my suggestion would be to simply close this bug and take care of the backouts on the respective bugs (since that's where the patches live).

Vlad, can you identify ALL the tests/bugs which need backout?
No, this bug is perfectly valid for this task but we have to reference the bugs for removal of those pages.
(In reply to Henrik Skupin (:whimboo) from comment #5)
> No, this bug is perfectly valid for this task but we have to reference the
> bugs for removal of those pages.

Alright then. We have two bugs in comment 0. Vlad are there any other bugs?
No. I've provided all the needed data
Depends on: 664019, 657492
I do not talk about the bugs which cover the test implementation, but the bugs for the removal of those panels on the product side. Also I don't see any failure for the pick of the month test. Can you please list those failing results?
No longer depends on: 657492, 664019
I can't find the bugs for removal on the product side. I'm sure I am not looking in the right place. 

I've spoken with Krupa about it and she said that marketing department handles those pane and that there isn't a specific development schedule for those. 

I also don't see errors from Pick of the Month pane test, but keep in mind that the test is skipped on aurora and default branch - it only works for beta and release - and I don't see those branches in the report as well 

Either way, those panes do not exist anymore in the current Discovery Pane.
So bug 661651 is about some changes of the pick of the month panel. But I don't follow it completely. Even with an admin tool now in place, where does this selected add-on now appear? Can you please get an answer for that question? If it simply has another location we only would have to update the current test.

Well with 7.0 it fails:
http://mozmill-release.brasstacks.mozilla.com/#/remote/report/cbba999432ff4d1adda4c1946a122e6f
OS: Linux → All
Hardware: x86 → All
It seems that panels reappear in the disc pane, i believe the marketing team decides which panels are active and when - we should not back out the tests just come up with some kind of a runtime decision algorithm in the API which would tell us if a panel is active/visible or not - then do the test, if not, skip test. 

wontfix then
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
If marketing is going to decide when and what panes will be visible without a moment's notice I suggest we shouldn't have tests for this feature. It's going to make them far too unstable. On the other hand, if there will be a set of baseline panes which will always be there it should be safe to have tests for that.

Vlad can you please find out which panels will never change and which panels will change? We should then back out the tests for changing panels and scrap them. Reopening for now so we can do some more investigation.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
I'll get with Krupa in one of the webqa meetings see if we can get someone to help asap. 
My answer will come in the bugs and also on the e-mails
Well there seem to be no specific rule in to how or which the panes change. Our disc pane tests will fail if there are not day-to-day monitored, and I don't think anyone has time to do that right now.
This is a wontfix. the best decision is to disable the tests because the panes may reappear again and we can re-enable and make usage of the disc pane remote tests. 

A good example of a pane coming back is "Pick of the Month" add-on pane 
Wontfix here and continue this on bug 732353
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → WONTFIX
Thanks Vlad, I agree with this assessment.
Status: RESOLVED → VERIFIED
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.