Closed Bug 503285 Opened 16 years ago Closed 16 years ago

Clone 3.5 testrun for new features in 3.6

Categories

(Webtools Graveyard :: Litmus, defect, P2)

Product:
Webtools Graveyard
Component:
Litmus
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: whimboo, Assigned: coop)

Details

We should consider to clone the 3.5 testrun at some stage to be able to cover the new features for Firefox 3.6 (3.next). Before we do this we definitely have to clean up the 3.5 testrun as best as we can. Otherwise the work would be twice as much. I have created this bug to have a reminder and which I can reference in other bugs which cover new features.
Agreed - before we do this we definitely need to clear up as many of the unclear test cases as we can. A few suggestions: 1. We should try to get as many test cases as we can checked in internally instead of relying on test cases that point to outside servers. When those servers go away then we are left with a lot of broken test cases. An example I can think of right off is Sarah's Javascript console error test case which still resides on her server. 2. Maybe we create a project on QMO called "Litmus Cleaners" and try to get more community to contribute to this. That is how Aakash started out and it might be worthwhile to get some people interested in this.
(In reply to comment #1) > 1. We should try to get as many test cases as we can checked in internally > instead of relying on test cases that point to outside servers. When those > servers go away then we are left with a lot of broken test cases. An example I > can think of right off is Sarah's Javascript console error test case which > still resides on her server. Point me to such tests by sending me an email with the test #. I can move the testcases to the Litmus testcase repository. I don't see a simple of searching for such tests on Litmus. > 2. Maybe we create a project on QMO called "Litmus Cleaners" and try to get > more community to contribute to this. That is how Aakash started out and it > might be worthwhile to get some people interested in this. Good idea. Lets cc Aakash to the bug.
I was sure that I have updated the test after the refactoring of the litmus testcase storage. Thanks for pointing that out. I have updated the test and it works again.
As talked in the meeting we should do this a week after the alpha1 release. We should check at least failed tests so we have at least those tests updated. If we are not ready we should punt it a bit. But seeing the schedule for 3.6 we don't have so much time. Would be great to have it asap.
Let's shoot for August 6th as the cloning date. That gives us 2 weeks to do any final cleanup on 3.5. I don't think anyone is going to be ready with new, 3.6 specific, test cases much earlier than that.
3.6 will be nearly the same as 3.5 as far as UI is concerned. Most of the work will be in the backend, some in performance and some component re-writes. As Henrik and Tracy have pointed out, the best thing to do right now would be to clean up the existing 3.5 test suite.
3.5 BFT reduction is complete. Coop, please clone the entire 3.5 suite to 3.6 at your discretion.
I'll try to get this done today.
Assignee: nobody → ccooper
Priority: -- → P3
Status: NEW → ASSIGNED
Priority: P3 → P2
This is done now. I've created a 3.6 catch-all test run that contains the same testgroups as for 3.5. This test run is not flagged as recommended. QA can toggle that when you're ready. Also, someone will want to fix up the Tegra test run (id: 44). This is currently marked as belonging to the 3.5 branch because it was created before the 3.6 branch existed (I just created the branch now). It includes a testgroup Tegra (id: 130) that also belongs to the 3.5 branch, with corresponding subgroups, etc. There are 17 results belonging to this test run at present. A new Tegra testgroup has been automatically created for 3.6 (id: 132). It's up to you whether you want to update the original test run/testgroup/subgroups/testcases to preserve those results, or simply start using the new Tegra testgroup, i.e. fixing the existing Tegra test run to point to the new testgroup. Either way, the unused components should be deleted.
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Thanks Coop! It looks good. We will mark the 3.6 testrun as recommended later when the tegra testgroup has been updated.
Status: RESOLVED → VERIFIED
Unless anyone has a strenuous objection, I am going to delete the 3.5 Tegra testgroup since there are so few results. I will continue to refine the 3.6 testgroup and then enable it when it is ready for prime time.
Status: VERIFIED → RESOLVED
Closed: 16 years ago → 16 years ago
(In reply to comment #12) > Unless anyone has a strenuous objection, I am going to delete the 3.5 Tegra > testgroup since there are so few results. I will continue to refine the 3.6 > testgroup and then enable it when it is ready for prime time. no objections from me. I have already started editing the testcases in the 3.6 testgroup
Status: RESOLVED → VERIFIED
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.