Need to clone 3.6 test groups for 3.7 in order to create a 3.7 "catch-all" test run

RESOLVED FIXED

Status

RESOLVED FIXED
9 years ago
2 years ago

People

(Reporter: jbecerra, Assigned: marcia)

Tracking

Details

(Reporter)

Description

9 years ago
As we move into Firefox 3.7 territory we need to create a new "catch-all" test run to capture our test results in litmus on that coming branch. We can use the 3.7 designation.

Could we have this done before the end of this week?
i have no clue how to create this, but maybe marcia
I think the last time for 3.6 was clone the existing 3.5 group for the areas we wanted - FFT, BFT, smoketest, etc. But I think this would be faster to do on the backend which I believe is the way it was done last time. Once that is done then we can create the catch all test run.
If there are new features that will need subgroups then we will need to sort that out as well.
Summary: need new branch for "catch-all" test run → Need to clone 3.6 test groups for 3.7 in order to create a 3.7 "catch-all" test run
The cloning process has been done by Chris the last time. I'm not really sure which way he has used for. Can you enlighten us, Chris?  Bug 503285 tracked the 3.6 cloning request.
(In reply to comment #4)
> The cloning process has been done by Chris the last time. I'm not really sure
> which way he has used for. Can you enlighten us, Chris?  Bug 503285 tracked the
> 3.6 cloning request.

In theory, anyone with superuser admin access should be able to do this.

* select Manage Categories
* select Manage Branches
* select 3.6 branch (Firefox)
* select Clone Branch
* select Recursive
* update name and regexp
* grab a coffee, it might take a while

This is what I did for 3.5 -> 3.6 FWIW. I can do it for 3.7 too if other people are nervous about pulling the trigger.
I just went into Litmus and followed the Steps in Comment 5. I think after that task completes, we will still have to make clones of all component testgroups/subgroups/testcases.
Actually from looking at the UI it looks as if it created the groups but we will just have to change the names from 3.6 to 3.7
Yes, clones have been created but someone would have to change the names. Btw. I have updated the regex detection because it was broken.
Blocks: 547838
Would rather not have to change all the subgroup names manually since we append all groups with "3.6" - investigating other options as to an easier way to do this.
(In reply to comment #9)
> Would rather not have to change all the subgroup names manually since we append
> all groups with "3.6" - investigating other options as to an easier way to do
> this.

Did you change the name and regexp when cloning initially? This was all meant to be done automatically. <marge>hrmmmmmm</marge>

I just did a sweep of subgroups and testgroups in the database and replaced all occurrences of 3.6 with 3.7 in the names of the new groups. You'll want to do a sweep to verify them, especially the update testcases which might contain more than one version string.

No test runs created yet, but you'll probably want to do that from scratch, making sure there are no tegra tests in there.

Do you want me to do the same search/replace for testcase summaries and expected/actual steps?
I did change the regex, but it looks as if Henrik also did something to it according to Comment 8.
(In reply to comment #11)
> I did change the regex, but it looks as if Henrik also did something to it
> according to Comment 8.

The regex had too many slashes in it. I have just copied the the value from the 3.6 branch and replaced 3.6 with 3.7.

Coop, it would be great when we could get this documented very well on the wiki.
(In reply to comment #12)
> Coop, it would be great when we could get this documented very well on the
> wiki.

Sure would, but I'm really not supposed to be spending time on Litmus any more. Anyone can add content to http://quality.mozilla.org/litmus-admin-tutorial, but I'll try to add some more info to it when I have a chance.

How's Litmus2 coming along? Is someone taking note of all these pain points?

Comment 14

9 years ago
(In reply to comment #13)
> How's Litmus2 coming along? Is someone taking note of all these pain points?

Yeah this was one of the primary pain points we wanted to address in Litmus2.  That work is stalled right now because mevans wanted to do a little more research on existing solutions before having us go off and write something.

I think having the QA team (I'm not sure who on the team, but maybe they can respond to that) research existing, off-the-shelf test case managers would be a great win for us.  It will either come down to:
1. They find an off-the shelf tool that works with some minimal amount of customization.
-- or --
2. They will have very solid requirements and a good design for what they want in Litmus2.  

I'm excited to see what they discover.  But yeah, short answer: this is definitely something we would fix in Litmus2 (Heather already designed a system to address it) but we're not actively coding on Litmus 2 right now due to the ongoing survey of the off-the-shelf solutions.
(In reply to comment #13)
> Sure would, but I'm really not supposed to be spending time on Litmus any more.
> Anyone can add content to http://quality.mozilla.org/litmus-admin-tutorial, but
> I'll try to add some more info to it when I have a chance.

There's now a section on Cloning Branches in the admin tutorial.

Comment 16

9 years ago
Clint, Chris is there any doc/notes that describe the current and/or planned features of Litmus2? This would help guide the evaluation of existing solutions.

Comment 17

9 years ago
(In reply to comment #16)
> Clint, Chris is there any doc/notes that describe the current and/or planned
> features of Litmus2? This would help guide the evaluation of existing
> solutions.
This was in the presentation we made last month for the QA team, and its in our implementation notes, but I can write something up more formally on the wiki. I'll add it to my list of doc things I need to do this week.
I am resolving this fixed now, the 3.6 test group has been successfully cloned and a 3.7 catchall run has been created for now with just the smoketest and the BFT. Before the others are added to the run I think we might want to a do a bit of cleanup and we can add the others later.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Thanks Marcia!
Assignee: nobody → marcia
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.