we need product build numbers



7 years ago
8 days ago


(Reporter: camd, Unassigned)





7 years ago
When I was thinking why someone would have multiple test runs in a cycle, I came up with 2 reasons:
1. The different test runs apply to different parts of the product
2. The different test runs apply to the same parts of the product, but different builds

1 is handled just fine.  But we don't handle 2.

Web apps don't tend to have build numbers, so they won't really care about this.  But products like Firefox do, as do most other products that get "built."

So here's my build proposal:

Build numbers can be any string

* Products
** build numbers are optional (web apps often don’t have a version)
** can have many build numbers.  Can be added at any time.

* Test Cycles
** If product has build numbers, you can assign them to cycles.  If you do, then runs inherit that build number and can’t be different from the cycle.
** cycles do not have to have a build number assigned

* Test Runs
** If the product has build numbers, then a run needs a build number.
** It can be inherited from the cycle, if set
** or you can set it via drop-down in the run edit page.
** but if the product has no build numbers, it’s not required.
** you can also create a new build number for the product from the “new run” dialog
** once a test run has been activated and has test results, you can no longer change the build number

* Viewing
** run or manage details needs to show the build number for the test run

* Running
** When running a test, the build number the tester should be running against should show at the top of the run page somewhere (perhaps with the breadcrumb?)

Comment 1

7 years ago
A Pivotal Tracker story has been created for this Bug: http://www.pivotaltracker.com/story/show/20704405

Comment 2

7 years ago
Carl Meyer added a comment in Pivotal Tracker:   
This is certainly all doable, but it also strikes me that its a lot of added feature for potentially not much gain over simply a convention of putting the build number in the test run (or cycle) name. It seems to me that the latter is flexible enough to cover a wider range of workflows, and what else are you gonna use the test run name for? 

If its important to have this metadata be enforced by the product (i.e. you can't trust test managers to name the test runs correctly), then that would be a reason to do this as outlined.

Comment 3

7 years ago
Cameron Dawson added a comment in Pivotal Tracker:   
:)  K.I.S.S. ...  Yep...  I think your point is a good one.  And I think this is an excellent use of our dogfooding period to see if people run into frustration with the subject.  I'm going to put this story way at the bottom of the priority, and we'll see if we get an outcry for this functionality.

If the user wants the test cycle to be about the build, then name the cycle with that build number.  If it's the run that's specific to the build, then use it in the name of the run.  

But I will make sure to document this in a screencast.  so adding a tag for that.

Comment 4

7 years ago
Cameron Dawson added a comment in Pivotal Tracker:   
I wonder if a better way to handle this would be to have tags for test runs (and possibly test results)

The test run creator could tag the run with the build that they want it to apply to.

Also: as Jan Leger mentioned, they may want to have 1 run, but have the user continue with the next day's build and pick up where they left off.  What if we had a field during run that said "tags to apply to all results"?  the user would set that to the build number (and anything else they want) and then every result they set has that build number on it.

This wouldn't require a special field, and may have all kinds of nice uses that I can't think of...  :)

Comment 5

7 years ago
Cameron Dawson added a comment in Pivotal Tracker:   
Recommend: just use tags for build numbers.

Comment 6

6 years ago
Cameron Dawson deleted the linked story in Pivotal Tracker

Comment 7

6 years ago
This is handled by the new runseries feature.  closing.
Last Resolved: 6 years ago
Resolution: --- → FIXED

Comment 8

5 years ago
(In reply to Cameron Dawson [:camd] from comment #5)
> Cameron Dawson added a comment in Pivotal Tracker:   
> Recommend: just use tags for build numbers.

(In reply to Cameron Dawson [:camd] from comment #7)
> This is handled by the new runseries feature.  closing.

I don't understand what the above comments mean, if build ID is being used.

And if build id is not being used, how do bug 879929 and bug 906000 fit in?


8 days ago
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.