Closed Bug 838733 Opened 7 years ago Closed 7 years ago

generate_diff.py should have a convenience mode for staging

Categories

(Testing :: Mozbase, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: k0scist, Assigned: k0scist)

References

Details

Attachments

(2 files)

It is now relatively easy to mirror mozbase changes to mozilla-central
from github.  However, if you are developing mozbase-based
infrastructure on m-c, as we are for
https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Automation-Refactor
, we need a way to add the HEAD (prerelease) of certain packages to
e.g. a patch queue for development and testing.  This will avoid
multiple unnecessary releases and generally make life hummy-chummy.
Blocks: 838374
Assignee: nobody → jhammel
Looking at the code, I think I am inclined to add a --develop flag that will enable this for all non-specified dependencies.  That is, for

  ./generate_diff.py mozprocess

but not

  ./generate_diff.py mozrunner=5.14

(I could go with --staging or --stage if there are strong opinions about the flag name.)

The current behaviour if you do not specify a tag, generate_diff.py will read it from setup.py. Note that this is not what is desired for this bug, since this will require that the tags exist.  Since the tags are created at package release time, and we want to test with pre-released packages here (the whole point of this bug), that dog won't hunt.

The alternative that I can think of is using a special marker, e.g. `dev`, to point to dev versions:

  ./generate_diff.py mozrunner=dev

(Could also be `tip`, `HEAD` etc.)

Thinking it over, while the latter way appears more straight-forward, I prefer the former way for a few reasons:

* When you want to do this, you generally want to do this across the board.  `generate_diff.py mozrunner=dev mozprocess=dev mozprofile=dev` is just silly, and error-prone.

* While I may or may not add any along with this bugs, a --develop flag could also enable other conveniences for staging.  generate_diff.py, along with other write-in-pen scripts, is difficult to debug because it tries to ensure sanity before writing in pen, since you're sure as hell not going to ensure sanity otherwise. (To be fair, it writes in pen and reverts, but you need to make sure you are getting back to the right state.  Longer explanation available on request but please lets use email or irc; doesn't belong in this bug.) --develop could be used to make this somewhat easier.  You might also imagine other cute features for staging as would make whoever's life easier.

* As hinted at earlier, doing `mozrunner=HEAD` or other revision could be useful for staging (and, as per policy *STAGING ONLY!*), I view this as other functionality, not something we'll want in general.  Of course, I give HEAD as an example, but obviously not HEAD since that's the same as unspecified. Sigh.  Communicating this badly.  Hope you get idea. -- caveman
Addendum: but meant to add, if anyone has any strong opinions...now's the time to speak up ;)
P.P.S. Also, there is the niceity that if you have a command line for --develop and you bump versions, you should be able to use the same command line, sans `--develop`, post-version bumping assuming you don't screw up anything.  This makes me think that we have workflow, and where there is definable workflow, there is space to make things easier.  That whole "automation" thing the kids are talking about these days...
Attached patch patchSplinter Review
Could do more here, but I *think* this suffices.

:ahal, feel free to pass on the review if you're busy and/or you think e.g. Will is more appropriate.
Attachment #724038 - Flags: review?(ahalberstadt)
Comment on attachment 724038 [details] [diff] [review]
patch

Review of attachment 724038 [details] [diff] [review]:
-----------------------------------------------------------------

Makes sense to me
Attachment #724038 - Flags: review?(ahalberstadt) → review+
Thanks for the quick review.

Going to land and close the bug as this shouldn't hurt anything, but something may not be right.  See https://bugzilla.mozilla.org/show_bug.cgi?id=838374#c15 . Rather, something isn't right, but slowly marching forward...
Summary: generate_diff.py (or other) should have a convenience mode for staging → generate_diff.py should have a convenience mode for staging
pushed: https://hg.mozilla.org/mozilla-central/rev/6c309875420c

Will close but reserving the right to reopen later
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
There is one serious bug fixed here:  git HEAD does not mean what i thought it meant.  Instead, AIUI, we want master.  

There's also the minor bug fix wrt forgetting store_true on --develop, a --no-check mode for easier debugging, and a TODO item for the future.
Attachment #724515 - Flags: review?(ahalberstadt)
Comment on attachment 724515 [details] [diff] [review]
follow up: a bug fix and some more niceness

Review of attachment 724515 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with one optional suggestion

::: testing/mozbase/generate_diff.py
@@ +360,5 @@
>  
>          # generate the diff and write to output file
> +        command = ['hg', 'addremove']
> +        # TODO: don't add untracked files via `hg addremove --exclude...`
> +        call(command, cwd=hg_root)

So does this mean that --no-check doesn't really work yet? If that's the case maybe we should just add it in when it does.
Attachment #724515 - Flags: review?(ahalberstadt) → review+
--no-check works currently, for what I intended it for: not checking if any changes have been made to m-c or if any untracked files are there.

The TODO, while vaguely related, is another matter.  I'll file a follow-up about it.
(In reply to Andrew Halberstadt [:ahal] from comment #9)
> Comment on attachment 724515 [details] [diff] [review]
> follow up: a bug fix and some more niceness
> 
> Review of attachment 724515 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> r+ with one optional suggestion
> 
> ::: testing/mozbase/generate_diff.py
> @@ +360,5 @@
> >  
> >          # generate the diff and write to output file
> > +        command = ['hg', 'addremove']
> > +        # TODO: don't add untracked files via `hg addremove --exclude...`
> > +        call(command, cwd=hg_root)
> 
> So does this mean that --no-check doesn't really work yet? If that's the
> case maybe we should just add it in when it does.

pushed: https://hg.mozilla.org/mozilla-central/rev/8e68f4d73ec4
(In reply to Jeff Hammel [:jhammel] from comment #12)
> Added minimal documentation at
> https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Using_generate_diff.py

Nice, thanks Jeff.
You need to log in before you can comment on or make changes to this bug.