974184 - Port mozbase versionbump.py to hg

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Ed Morley [:emorley]]

Broken out from bug 949600 comment 27.

Bug 949600 moved mozbase from https://github.com/mozilla/mozbase/ to https://hg.mozilla.org/mozilla-central/file/tip/testing/mozbase/

versionbump.py was deleted as part of the move, since it needs to be rewritten for hg & the functionality for repo tagging + diff-against-mozilla-central dropped.

The original, for reference:
https://github.com/mozilla/mozbase/blob/old-master/versionbump.py

Comment 1

[Ed Morley [:emorley]]

See also bug 926252 for some potential versionbump.py cleanup that didn't land before the github repo was depreciated.

Comment 2

[Ed Morley [:emorley]]

And also the following:
Bug 794553 - versionbump.py doesn't work with untracked files in the repo
Bug 826874 - versionbump.py should preemptively check for pypi ownership
Bug 831499 - versionbump.py should give difference from in-tree version from pypi version
Bug 835586 - versionbump.py should make use of `setup.py register`
Bug 841138 - versionbump.py should create a "mirror to puppetagain" bug
Bug 841944 - versionbump.py should ensure valid python versions
Bug 906572 - Verify that binaries have been uploaded to PyPi when running versionbump.py 
Bug 926410 - Not all versionbump.py failures clean up after themselves, causing "directory unclean" failures on next run

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It's currently a major pain in finding out details about the dependency tree between packages in mozbase without this script! Especially when we even don't have tags for releases. Can we please get at least the --info option working? That would be really appreciated.

Comment 6

[Luv Agarwal(:lagarwal)]

I want to take this up. Please assign it to me.

Comment 7

[Luv Agarwal(:lagarwal)]

Currently when we place versionbump.py in the mozbase directory and run 'python versionbump.py --info' then it gives the following output

mozfile 1.1 : 
moztest 0.6 : mozinfo
mozlog 2.0 : 
mozinfo 0.7 : mozfile >= 0.12
mozcrash 0.12 : mozfile >= 1.0, mozlog
mozversion 0.6 : mozdevice >= 0.29, mozfile >= 1.0, mozlog >= 1.5
mozInstall 1.10 : mozinfo >= 0.7, mozfile >= 1.0
mozprocess 0.19 : mozinfo
mozhttpd 0.7 : moznetwork >= 0.24
mozrunner 6.1 : mozcrash >= 0.11, mozdevice >= 0.37, mozfile >= 1.0, mozinfo >= 0.7, mozlog >= 1.5, mozprocess >= 0.17, mozprofile >= 0.18
manifestparser 0.6 : 
mozsystemmonitor 0.0 : psutil >= 0.7.1
moznetwork 0.24 : mozinfo
mozdevice 0.37 : mozfile >= 1.0, mozlog, moznetwork >= 0.24
mozprofile 0.21 : manifestparser >= 0.6, mozfile >= 1.0, mozlog


I think that this is what's required and so '--info' part according to me is working correctly. Rest other part of the script need to edited which i will do soon.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Great to see that you are working on it Luv! I assume Andrew can be the owner here, given that he did all the movement to mozilla-central for that repo.

Comment 9

[Luv Agarwal(:lagarwal)]

Andrew, i have problem regarding a line of code in the old versionbump.py file:

after executing line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L85 .egg-info directories would be created and after this when the line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L147 would be executed then it will consider the .egg-info directories which are created above. So it will show the status unclean and the program would be terminated. So how was this script used before.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Luv, you may want to add those subfolders to the .hgignore file. If there is no such file, it may have to be created. On github we had .gitignore, which you can still find in the repository but the old-master branch: https://github.com/mozilla/mozbase/blob/old-master/.gitignore

Comment 11

[Luv Agarwal(:lagarwal)]

Andrew, i have following problems in the old versionbump.py file:

1. after executing line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L85 .egg-info directories would be created and after this when the line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L147 would be executed then it will consider the .egg-info directories which are created above. So it will show the status unclean and the program would be terminated. So how was this script used before.

2. what is the use of line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L208 as we have not added any content to the files yet so this won't do anything. so should i remove it.

Comment 12

[Andrew Halberstadt [:ahal]]

To be honest, I'm not really sure. I never used this script for the exact reason you described above (I almost always have a dirty tree). So I'm not really sure why I was flagged as a mentor for this :). I know my preference would be to for this script to work even if the tree is dirty.. but I think there was opposition to that in the past.

I think Will may have used this before. Will do you know the answer to :lagarwal's questions?

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The reason was that when people bumped the version and pushed to pypi, unwanted files were in those packages. Those can even break a release, as what we have seen in the very beginning with Mozmill. So Jeff added this restriction to the version bump script.

Comment 14

[William Lachance (:wlach)]

Most of the functionality of versionbump.py is obsolete. I think it's best if we just lay out instructions clearly in https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Versioning (I added a note about making sure package directories are clean before uploading)

The one exception is the --info option. We could create a versioninfo.py with just that functionality ported over.

Comment 15

[Luv Agarwal(:lagarwal)]

Attachment: port_versionbump_git_to_hg.patch

Hey, the additions and changes are as follows:

1. Replaced the git commands with hg
2. Tried to improve the display in --dry-run but as there can be different sets of ways of running this script so it need to be decided how to implement --dry-run
3. before this script used to pull the tags explicitly in line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L297 even though we have pulled in line https://github.com/mozilla/mozbase/blob/old-master/versionbump.py#L190 so i commented this thing.
4. Also i have considered the changes proposed by edmorley in bug 926252 which were not ported

Comment 16

[William Lachance (:wlach)]

Comment on attachment 8458297 [details] [diff] [review]
port_versionbump_git_to_hg.patch

Review of attachment 8458297 [details] [diff] [review]:
-----------------------------------------------------------------

I feel bad r-'ing what is clearly a fair amount of work, but I don't think we want this to go in. We require explicit review when changing mozilla-central, which this doesn't really account for. Likewise, we shouldn't be adding tags to mozilla-central either. As I said before, I think we're better off just doing most of what versionbump.py used to do manually. It's a bit of extra work, but nothing drastic. Per comment 14 though, I think a versioninfo.py which listed dependencies could be useful. Would you be able to provide a patch which provides just that?

Comment 17

[Luv Agarwal(:lagarwal)]

Ya sure i could do that but i looked at the comments and patches related to this bug and thought that mozilla is interested in doing this so i did it and about the review flag, after reading the comments regarding this script it seemed that you have worked on this script so i just r- ed it to you and i am sorry.

Thank you

Comment 18

[Luv Agarwal(:lagarwal)]

Attachment: versioninfo.patch

Comment 19

[William Lachance (:wlach)]

(In reply to Luv Agarwal(:lagarwal) from comment #17)
> Ya sure i could do that but i looked at the comments and patches related to
> this bug and thought that mozilla is interested in doing this so i did it
> and about the review flag, after reading the comments regarding this script
> it seemed that you have worked on this script so i just r- ed it to you and
> i am sorry.
> 
> Thank you

Yeah, this bug has been a bit confusing. I do apologize for that.

Comment 20

[William Lachance (:wlach)]

Comment on attachment 8458882 [details] [diff] [review]
versioninfo.patch

I'll take this review.

Comment 21

[William Lachance (:wlach)]

Comment on attachment 8458882 [details] [diff] [review]
versioninfo.patch

Review of attachment 8458882 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good! Some minor nits, which I'll fix up myself.

::: testing/mozbase/versioninfo.py
@@ +7,5 @@
> +
> +# import setup_development.py from the same directory
> +import setup_development
> +
> +here = setup_development.here

No need to define this separately, since we only use it in one place.

@@ +8,5 @@
> +# import setup_development.py from the same directory
> +import setup_development
> +
> +here = setup_development.here
> +REPOSITORY_URL = 'https://hg.mozilla.org/mozilla-central/'

this line isn't needed

Comment 22

[William Lachance (:wlach)]

Attachment: Patch with nits addressed

Comment 23

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f4df26b326fd

Comment 24

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/f4df26b326fd