835603 - write an in-tree selenium script to propagate python ownership

Status: RESOLVED WONTFIX

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

The mozbase versioning portion of workflow,
https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Versioning , has
a pretty serious flaw.  Namely, when we have a new package or we have
a new contributor, we want to add all mozbase "owners" as pypi owners
of all mozbase packages.  See a little bit more in
https://bugzilla.mozilla.org/show_bug.cgi?id=794665 and
https://bugzilla.mozilla.org/show_bug.cgi?id=826874 (and see also
https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Versioning if you
are unfamiliar with the workflow).

I was initially inclined to include this functionality in
versionbump.py, but it seems that selenium is kinda the tool for the
job: it can deal with finding the elements (sadly, annoying in stdlib
python) as well as the CSRF tokens, and logs in and keeps a state.  So
I propose the following tool for inclusion in the mozbase repo
(e.g. `pypi_owners.py`):

- reads a list of owners for a package

- for each released package adds all owners in an in-tree file,
  e.g. pypi_owners.txt, to the pypi owners. Needless to say...you have
  to already be a pypi owner of the package to run it ;)

- (You could obviously target more specifically a subset of
  packages/owners if desired)

- (...and if you needed to, you could have a mode to delete a certain
  owner or set of owners)

- Like versionbump.py, this should not be mirrored to mozilla-central

And the best part: except for the list of packages, this is all
completely generic :)  I wouldn't mind even having this as a separate
mozbase package even though, unlike all other mozbase packages, its
target is currently developers (so I also understand if we don't want
to do that). I would express a strong preference to keeping a generic
selenium script and a mozbase-specific version, so that others can
consume what we write as desired.  I also wouldn't mind having the
generic script not live in the mozbase repo entirely and just live on
pypi and be installed with other extras (e.g. sphinx).  So many
options!

Comment 1

[Jeff Hammel]

I am more than happy to spec out this script (I'll write it eventually on my own, but very low in my priority queue at present), though if :automatedtester is interested he can literally selenium circles around me.

Comment 2

[Jeff Hammel]

For selenium see e.g. http://pypi.python.org/pypi/selenium

Comment 3

[Zuhao(Joe) Chen]

Hi Jeff, could I work on this bug? Recently I've been working on tools/automation for my company using Python thus I think I could learn something by working on this bug.

Comment 4

[Jeff Hammel]

Probably the first thing to do is to make a script that can read pypi owners given a package.  The function should be something like:

def owners(package):
    """                                                                         
    returns a list of pypi owners for a particular package.                     
    the package can be a URL of the package or the package name on              
    http://pypi.python.org/:                                                    
    >>> owners('http://pypi.python.org/pypi/decoupage')                         
    ['k0s', 'ejucovy']                                                          
    >>> owners('decoupage')                                                     
    ['k0s', 'ejucovy']                                                          
    """

Once that's solved, you'll need a function to set the pypi owners:

def set_owners(package, owners):
    """sets the owners of a pypi package"""

You can use the test pypi server for testing this: http://testpypi.python.org/pypi ; at this point it might be more helpful to make a class so that the package index may be stored on the object:

class PyPIowners(object):
    def __init__(self, index='http://pypi.python.org/pypi'):
        self.index = index
    def owners(self):
        pass
    def set(self):
        pass

We'll probably want a command line interface to this, something like:

def main(sys.argv[1:]):
    """                                                                         
    CLI usage:                                                                  
                                                                                
    Set the pypi owners from a text file, one owner per line:                   
    > pypi_owners decoupage set owners.txt                                      
                                                                                
    Get the owners of a package:                                                
    > pypi_owners decoupage list                                                
    k0s                                                                         
    ejucovy
    """

I'm happy to help out on this part once the first two are working.  It'd be nice to have tests, if you're inclined to lean towards TDD, but to be honest I'd probably do that as a follow-up.  We'll also have to deal with auth, which should be reading from ~/.pypirc by default or allowing --username, --password to be set

Once we have this, we'll have a generic piece of software.  We'll need to adapt this to mozbase, either making this a package or what not.  :wlach, what do you think would be appropriate?

The bulk of the work will be using selenium to duplicate the process by hand.  I generally just do this by hand.

Comment 5

[theod]

Any news on someone taking care of that? Is Zuhao(Joe) Chen finally working on that? I'd be interested in working on this bug if no one is.

Comment 6

[Zuhao(Joe) Chen]

(In reply to theod from comment #5)
> Any news on someone taking care of that? Is Zuhao(Joe) Chen finally working
> on that? I'd be interested in working on this bug if no one is.

I'm working on it but I don't know why it still appears to be assigned to nobody...

Comment 7

[Jeff Hammel]

There could be enough work here to break into multiple bugs, if interested. So there is:

- a generic script that will update the pypi owners
- a mozbase-specific script to use an in-tree file of owners

I'm not sure if there is any easy coordination that could happen, but just throwing the idea out there.

Comment 8

[Zuhao(Joe) Chen]

Attachment: First patch for bug 835603

First patch for bug 835603.

Comment 9

[Josh Matthews [:jdm]]

Jeff, could you address the feedback request here?

Comment 10

[Jeff Hammel]

Comment on attachment 724683 [details] [diff] [review]
First patch for bug 835603

Sorry for the delay.  The short answer is the first thing to do is to create a script that will retrieve and add owners to a package *generically* programmatically.  That is, not mozbase-centric.  This should use selenium.  The script is fine for what it is, but doesn't really solve this problem, so I'm giving an f-

Comment 11

[David Burns :automatedtester]

Closing this as the value we would gain is quite low and could put community on to more exciting bugs