835553 - document how python works on m-c

Status: RESOLVED INACTIVE

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

Filing this under mozbase, though i'm really talking about python
across the board in m-c:  mozbase, other scripts/librarys,
mozbuild/mach, etc.  Without more than superficial repition, it would
be nice to have a high google-fu place where contributors have a high
level view of:

- what is in python?
- how do the pieces fit together?
- how does our virtualenv/packaging/importing work? (maybe the most
  important question for nuts+bolts work)

Obviously, this should be as DRY as possible, otherwise it won't be
maintainable.  The documentation should probably mostly live on
https://developer.mozilla.org/en-US/docs/Python , which is currently
pretty pathetic.

Mozbase documentation should live on
https://wiki.mozilla.org/Auto-tools/Projects/Mozbase for now.

Comment 1

[Jeff Hammel]

I'll add some stubs on https://wiki.mozilla.org/Auto-tools/Projects/Mozbase today. As time permits, we can outline what all should be documented and a strategy for it

Comment 2

[Jeff Hammel]

Did this one: https://developer.mozilla.org/en-US/docs/Python/Virtualenv#The_Mozilla-Central_Virtualenv

It could really use some fleshing out, such as the format of the packages.txt file (it would be nice to have in-code documentation for that, for that matter) and the bug URL where we decided to use .pth files, etc.

Comment 3

[Jeff Hammel]

And a little stub here wrt mozbase: https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Mozbase_in_Mozilla-Central . Even more unfinished, but I need to turn my attention to other tasks at the moment, though of course I could spend hours/days/weeks on this.

Jumping back, one of my end goals here is to have Mozilla become more part of the python community.  It benefits them and it benefits us.  But most other pythonistas that I've talked to, despite the fact that Mozilla has a considerable python development effort, are downright surprised that Mozilla uses python.  If we want more help and visibility (don't get me wrong, I am impressed with what we've already done here in the last several years), we need to show what we're doing ;) (Of course this also helps us==Mozillians, which is the other more obvious goal.)

We should get on http://python.org/about/success/ , not that anyone cares about that page anymore

Comment 4

[Gregory Szorc [:gps]]

Flagging myself to write up something. If someone else beats me to it, go for it.

Comment 5

[Jeff Hammel]

(In reply to Gregory Szorc [:gps] from comment #4)
> Flagging myself to write up something. If someone else beats me to it, go
> for it.

Nice :)  I will write up more as time goes along; as much as I enjoy writing documentation (which I actually do, ignoring mediawiki) it is time consuming and i had to break off to do some more coding.

Comment 6

[Jeff Hammel]

I noticed that there was absolutely nothing documenting PYTHON_UNIT_TESTS either in-tree or on MDN. SO....I added that: https://developer.mozilla.org/en-US/docs/Mozilla/QA/Automated_testing , https://developer.mozilla.org/en-US/docs/Compiled-code_automated_tests#Python_unit_tests , https://developer.mozilla.org/en-US/docs/How_Mozilla%27s_build_system_works/Makefile_-_variables

Comment 7

[Jeff Hammel]

...but I guess I could have just run the code.  I also wrote some documentation for how writemozinfo.py and mozinfo.py fit into the grand scheme of things (brief, but perhaps sufficient):

= How writemozinfo + mozinfo works =

Mozilla-central has a utility, writemozinfo.py:
http://mxr.mozilla.org/mozilla-central/source/config/writemozinfo.py
. The purpose of this script is to take information from the build
process and for use as input to mozinfo, which then outputs a flat
key, value JSON object, mozinfo.json, at the root of the $(OBJDIR):
http://mxr.mozilla.org/mozilla-central/source/configure.in#9059 .
mozinfo was designed to accept a list of files which are JSON blobs
that override and extend the inspected system parameters:
https://github.com/mozilla/mozbase/blob/master/mozinfo/mozinfo/mozinfo.py#L135
In this way, the build information exposed by writemozinfo.py can be
made available to the test harness via mozinfo.py.  This general
pattern can be used to expose arbitrary environment factors, not
readily available available for inspection.

Comment 8

[Jeff Hammel]

Wrote a very bare stub at https://developer.mozilla.org/en-US/docs/Python#Python_in_Mozilla-Central ; please feel free to flush it out!

Comment 9

[Gregory Szorc [:gps]]

I wrote some stuff in the in-tree build docs at build/docs/python.rst. Viewable at https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/python.html.