Closed Bug 1490128 Opened Last year Closed Last year

Upgrade json-e in the Firefox source code

Categories

(Firefox Build System :: Task Configuration, enhancement)

enhancement
Not set

Tracking

(firefox64 fixed)

RESOLVED FIXED
mozilla64
Tracking Status
firefox64 --- fixed

People

(Reporter: dustin, Assigned: dustin)

Details

Attachments

(1 file, 1 obsolete file)

We've released a few new versions of json-e.  Let's upgrade the one used in tree.
Attached file Bug 1490128: upgrade to json-e 2.7.0 (obsolete) —
Notably, this adds support for `$match`
OK, I made this more or less by hand, and I think that was incorrect.  In the hg history I learned about `mach vendor python`.  So I'll give that a shot.  But first, hg is refreshing my stablerange cache, so it could be a while :)
Hm, Dave, I tried running `./mach vendor python json-e==2.7.0`, but it seems to update lots of other things, none of which json-e depends on (it has no dependencies):

(sandbox) dustin@lamport ~/p/m-c (bug1490128) $ hg status
M Pipfile
M Pipfile.lock
M third_party/python/atomicwrites/MANIFEST.in
M third_party/python/atomicwrites/PKG-INFO
M third_party/python/atomicwrites/README.rst
M third_party/python/atomicwrites/atomicwrites/__init__.py
M third_party/python/atomicwrites/docs/conf.py
M third_party/python/atomicwrites/setup.cfg
M third_party/python/atomicwrites/setup.py
M third_party/python/atomicwrites/tests/test_atomicwrites.py
M third_party/python/certifi/PKG-INFO
M third_party/python/certifi/certifi/__init__.py
M third_party/python/certifi/certifi/cacert.pem
M third_party/python/certifi/setup.py
M third_party/python/funcsigs/docs/index.rst
M third_party/python/json-e/PKG-INFO
M third_party/python/json-e/jsone/builtins.py
M third_party/python/json-e/jsone/render.py
M third_party/python/json-e/package.json
M third_party/python/json-e/setup.py
M third_party/python/py/AUTHORS
M third_party/python/py/CHANGELOG
M third_party/python/py/PKG-INFO
M third_party/python/py/README.rst
M third_party/python/py/appveyor.yml
M third_party/python/py/bench/localpath.py
M third_party/python/py/doc/example/genhtml.py
M third_party/python/py/doc/example/genhtmlcss.py
M third_party/python/py/doc/example/genxml.py
M third_party/python/py/doc/install.txt
M third_party/python/py/py/_io/terminalwriter.py
M third_party/python/py/py/_version.py
M third_party/python/py/setup.py
M third_party/python/py/tox.ini
A third_party/python/json-e/README.md
A third_party/python/py/testing/io_/test_terminalwriter_linewidth.py

I'm guessing those are transitive dependencies of *other* things that aren't listed in Pipfile?  Is there any way to avoid updating those at the same time?  Shouldn't the Pipfile.lock do so?
Flags: needinfo?(dave.hunt)
(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #3)
> I'm guessing those are transitive dependencies of *other* things that aren't
> listed in Pipfile?  Is there any way to avoid updating those at the same
> time?  Shouldn't the Pipfile.lock do so?

Unfortunately the default behaviour of pipenv is to update all transient dependencies. Many users agree that this is not desirable, and the `--keep-outdated` command line option doesn't quite do what is expected. This has at least been acknowledged, and there are suggestions that it fixed in the near future to allow updating only the targeted packages and any direct transient dependencies. I'll open a bug for this issue as we'll at least need to update the version of pipenv. In the meantime, a workaround is to tweak the Pipfile.lock file by hand to remove the undesired package updates. Once done, running |mach vendor python| *should* remove the changes from third_party.
Flags: needinfo?(dave.hunt)
To be clear, these aren't transient dependencies at all (direct or indirect) -- json-e has no dependencies.
Sadly, that workaround helpfully returned the changes to Pipfile.lock.  If you're comfortable reviewing a diff that upgrades all the things, I can just push that to phabricator.  Otherwise I can to to do a surgical `hg commit`.
OK, that surgery was pretty simple, patch in a moment.
This was done with |mach vendor python| followed by a manual backout of changes
to unrelated packages.
(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #6)
> To be clear, these aren't transient dependencies at all (direct or indirect)
> -- json-e has no dependencies.

That's correct, they're dependencies of other packages. Ideally pipenv would only update the dependencies of the package you're vendoring, taking into account constraints from the existing packages and dependencies.

(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #7)
> Sadly, that workaround helpfully returned the changes to Pipfile.lock.  If
> you're comfortable reviewing a diff that upgrades all the things, I can just
> push that to phabricator.  Otherwise I can to to do a surgical `hg commit`.

The following worked for me:

1. mach vendor python json-e==2.7.0
2. Copy _meta>hash>sha256 and default>json-e values from Pipfile.lock
3. hg revert Pipfile.lock
4. Paste values from step 2 in Pipfile.lock
5. mach vendor python

With the exception of a single .rst file from funcsigs (maybe related to bug 1483651) I then only had changes related to json-e:

> $ hg status
> M Pipfile
> M Pipfile.lock
> M third_party/python/funcsigs/docs/index.rst
> M third_party/python/json-e/PKG-INFO
> M third_party/python/json-e/jsone/builtins.py
> M third_party/python/json-e/jsone/render.py
> M third_party/python/json-e/package.json
> M third_party/python/json-e/setup.py
> A third_party/python/json-e/README.md

We could automate this, however it will become problematic when the package being vendored has dependencies.
Attachment #9007927 - Attachment is obsolete: true
Comment on attachment 9008060 [details]
Bug 1490128: upgrade json-e==2.7.0

Dave Hunt [:davehunt] ⌚️UTC+1 has approved the revision.
Attachment #9008060 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/25aa8af087a1
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
You need to log in before you can comment on or make changes to this bug.