1426403 - Enable the Renovate bot on Taskcluster JS repositories

Status: RESOLVED WONTFIX

(Taskcluster :: Services, enhancement)

Description

[Ed Morley [:emorley]]

Neutrino and Treeherder are both using the Renovate bot (a superior Greenkeeper alternative that has native support for yarn) to keep JS dependencies up to date.

It's use has been approved for Mozilla repositories in bug 1416054, plus since then it turns out there is also a "forking" variant of the Renovate app, which needs even fewer permissions (it opens PRs from a fork of the repo so doesn't need code write permissions).

It would be great to have Renovate enabled on the Taskcluster JS repos, in particular those for client libraries - since at the moment taskcluster-client is pulling in an old version of Hawk, meaning the Treeherder frontend bundles two versions of Hawk.

Comment 1

[Ed Morley [:emorley]]

Overview:
https://renovateapp.com/

Security details:
https://renovateapp.com/docs/all-other/data-security

Docs:
https://renovateapp.com/docs/

Version that creates PRs from it's own fork, so needs fewer permissions (should this be desired):
https://github.com/apps/forking-renovate

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

We are pulling an old version of Hawk intentionally -- the new version of Hawk has a very narrow range of supported Node versions that does not support the version we are using on some of our services
  https://github.com/hueniverse/hawk/commit/8dc1c1d43338a0a8aa9d38df0d688fe80b8fcded#diff-b9cfc7f2cdf78a7f4b91a753d10865a2L16

Once that's changed, or once all of our services are running Node 8.9.0 or higher, we will upgrade it.

That said, I'm happy to have you work on enabling this for or repos, if you're interested!  If you don't want to do it, can I mark you as a mentor for it?

Comment 3

[Ed Morley [:emorley]]

Ah makes sense (that limitation needn't apply for taskcluster-client-web at least though, which is handy).

The Treeherder team is really understaffed, so ideally we're looking for help from elsewhere in the org, rather than the other way around - but more than happy to mentor this.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

https://github.com/taskcluster/taskcluster-client-web/pull/23 for that library.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

That doesn't work by just upgrading: https://github.com/hueniverse/hawk/issues/217

So this will need someone to have a look at it and modify all of the uses of hawk to be compatible with the new version.

Comment 6

[Ed Morley [:emorley]]

This bug was about using a bot to open dependencies in general, rather than just for updating Hawk though? Could you revert the changes?

Comment 7

[Ed Morley [:emorley]]

s/open dependencies/open PRs for updating dependencies/

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

I don't think we want to use the bot for that purpose -- I read your suggestion as a generalization of the more specific need for Hawk.  I can open a new bug for Hawk, in which case I'm inclined to WONTFIX this one.

Comment 9

[Ed Morley [:emorley]]

Ah right. Are there specific concerns about the use of a bot like that, that I can either alleviate or file issues against the bot as feature requests?

In the general case I see having up to date dependencies as a good thing, and therefore any automation that helps a human update those dependencies (when they've had a chance to review them) as a good thing.

Eli, could you give some thoughts here too, as someone who set up Renovate on the Neutrino repo?

Comment 10

[:Eli Perelman]

I love the Renovate bot we have on Neutrino. It isn't perfect, but it is improving. It's basically like Greenkeeper, which I thought we wanted to use in the past. What is the argument for not wanting to automate PRs for dependencies?

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

I'm worried it will open a lot of PRs, mostly -- we ship new versions of our libraries every few days, and have a lot of services.  Also, our tests are of varying quality and may not always catch bugs caused by library version changes, so there's some risk involved.

I'm not opposed to it, I just want to have a conversation about how we will handle it.  Just opening 100's of PRs that we all ignore won't be helpful.

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

This is probably not a good mentored bug anyway.

I think Snyk is another option.  Eli, can you compare the merits of the two?  Maybe we can start this with one of our services or libraries and see how it feels.

Comment 13

[:Eli Perelman]

Snyk is about monitoring the security vulnerabilities of a project's dependencies.
Renovate is about automating the updating of a project's dependencies.

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

This would be fine as a housekeeping thing if it was something someone on the team was interested in implementing.  But nobody seems that interested.  We're getting security vuln warnings from github itself.  So I'm going to close as WONTFIX until someone gets the interest to do this.

Comment 15

[Ed Morley [:emorley]]

Having Renovate enabled can be a big timesaver for maintainers, avoids stale dependencies in the dependency tree (thereby reducing the amount of de-duplication yarn can perform), and is extremely quick to set up (but requires admin permissions, so isn't really something that's easy for contributors to do).

For me personally, seeing a repo that doesn't have a bot enabled to update dependencies, is the 2018 equivalent of seeing a library with no linting set up -- a bit puzzling and clear low-hanging fruit. But I understand that maybe it will take a bit longer for everyone else to come around to that way of thinking :-)