1444458 - [taskcluster-cli] replace ping URL scraping with use of the cached API methods

Status: RESOLVED WORKSFORME

(Taskcluster :: Services, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

When you run taskcluster-cli, typically it says

Scraping ping URLs from https://references.taskcluster.net/manifest.json

and takes a little while.  It's almost certain that you're not using those URLs, so this is sort of wasted time.  It's definitely wasted time in that the library already contains information about those ping endpoints in https://github.com/taskcluster/taskcluster-cli/tree/master/apis

So this bug is to factor out that ping URL generation and either use the information in the existing code generation, or add a bit of new code generation that runs at the same time.

Comment 1

[tuhina chatterjee]

Hi  Dustin.:)
Can I start working on this bug ?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Sure, please have a look!

Comment 3

[akshitadvlp]

Hello Dustin,
Is this bug up for solving?

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

I'll assign it to whoever actually starts working on it first :)

Comment 5

[akshitadvlp]

Ohkay Dustin,
Actually, My setup is done for taskcluster-cli but when I write taskcluster signin: It says  command not found.
Am I missing something here?

Comment 6

[akshitadvlp]

and also when I run make generate-apis,
It first fetches the apis and then terminates with
>2018/03/22 17:38:33 error: failed parse API  Index :  json: cannot unmarshal string into Go struct field Entry.scopes of type   [][]string exit status 1

Comment 7

[Pete Moore [:pmoore][:pete]]

Attachment: Github Pull Request for taskcluster-cli

@:dustin
While we wait for a complete solution to bug 1444458, this is a partial solution so the ping urls cache only expires after 365 days, rather than after one day.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Pete, will the work on API generation solve this?  Or a slightly different question: in light of that project, is this bug still a good-first-bug?

Comment 10

[Pete Moore [:pmoore][:pete]]

(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #9)
> Pete, will the work on API generation solve this?

No - the API generation is only useful for the generation of language libraries, tc-cli should interpret the schemas/references directly. I think we can remove the code generation step entirely from tc-cli, and ship it with default references and schemas, that it can refresh on demand (perhaps a separate tc-cli sub command to fetch/update schemas from a root URL, or user can simply pass in the path to a directory that contains local references and schemas).

> Or a slightly different
> question: in light of that project, is this bug still a good-first-bug?

Good point - this probably isn't a good first bug, and we maybe can even close it.

Note, with the changes landed (comment 7) the scraping only happens once per year now (instead of potentially every day), so I think that this issue is far less concerning now.