1287935 - Need a way to get the Nightly revisions for a specific day

Status: RESOLVED WORKSFORME

(Tree Management :: Treeherder: API, defect)

Description

[Matthew N. [:MattN]]

Prior to bug 1287501 I would use the following to get all Nightly revisions for a day from a daily cron job:
/api/project/mozilla-central/jobs/?count=100&last_modified__gte=2016-07-19T00:00:00.000&last_modified__lte=2016-07-19T23:59:59.999&job_type_name=Nightly&exclusion_profile=false

but now that returns an HTTP 400:
> "`last_modified` of 2016-07-19T00:00:00.000 is not within last 30 minutes" 

What can I replace this with? I'm trying to get the Nightly revisions so I can get the artifacts from M(ss) jobs for those builds.

Comment 1

[Matthew N. [:MattN]]

Taking a step back, this is what I'm currently doing for fetch_screenshots[1] when --nightly is used which is run from a cron job with the current date:
1) Figure out which result sets have Nightlies for the last day with last_modified__* on …/jobs/ and job_type_name="Nightly"
2) For each resultset ID from #1, fetch the resultset
3) For each resultset, fetch the job_type_name="Mochitest Browser Screenshots" jobs
4) For each job, download the image artifacts for the job using …/jobdetail/?job__guid=%s

At a high level I need to download the image artifacts for the nightlies of a given date.

[1] https://github.com/mnoorenberghe/mozscreenshots/blob/master/mozscreenshots/fetch_screenshots.py

Comment 2

[Ed Morley [:emorley]]

Some additional context:

<emorley> are they taskcluster jobs?
<MattN> not yet
<emorley> just since I think they have a handy api for getting by job name
<emorley> if all you need is the artifact etc
<emorley> in fact, it almost sounds like you want a listener to the Pulse events that are published as builds complete, that can then fetch the screenshot for you?
<MattN> I'll have to make it work with TC eventually
<MattN> yeah, that's one solution which I have been putting off
<MattN> I want that for Try pushes mostly
<MattN> for a different use case

Comment 3

[Matthew N. [:MattN]]

An API for this would also be useful for mozregression which currently seems to crawl the build directories to figure out what Nightlies are available IIUC.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It doesn't matter if Nightly builds are getting created via Taskcluster or Buildbot. At least the revisions are getting indexed in various ways. It means you could easily query the TC index for them. Here an example for all nightly builds on mozilla-central on July 18th:

https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-central.nightly.2016.07.18.revision/gecko.v2.mozilla-central.nightly.2016.07.18.revision

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Matthew N. [:MattN] from comment #3)
> An API for this would also be useful for mozregression which currently seems
> to crawl the build directories to figure out what Nightlies are available
> IIUC.

No, mozregression should already use Taskcluster for that part.

Comment 6

[Matthew N. [:MattN]]

(In reply to Henrik Skupin (:whimboo) from comment #5)
> (In reply to Matthew N. [:MattN] from comment #3)
> > An API for this would also be useful for mozregression which currently seems
> > to crawl the build directories to figure out what Nightlies are available
> > IIUC.
> 
> No, mozregression should already use Taskcluster for that part.

That's all I needed to know. I've switched to using TC too.