1057314 - Guess the build directory for Try jobs, so we can link to them before the jobs have completed, like TBPL

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

TBPL currently links to the build binary/log directory once a job has completed, and bug 1045846 is filed for matching this in treeherder (and is a blocking bug, since it is preserving a TBPL feature).

Something that TBPL doesn't do, is show the link to the build directory for non-try repos, if the job hasn't yet completed.

This is useful since the binaries are uploaded quite a bit before the build job is complete, since the build is still performing |make check| tests.

For try jobs, TBPL currently guesses the build directory even when the jobs are still running, since the location is predictable.

For non-try the location is something like:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux64-debug/1408700153/

Where the "1408700153" is some buildid/timestamp, that I'm not sure we have access to when the job is only listed on builds-running.js

As such for this bug we'll have to:
1) Double check the build id isn't available or inferable from the contents of builds-running.js
2) If not, adjust builds-running.js so that it is.

Comment 1

[Ed Morley [:emorley]]

Bug 1045846 didn't fix the "try job that is not yet completed" case that TBPL supports, so this bug is about both try and non-try now.

As part of fixing this bug we should add a property for the build directory, so the UI can use that directly rather than having to infer it from the log URL.

Comment 2

[Armen [:armenzg]]

For a running job, how do you query which properties are already set?
Do you query differently for a completed job?

Comment 3

[Ed Morley [:emorley]]

Same API call, the UI only shows the links when there is a log URL present in the response:
https://github.com/mozilla/treeherder-ui/blob/0dafe7da0d140a99ba3fe468a90fa28eda50463d/webapp/app/plugins/pluginpanel.html#L161

Comment 4

[Ed Morley [:emorley]]

We may be best splitting this bug into two:
1) Re-implement the TBPL feature that bug 1045846 omitted - ie: fake up the URl for not-yet-completed _Try_ pushes, since they are more predictable.
2) Try to figure out if we can derive the URL before a job has completed for non-try pushes. This will likely require upstream buildbot changes and so be harder (or impossible/not worth it if we're going to switch to taskcluster at some point).

For #1, let's do this client side for now and leave migrating the buildURL handling to the service to bug 1060649.

I'll file a new bug for #2.

The location for the current handling is:
https://github.com/mozilla/treeherder-ui/blob/aa213c042d6204bfdd4c3b085504eb74b1e4d66e/webapp/app/js/models/job_log_url.js#L45

Comment 5

[Ed Morley [:emorley]]

(In reply to Ed Morley [:edmorley] from comment #4)
> 2) Try to figure out if we can derive the URL before a job has completed for
> non-try pushes. This will likely require upstream buildbot changes and so be
> harder (or impossible/not worth it if we're going to switch to taskcluster
> at some point).
...
> I'll file a new bug for #2.

In fact, I'm not even going to file another bug: looking at builds-running the field we need just isn't there - and I doubt we'll be able to get it added by release engineering, and I think the win is much smaller than compared to Try (which we can do already).

Comment 7

[Ed Morley [:emorley]]

ftp.m.o is going away soon, likely in Q3.
I'm sure this will change the way we have to handle things like this.

Comment 8

[Mike Hommey [:glandium]]

(In reply to Ed Morley [:emorley] from comment #7)
> ftp.m.o is going away soon, likely in Q3.

what the what? Where can I read about this?

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

(In reply to Mike Hommey [:glandium] from comment #8)
> (In reply to Ed Morley [:emorley] from comment #7)
> > ftp.m.o is going away soon, likely in Q3.
> 
> what the what? Where can I read about this?

http://atlee.ca/blog/posts/releng-retrospective-q1-2015.html

Comment 10

[Armen [:armenzg]]

https://ftangftang.wordpress.com/2015/04/21/changes-coming-to-ftp-mozilla-org/

If the build is run and if we get pulse messages per mozharness steps we could listen to the "generate-build-stats".
06:32:02     INFO -                 u'packageUrl': u'https://queue.taskcluster.net/v1/task/ewVl6f2gQA2ztchU2h4LJA/artifacts/public/build/firefox-40.0a1.en-US.linux-x86_64.tar.bz2',

We could tell mozharness to give us that info as early as possible by giving us pulse messages.

For some other jobs, we can grab the ftp dir before a build is done by listening to pulse messages (hopefully) for steps called "set_script_properties":
http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.py#l5155

Comment 11

[Ed Morley [:emorley]]

Treeherder doesn't consume releng Pulse messages at all, since initial attempts to do so when Treeherder was first created showed that it was missing too much data to be useful, and attempts at migrating over later have been wontfixed since "everything's moving to taskcluster".

Ideally we'd:
1) Allow API submitters (ie taskcluster) to submit a build directory URL, and ideally able do so whilst the job is still running
2) Make taskcluster do so
3) Update the treeherder UI accordingly

However
(a) Taskcluster if anything is looking at hiding that info until the job is complete (see bug 1155645 - motivation is bug 1154248 comment 14)
(b) with artefacts uploaded to S3, do we even have a "build directory" any more? If not, this all needs to change anyway.

Comment 12

[Ed Morley [:emorley]]

I've added the Treeherder use case to https://etherpad.mozilla.org/ateam-ftp-consumers

Comment 13

[Armen [:armenzg]]

As you hint, should we drop the "build directory" concept?

Comment 14

[Ed Morley [:emorley]]

We need a way to solve these use cases:
1) A dev is trying to fix an issue that a member of the public has reported, and pushes to try with extra debugging/a prospective fix. They want to paste a URL in the bug and say to the bug reporter "please try the build(s) there for me". They ideally want to have the URL ready to paste, before the job has completed.
2) A dev wants to find the symbols/some other file that gets put in the uploaded build directory at present.
3) A downstream consumer of Treeherder's API wants to find out the binary location for a job, so it can download it and run it. eg: a bisecting tool that uses Treeherder's API to (a) avoid known busted builds, (b) as a data source to find existing binaries, to avoid having to compile each one locally.

#2 could be solved by having links to these in the job artefacts, and displaying them in the UI like we do tinderboxprint links.
Same for #3 (minus the UI part).

#1 is harder, since often the dev wants "one link" to be able to paste in a bug, since there may be a few people in the bug and they don't know which platforms they will be trying. I guess we could just have a new Treeherder /embed/ endpoint for surfacing all build links for jobs on that revision?

Comment 15

[Ed Morley [:emorley]]

Build directories don't really exist in an S3 world. Instead people can link to the files list on the taskcluster task inspector.