Closed Bug 1057345 Opened 10 years ago Closed 10 years ago

logviewer: "builder" rather than "Buildbot job name" is shown

Categories

(Tree Management :: Treeherder, defect, P2)

Product:
Tree Management
Component:
Treeherder
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: emorley, Assigned: jeads, Mentored)

References

Details

(Whiteboard: [good first bug]) 

eg:
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=282513&repo=mozilla-central

builder: mozilla-central_panda_android_test-remote-tspaint

vs the main treeeherder UI...

Buildbot job name: Android 4.0 Panda mozilla-central talos remote-tspaint

Is 'builder' used for anything?
If not, let's just ditch it and use "Buildbot job name" instead, for parity with TBPL.
Summary: Structured log viewer displays "builder" rather than "Buildbot job name" → Structured log viewer lists "builder" rather than "Buildbot job name"
Summary: Structured log viewer lists "builder" rather than "Buildbot job name" → logviewer: "builder" rather than "Buildbot job name" is shown
Just curious, is this a bug which prevents adoption of Treeherder? Afaik preventing adoption is the criteria for being marked as a blocker.. but that's just my understanding and it could be wrong :)

I agree they should be consistent if they are the same thing. A consideration in the proposed change is the additional space consumed in the structured log for what is a much longer string, and which would make that block ragged looking, and might require reduced font sizes for that top run-data container.

Builder appears to be present throughout the treeherder repos. And of course in the raw log. I think it gets mapped here to 'Buildbot job name', for use in the jobs detail panel.
https://github.com/mozilla/treeherder-ui/blob/1ab94c593182a6283013a31b1288e04edcbc3180/webapp/app/plugins/controller.js#L65

Whereas the structured log viewer, appears to take the value of 'builder' from artifact.header.
(In reply to Jonathan French (:jfrench) from comment #1)
> Just curious, is this a bug which prevents adoption of Treeherder? Afaik
> preventing adoption is the criteria for being marked as a blocker.. but
> that's just my understanding and it could be wrong :)

The problem is that the log URLs are used pretty much any time someone is referring to a failure in a bug or on IRC etc. Then people open the log, try to grok the issue, and inevitably need to quickly understand what job it correlates to - both in terms of their memory of job types, and what they see in the main results UI.

The "builder" string isn't something with which people are familiar, since it's not surfaced anywhere in the TBPL UI - only the buildername is (what treeherder calls the "buildbot job name"). As such, (and unfortunately like a lot of the other treeherder bugs filed at the moment) this seemingly nit-picky bug is one of the causes of workflow/UX death by 1000 cuts that makes treeherder a lot slower and more frustrating to use compared to TBPL at present :-)
(In reply to Ed Morley [:edmorley] from comment #2)
> and inevitably need to quickly understand what job it
> correlates to - both in terms of their memory of job types, and what they
> see in the main results UI.

Perhaps with this latter part (main results UI), this would be helped by showing the platform name and symbol in the log viewer, so they can immediately visually recognise it compared to the main results UI?

eg:

"Android 4.0 M(1)"

Something that would also be great (but a feature request since TBPL doesn't do it, and so could be saved until later) is to have the existing link from the log viewer to the main ui (the SHA link) actually make that job be highlighted, so you could easily find your way back to it.
(In reply to Ed Morley [:edmorley] from comment #3)
> Something that would also be great (but a feature request since TBPL doesn't
> do it, and so could be saved until later) is to have the existing link from
> the log viewer to the main ui (the SHA link) actually make that job be
> highlighted, so you could easily find your way back to it.

Filed bug 1057707 for this.
(In reply to Ed Morley [:edmorley] from comment #2)
> 
> The "builder" string isn't something with which people are familiar, since
> it's not surfaced anywhere in the TBPL UI - only the buildername is (what
> treeherder calls the "buildbot job name"). As such, (and unfortunately like
> a lot of the other treeherder bugs filed at the moment) this seemingly
> nit-picky bug is one of the causes of workflow/UX death by 1000 cuts that
> makes treeherder a lot slower and more frustrating to use compared to TBPL
> at present :-)

I wonder if there is merit in having a [Meta] bug live in the blockers list for all UX/DBTC's, having them marked as blocking that Meta bug, and having any DBTC's which by themselves are not blockers live in the non-blockers list. ie. it sounds like their cumulative effect which is a blocker, but individuals may not be.

At some point down the road, there may be few enough open bugs against that Meta UX/DBTC bug it could also be dropped off the blockers list and while still open, by no longer being a blocker indicating that Treeherder is ready to be adopted by users from a UX standpoint.

I totally get the issue (I have been there so many times myself with other applications as an end user), but my impression is despite our best efforts our blockers fix rate, vs. incoming blockers is negative. Ie. the blockers list continues to grow rather than shrink and we are not going to catch up. There is also a triage cost, since weekly triage of blockers is a focus at the weekly mtgs.

Maybe there simply aren't any blocker UX/DBTC's which qualify at present, or could be moved to non-blocker status, but just an idea.
(In reply to Jonathan French (:jfrench) from comment #5)
> I wonder if there is merit in having a [Meta] bug live in the blockers list
> for all UX/DBTC's, having them marked as blocking that Meta bug, and having
> any DBTC's which by themselves are not blockers live in the non-blockers
> list. ie. it sounds like their cumulative effect which is a blocker, but
> individuals may not be.
> 
> At some point down the road, there may be few enough open bugs against that
> Meta UX/DBTC bug it could also be dropped off the blockers list and while
> still open, by no longer being a blocker indicating that Treeherder is ready
> to be adopted by users from a UX standpoint.

Yeah agree - that was the plan I had with bug 1053985 - might be worth having some more like that for other areas too.
Priority: P3 → P2
Whiteboard: [good first bug]
Mentor: mdoglio
Blocks: 1060482
In the example, it looks like we're getting the buildbot job name from this endpoint:

https://treeherder.mozilla.org/api/project/mozilla-central/artifact/?job_id=282513&name=buildapi&type=json

Whereas we get all the information in the logviewer from this endpoint:

https://treeherder.mozilla.org/api/project/mozilla-central/artifact/?job_id=297678&name=Structured+Log

I am not sure whether the best way forward is to change the frontend to get information from both endpoints, or to change the backend so (e.g.) the structured log endpoint returns build information in the desired format. Jeads?
Flags: needinfo?(jeads)
This problem is caused by the buildername available in the raw logs having the format "builder: mozilla-central_panda_android_test-remote-tspaint" and not the more verbose/human readable buildbot job name "Android 4.0 Panda mozilla-central talos remote-tspaint". The readable version is available from the buildapi builds4h.js data but is not found in the raw logs. In treeherder the structured log artifacts are built asynchronously, the parser has direct access to the log not the buildapi at parse time.

There are a couple ways we could solve this problem but I think the best way, for now, would be to add the buildbot job name to the structured log artifact before the structured log is returned from the web service endpoint. This would be done entirely in treeherder-service. This way we don't have to do another asynchronous call and if there are ever downstream consumers of the log artifacts they will have immediate access to both types of builder names. Note, another way of doing this would be to add the buildbot job name as a parameter in the url, we have access to this when the href to the structured log viewer is generated/selected but this strategy would leave downstream consumers out in the cold.

Another thing to keep in mind here is non-buildbot reporting systems may not have an equivalent of a buildbot builder name or job name so we need to be careful not to depend on it.

Let's chat directly regarding how to implement this.
Flags: needinfo?(jeads)
Assignee: nobody → jeads
Status: NEW → ASSIGNED
(In reply to Jonathan Eads ( :jeads ) from comment #8)
> This problem is caused by the buildername available in the raw logs having
> the format "builder: mozilla-central_panda_android_test-remote-tspaint" and
> not the more verbose/human readable buildbot job name "Android 4.0 Panda
> mozilla-central talos remote-tspaint". The readable version is available
> from the buildapi builds4h.js data but is not found in the raw logs. In
> treeherder the structured log artifacts are built asynchronously, the parser
> has direct access to the log not the buildapi at parse time.

I think what this bug and bug 1057341 come down to is that the log header isn't actually what we want most of the time (it isn't parsed at all by TBPL), and in fact it would be better to ditch the header from the artefact the log viewer requests:

[{
	"blob": {
		"header": {
			"slave": "b-2008-sm-0037",
			"buildid": "20140724063607",
			"builder": "mozilla-central-win64-debug",
			"results": "warnings (1)",
			"starttime": "1406208970.18",
			"builduid": "295f9e2fcc8a47c9bd4d1c9ac8693c3b",
			"revision": "616e6924cb0b"
		},

...and instead return values found from builds-4hr when we ingest the job.

I understand that job ingestion is asynchronous, but aiui the log viewer only works when the log has already been parsed (since we need the steps and all_errors), and we only even know about the log, when we've seen the job in builds-4hr in the first place. Or failing that, we could trigger a priority ingestion for that job / use placeholders in the log viewer for data not already populated.
We can do that, just need to know exactly what you want displayed. I will mung the builds4h-ish values into the log viewer json artifact before it's returned from the web service.

Based on what's displayed at the top of the brief/full log viewer in tbpl you will need the following:

---------------------------
Ubuntu VM 12.04 mozilla-central debug test mochitest-browser-chrome-1 on 2014-09-04 07:50:57 PDT for push da04cf3f8a06

slave: tst-linux32-spot-340
---------------------------

What else would you like there? and do you you still want it formatted as field name/value pairs or more tbpl-ish?
Flags: needinfo?(emorley)
Will put a list together after my flight (leaving tomorrow morning) :-)
(Added to my todoist)
Flags: needinfo?(emorley)
Perhaps something like...

Job name: B2G ICS Emulator debug Mochitest 6
Request Time: 2014-09-08 5:21 AM
Start Time: 2014-09-08 5:21 AM
End Time: 2014-09-08 6:33 AM
Duration: 71 minutes
Result: success
Machine name: tst-linux64-spot-1131

Where "job name" is roughly:
  build_platform + platform_option + job_group_name + job_type_symbol
...but where 'build_platform' has to be converted first (from eg "b2g-emu-ics") using the mapping in treeherder-ui

And job name links to the main treeherder UI, with that job filtered

Optionally, if it's easy, machine name linking to slave health, similar to the job details panel - but we can save that for another bug if it's a pain.
Oops, missed "revision".

Revision: SHA (linking to the result set on the treeherder main view).

I'm also pretty flexible on the labels for each property, if having set names is a pain for making this generic for non-buildbot sources.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
No longer blocks: 1060482
You need to log in before you can comment on or make changes to this bug.