1251988 - Add UI features to Treeherder to enable comparison of structured logs in WPT Viewer

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect)

Description

[Kalpesh Krishna [:martianwars]]

The Web Platform Test Viewer, which was originally a part of my Quarter of Contribution, finally covers most of the useful cases and filters. We have currently hosted it on jgraham.github.io/wptview
Since this project works best when used to compare structured logs, it makes sense adding some form of integration with Treeherder's active runs.

The result viewer accepts query strings and can ingest structured logs in the following format :-
jgraham.github.io/wptview/#/?urls=<url1>,<run_name>;<url2>,<run_name;...
An example of the same,

http://jgraham.github.io/wptview/#/?urls=http:%2F%2Fmozilla-releng-blobs.s3.amazonaws.com%2Fblobs%2Fmozilla-inbound%2Fsha512%2F512fb0af73ffa1dd7417d93884b56fb05b0ed4bbb327eaa387517fce12d0ae26da44d9207ae84ea6ca56328b1a8aa431d24718ed57b5c43fe9b593d30db17294,W-1

Cameron, would it be possible to integrate this? I wouldn't mind working on this.

Comment 1

[William Lachance (:wlach)]

Is there come kind of signifier that a test job contains wptview structured logs? If so, it should be relatively straightforward to link to the jgraham.github.io url from the treeherder job's panel, see the `ui/plugins/` directory for some existing examples of this.

Comment 2

[Kalpesh Krishna [:martianwars]]

William,
The interface accepts files of the format <something>_raw.log, which would be structured log files I suppose. It isn't specific to Web Platform Tests, though the name suggests otherwise. Also, the aim would be to **compare** files having the same set of tests rather than just add an external link on each job's panel. 

I'm not sure about the existing mechanisms when a job is selected, but maybe if there was a way to select one or more jobs having structured logs with the same set of tests, and a single button which opens then up in wptview it would be great! What does pinboard do?

Comment 3

[James Graham [:jgraham]]

Structured logs are signified by an artifact with the name [something]_raw.log. In the first instance I suggest we just add a link for viewing individual instances of results from these files. Since the viewer loads the file into IndexedDB it should be possible to make comparisons just by clicking multiple links and then enabling all the different logs you want to view. We can make a more complicated interface later.

Comment 4

[Kalpesh Krishna [:martianwars]]

Attachment: Adding wptview support

Lots of work left, but a start nevertheless :)

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8724768 [details] [review]
Adding wptview support

lgtm!

Comment 6

[Cameron Dawson [:camd]]

I actually get a 404 when I try that link.  Maybe I'm missing something, or it changed?  Is there an updated one?  

I agree with wlach about the angular directive.  And it looks like it adds the "open" link for anything in Job Info, but I presume it'll only work for structured logs.  Will need some kind of check so that it only adds for "_raw..." etc...

Comment 7

[Kalpesh Krishna [:martianwars]]

Comment on attachment 8724768 [details] [review]
Adding wptview support

@camd, the URL has changed since jgraham moved the project to Mozilla. I've updated the PR.
I've added an appropriate filter. I hope it's okay now.
One issue that I see is that the run name is very long. Could I get a smaller name somewhere in the data? Was thinking of doing a controller in the next PR

Comment 8

[William Lachance (:wlach)]

Comment on attachment 8724768 [details] [review]
Adding wptview support

Ok, I know we're just going for an minimal viable product here, but after trying this, I think we should try to have a better user interface here than just having a link called "Open!" next to the log interface. That's completely unclear.

I think the *very least* the link should be on a separate line saying something like "View test pass/fail summary in wptview". That at least is somewhat understandable.

Comment 9

[William Lachance (:wlach)]

By the way, it's not necessary to both ask for a review request and needinfo someone. The review request implies the needinfo. :)

Comment 10

[William Lachance (:wlach)]

Comment on attachment 8724768 [details] [review]
Adding wptview support

Looks good now!

Comment 11

[William Lachance (:wlach)]

Merged https://github.com/mozilla/treeherder/commit/13639885f4cacbbcc398b5af32284845310d39b6

Comment 12

[Kalpesh Krishna [:martianwars]]

So I see a small problem here. I've taken the last string as the run name while importing the log into wptview. I'll change this in another PR soon :)

Comment 13

[GitHub Autolander Bot]

Attachment: [treeherder] martiansideofthemoon:raichu > mozilla:master

Comment 14

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/3cb288dd6909db5c990e41d6c0277f5563761bcd
Bug 1251988 - Adding conditions for cases where Buildername isn't last string

https://github.com/mozilla/treeherder/commit/4a6a0768f8169ad70a9a854ae02e3ca4a0be8349
Merge pull request #1335 from martiansideofthemoon/raichu

Bug 1251988 - Adding conditions for cases where Buildername isn't las…

Comment 15

[GitHub Autolander Bot]

Attachment: [treeherder] martiansideofthemoon:kakuna > mozilla:master

Comment 16

[William Lachance (:wlach)]

Comment on attachment 8728952 [details] [review]
[treeherder] martiansideofthemoon:kakuna > mozilla:master

One more minor naming nit, but this looks fine otherwise

Comment 17

[William Lachance (:wlach)]

Comment on attachment 8728952 [details] [review]
[treeherder] martiansideofthemoon:kakuna > mozilla:master

This doesn't work as we'd like to at the moment, it needs to behave properly when a user opens wptview again from another job. So r-'ing for now :)

Comment 18

[Kalpesh Krishna [:martianwars]]

Attachment: fixing <a> text

Comment 19

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/df69f97eebb4ee4caf56c7c71600d16ddb950406
Bug 1251988 - Update name of <a> element for wptview

https://github.com/mozilla/treeherder/commit/1b6ca587f120fd3b3102ab3633b1ad313a0d4aee
Merge pull request #1365 from martiansideofthemoon/ditto

Bug 1251988 - Update name of <a> element for wptview

Comment 20

[Ed Morley [:emorley]]

Ping for questions in one of the PRs that is still open:
https://github.com/mozilla/treeherder/pull/1345#issuecomment-230437394