Closed Bug 1049723 Opened 10 years ago Closed 10 years ago

Add log reference to Treeherder report

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Product:
Firefox OS Graveyard
Component:
Gaia::UI Tests
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: davehunt, Assigned: davehunt)

References

Details

Attachments

(2 files, 1 obsolete file)

Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/23586/files
52 bytes, text/plain
rwood
: review+
jlal
: review+
Details
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/24480
46 bytes, text/x-github-pull-request
mdas
: review+
mdoglio
: feedback+
Details | Review 
Once bug 1043718 is complete we should follow up with attaching the log reference to the Treeherder report. This will provide details of failures, making the report much more useful.
As I understand it Treeherder accepts a URL referencing the log, which means we'll need to upload the file to a publicly accessible location (perhaps using blobber). We could either do this from within the test harness and include the log reference when we submit results to Treeherder, or we could upload the file as a subsequent process, and update the Treeherder results to include the reference.

I think I prefer the idea of a separate step for the upload, as it would allow us to use the blobber uploader command line tool. It also means we can upload any arbitrary files from disk (we also want to upload the HTML and XML reports) rather than hooking something like blobber into the test harness. I'm not sure if the Treeherder client currently supports updating an existing job though, and we'd somehow need to keep a reference to the job in Treeherder.
At the moment the service api doesn't support adding a log reference to a job previously stored. That would require some development on treeherder's side.
Depends on: 1055537
(In reply to Mauro Doglio [:mdoglio] from comment #2)
> At the moment the service api doesn't support adding a log reference to a
> job previously stored. That would require some development on treeherder's
> side.

Thanks Mauro, so it sounds like we'll need to upload before we submit to Treeherder.
I've tested this locally and I can get Treeherder to be in a 'pending' state for the log parsing. I suspect I need to change something in order for it to actually process it.

The attached pull includes a couple of improvements to the command line options that could ride-along, though if you'd prefer me to split this into a separate pull request I can do that.
Assignee: nobody → dave.hunt
Status: NEW → ASSIGNED
Attachment #8482335 - Flags: review?(rwood)
Attachment #8482335 - Flags: feedback?(mdoglio)
Comment on attachment 8482335 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/23586

Clearing review/feedback request due to an issue with log parsing.
Attachment #8482335 - Flags: review?(rwood)
Attachment #8482335 - Flags: feedback?(mdoglio)
I tried running my patch against the staging Treeherder instance and have seen the same log parsing issue. It would appear that a 'clean' TBPL log is not suitable for parsing. It must be that mozharness or something else produces a different format for parsing.

See https://treeherder.allizom.org/ui/#/jobs?repo=b2g-inbound&revision=e5d7e65f7087 and the last 'Gip' to see the pending log parsing.

Mauro: Any thoughts on what we could do here? I suspect either adjusting the log parsing to work with a clean TBPL log, or perhaps we could use the raw log?
Flags: needinfo?(mdoglio)
Requesting review again now that I've fixed a couple of issues with uploading logs. At this time the log parsing is not working (I need some assistance from someone more familiar with Treeherder) but this patch will at least upload the log files.
Attachment #8482335 - Attachment is obsolete: true
Attachment #8484918 - Flags: review?(rwood)
James: Any thoughts on comment 6 or should I wait for Mauro? Also, I noticed that my current patch is not allowing the files in S3 to be served with an appropriate content type. It works for me locally but not from Jenkins, which uses a dedicated gaiatest user. See https://gaiatest.s3.amazonaws.com/912ce3a96ba24ad45927bdfaf2dddddbd063ed705e6c0c229ec5a24905fcb328647f57ff01a8356b6566001eb606cf5539c2a28db39a7db442a403517cdc3f94?Content-Type=text/plain as an example of the log being served as Content-Type 'application/octet-stream'
Flags: needinfo?(jlal)
Nevermind about the content-type, I have been able to resolve this by setting the appropriate metadata on the key if the file extension is .log. I've updated my patch.

Rob: If you take a look at https://treeherder.allizom.org/ui/#/jobs?repo=b2g-inbound&revision=11bc0a1ae139 and click the 5th 'Gip' you will see the logs attached in the bottom left frame.
Comment on attachment 8484918 [details]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/23586/files

LGTM and thanks for the TreeHerder link, the logs look great. Only question I have, is if an S3UploadError is raised, will that be reflected on treeherder somehow or in the jenkins build (i.e. if for some reason uploading the logs breaks, will it be made obvious or flagged somewhere other than the link to the logs won't be there)?
Attachment #8484918 - Flags: review?(rwood) → review+
Flags: needinfo?(dave.hunt)
Comment on attachment 8484918 [details]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/23586/files

(In reply to Robert Wood [:rwood] from comment #10)
> Comment on attachment 8484918 [details]
> Link to Github pull-request:
> https://github.com/mozilla-b2g/gaia/pull/23586/files
> 
> LGTM and thanks for the TreeHerder link, the logs look great. Only question
> I have, is if an S3UploadError is raised, will that be reflected on
> treeherder somehow or in the jenkins build (i.e. if for some reason
> uploading the logs breaks, will it be made obvious or flagged somewhere
> other than the link to the logs won't be there)?

That's a great suggestion Rob, I've addressed it in the latest patch. We'll now see the following in Treeherder if all artifacts failed to upload:

Error: Failed to upload raw.log
Error: Failed to upload tbpl.log
Error: Failed to upload results.html
Error: Failed to upload results.xml

James: Would you mind doing an additional review here?
Attachment #8484918 - Flags: review?(jlal)
Flags: needinfo?(dave.hunt)
Attachment #8484918 - Flags: review?(jlal) → review+
Flags: needinfo?(jlal)
Okay, so now only the log parsing question remains. Hoping Mauro can help out with that one when he returns from PTO.
Depends on: 1063922
Flags: needinfo?(mdoglio)
:davehunt I just filed bug 1063922 and bug 1063923 for it. If you want to circumvent this problem you gotta gzip the log file
Depends on: 1064798
Bug 1064798 is about the logcat, whereas this bug is about the TBPL log.
No longer depends on: 1064798
Depends on: 1064805
Mauro: Is this the only thing blocking log parsing? I've implemented gzip compression in bug 1064805 and attempted to include a log reference again, but the parsing still just says 'pending'...
Flags: needinfo?(mdoglio)
That was the first thing that I noticed when I digged into this. I'll check again to see what's the problem now.
Flags: needinfo?(mdoglio)
Any updates on this Mauro?
Flags: needinfo?(mdoglio)
bug 1063923 was fixed recently, it should be already deployed on stage. bug 1063922 is still there, but I guess is less urgent?.
Flags: needinfo?(mdoglio)
(In reply to Mauro Doglio [:mdoglio] from comment #20)
> bug 1063923 was fixed recently, it should be already deployed on stage. bug
> 1063922 is still there, but I guess is less urgent?.

We've already switched to compressing the log file (see bug 1064805) and it still didn't work. If you think it should be working now I can prepare a patch that includes the log reference.
Here's the patch that adds the log reference. It's still showing the same issue - I pushed a report to staging for you to see. Click the last Gip under "B2G Device Image opt" here: https://treeherder.allizom.org/ui/#/jobs?repo=b2g-inbound&revision=815521e3bc16
Attachment #8496038 - Flags: feedback?(mdoglio)
At the moment we need a bit of coordination to test this on stage because the log parser machine is not keeping the pace (see bug 1074213). :davehunt if you submit a new job I can trigger manually the log parser on it, skipping the queue. I just need to know the job guid to do that.
Attachment #8496038 - Flags: feedback?(mdoglio) → feedback+
(In reply to Mauro Doglio [:mdoglio] from comment #23)
> At the moment we need a bit of coordination to test this on stage because
> the log parser machine is not keeping the pace (see bug 1074213). :davehunt
> if you submit a new job I can trigger manually the log parser on it,
> skipping the queue. I just need to know the job guid to do that.

The first Gip in https://treeherder.allizom.org/ui/#/jobs?repo=b2g-inbound&revision=3097d179ca7c has a log reference. The job guid is 8861999b-41e2-41a0-acc2-12060ff74ea7.
Flags: needinfo?(mdoglio)
It worked! you can see the status has changed to parsed and a summary is available for that job here https://treeherder.allizom.org/ui/logviewer.html#?job_id=573506&repo=b2g-inbound.
Flags: needinfo?(mdoglio)
Comment on attachment 8496038 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/24480

Great, although it looks a little empty. I wonder if we might need to log some extra stuff for Treeherder, or perhaps tweak Treeherder's parsers for these logs. Let's take care of that in a separate bug though.
Attachment #8496038 - Flags: review?(mdas)
Landed in:
https://github.com/mozilla-b2g/gaia/commit/c7fc067d78e33543a3f82cd8fb91349d4c444c84
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: