1233164 - Autoclassification exceptions "IntegrityError(1062, "Duplicate entry 'SNIP' for key 'failure_line_job_guid_7d547076_uniq'")"

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

Broken out from bug 1205049:

(In reply to Mauro Doglio [:mdoglio] from comment #7)
> It looks like in some cases we have multiple `*_error_summary` logs for the
> same job. Because of this sometimes we have a collision between failure
> lines belonging to the same job on different logs with the same line number.

(In reply to Mauro Doglio [:mdoglio] from comment #8)
> Here is an example of such a job:
> https://treeherder.allizom.org/api/project/try/jobs/13759185/
> And here you can see we have 2 error_summary logs (a11y_errorsummary.log and
> chrome_errorsummary.log) for that job:
> https://treeherder.allizom.org/api/project/try/artifact/61479122/

https://rpm.newrelic.com/accounts/677903/applications/5585473/filterable_errors#/show/556198-9a895ca7-6c4d-11e5-97ea-b82a72d22a14/stack_trace?top_facet=transactionUiName&bottom_facet=host&primary_facet=error.class&_k=1hbp1x

James/Cameron, could you take a look at this?

Comment 1

[James Graham [:jgraham]]

This is a bit annoying. Obviously we can add more data to indicate which log line comes from which summary file, but we would also like to know which order they are generated in for the purposes of correlating with unstructured lines. At present I don't think we have an easy way of doing that, so it might be necessary to add something on the mozharness side, or rethink the way that we do the correlation (e.g. returning to the plan of delimiting the bits of the logs that should produce structured output and ignoring that in the UI).

Comment 2

[Cameron Dawson [:camd]]

I believe this is going to be a large part of what jgraham works on in Q1-2016.  Please n-i me again if you need anything from me.  :)

Comment 3

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:failureline_log_id > mozilla:master

Comment 4

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:failure_line_remove_unique > mozilla:master

Comment 5

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/0692d11a6e409170f9970806deab691c2ecc31b8
Bug 1233164 - Remove incorrect unique constraint on failure_line

This is causing errors. In the future it will be replaced by a
more correct constraint that makes failure lines unique per log

https://github.com/mozilla/treeherder/commit/2858571ae2bb36daa6725445a9bbe9a5295a300c
Merge pull request #1511 from mozilla/failure_line_remove_unique

Bug 1233164 - Remove incorrect unique constraint on failure_line

Comment 6

[James Graham [:jgraham]]

Comment on attachment 8751845 [details] [review]
[treeherder] mozilla:failureline_log_id > mozilla:master

Significant update now wlach's patches landed.

Comment 7

[William Lachance (:wlach)]

Comment on attachment 8751845 [details] [review]
[treeherder] mozilla:failureline_log_id > mozilla:master

This all looks good to me! Great to see the new job log model getting used.

Comment 8

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/f4a5ccd403ae5937288095c1569cd3785e11c9d9
Bug 1233164 - Store job log id in failure_line.

Associate each failure_line with the id of the log that it came
from. This allows a correct unique constraint on failure_line of
(job_log_id, line).

https://github.com/mozilla/treeherder/commit/480dddd217a6ced5b7336d203720f07cfa98f3ec
Merge pull request #1489 from mozilla/failureline_log_id

Bug 1233164 - Store log id in failure_line.

Comment 9

[Ed Morley [:emorley]]

I'm still seeing exceptions on stage, eg:
https://rpm.newrelic.com/accounts/677903/applications/5585473/traced_errors/556193-0e9afc24-28a8-11e6-b947-b82a72d22a14

Given bug 1277436 comment 5, I think it's even more important that we try to stamp out spurious exceptions on New Relic, so the alert thresholds can be tighter, and anything unusual stands out more.

I don't suppose you could take another look?

Comment 10

[James Graham [:jgraham]]

That seems to be an unrelated constraint. It should be fixed of course, but isn't part of this bug.

Comment 11

[Ed Morley [:emorley]]

Ah sorry I misread it. Since it's still autoclassify related, would you mind filing the separate bug? :-)

Comment 12

[Ed Morley [:emorley]]

Ah raced you doing so (bug 1277575). Thank you :-)