UI displays duplicate pushes with the same revision due to jobs having a different revision_hash

RESOLVED FIXED

Status

Tree Management
Treeherder: Data Ingestion
P1
normal
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: armenzg, Unassigned)

Tracking

Details

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
This is very odd.

If you load treeherder for cedar:
https://treeherder.mozilla.org/ui/#/jobs?repo=cedar

You will see revision 2dea8b3c6c91 twice:
http://people.mozilla.org/~armenzg/sattap/b6ca9d97.png

Comment 1

3 years ago
That's.. odd :-(
Component: Treeherder → Treeherder: Data Ingestion
OS: Linux → All
Priority: -- → P1
Hardware: x86_64 → All

Comment 2

3 years ago
URL to reproduce:
https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=2dea8b3c6c91

The API request:
https://treeherder.mozilla.org/api/project/cedar/resultset/?format=json&revision=2dea8b3c6c91

And response:
{
	"meta": {
		"count": 2,
		"filter_params": {
			"format": "json",
			"revision": "2dea8b3c6c91"
		},
		"repository": "cedar"
	},
	"results": [{
		"repository_id": 23,
		"revision_hash": "9c9efe6afba67d2823db093e523ff79a03f634e5",
		"revision_count": 200,
		"author": "jgriffin@mozilla.com",
		"comments": "Merge m-c -> cedar",
		"revisions_uri": "/api/project/cedar/resultset/162/revisions/",
		"push_timestamp": 1421971493,
		"revisions": [{
...
		}],
		"id": 162,
		"revision": "2dea8b3c6c91"
	},
	{
		"repository_id": 23,
		"revision_hash": "19164a1f9ef3fa83d7deac842f116bf944106e5a",
		"revision_count": 329,
		"author": "jgriffin@mozilla.com",
		"comments": "Merge m-c -> cedar",
		"revisions_uri": "/api/project/cedar/resultset/160/revisions/",
		"push_timestamp": 1421971493,
		"revisions": [{
...
		}],
		"id": 160,
		"revision": "2dea8b3c6c91"
	}]
}

...same revision but different revision_hash.

Seeing as this hasn't happened more often, dropping priority, but we should definitely figure this out.
Priority: P1 → P3

Comment 4

3 years ago
Created attachment 8569365 [details]
buildapi artefacts for "good" job

Comment 5

3 years ago
Created attachment 8569371 [details]
buildapi artefacts for "bad" job

To add context: The revision hash is generated from a number of the job's properties, with the original intention that we could support the display of jobs who pull from multiple unpinned repos - and slice by repo of choice (ie the "how do we display jobs against both mozilla-central and comm-central pushlog" problem). 

We need to check:
(a) what difference in properties caused this difference revision_hash (eg do we use the mozharness revision as part of the hash? I forget)
(b) was this correct, or should we change the way we calculate the revision hash
(c) should the UI (/API) be combining the display of jobs that have the same revision (for the repo currently being viewed) but different overall revision hash.

The "two revisions are shown" problem really is (c) - we quite possibly just have not ever come across this case (seeing as we use pinned revisions everywhere now rather than true multi-repo revision combinations) and so not realised we implemented it incorrectly in the UI/API.

Updated

3 years ago
Summary: Treeherder is showing the same revision twice → Treeherder is showing the same revision twice due to jobs having a different revision_hash

Comment 6

3 years ago
Making summary more keyword rich
Summary: Treeherder is showing the same revision twice due to jobs having a different revision_hash → Treeherder is display duplicate pushes with the same revision due to jobs having a different revision_hash

Updated

3 years ago
Duplicate of this bug: 1144570

Updated

3 years ago
Summary: Treeherder is display duplicate pushes with the same revision due to jobs having a different revision_hash → UI displays duplicate pushes with the same revision due to jobs having a different revision_hash

Comment 8

3 years ago
Another example:
https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=bf7938244daf
Assignee: nobody → emorley
Priority: P3 → P2
(In reply to Mike Hommey [:glandium] from comment #9)
> Happened twice in three days.

With one push per day. So twice in three pushes.
I'll take a look at this in after the weekend :-)
Status: NEW → ASSIGNED
See Also: → bug 1193048

Updated

3 years ago
Priority: P2 → P1
I'm not going to have time to look at this until after deliverables; someone else feel free to take a look in the meantime :-)
Assignee: emorley → nobody
Status: ASSIGNED → NEW
Wondering if somewhat-recent improvements have resolved this...
Flags: needinfo?(emorley)
There's a chance they have - let's leave open for a bit longer and see if it reoccurs now that we're using 40 character SHAs and the underlying schema has changed slightly :-)
Flags: needinfo?(emorley)
Spot checking different repos, the last occurrence of this is from March 2016.  Which coincides about with this commit which should have fixed it:

https://github.com/mozilla/treeherder/commit/2a9dbefa49d26d8d0ddc7c402c571691b1679770

I think this is fixed.  :)
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED

Updated

2 years ago
Duplicate of this bug: 1174273
You need to log in before you can comment on or make changes to this bug.