1618667 - In jobs view show revisions since the parent

Status: NEW

(Tree Management :: Treeherder: Data Ingestion, enhancement, P3)

Description

[Armen [:armenzg]]

Attachment: [screenshot] showing 3 pushes

The screenshot shows three pushes that are based off each other. The fuzzy commits are throw away commits. I would like all revisions since branching off from mozilla-central to show up.

This will make it more clearly what the difference is since the parent revision and visit the parent revision for visual comparison if wished for.

Instead of this (topmost push):

• Fuzzy commit
• 5f0293ea3c18c0034705eb1783d5e5f18f38cf9a

It should include:

• Fuzzy commit
• 5f0293ea3c18c0034705eb1783d5e5f18f38cf9a
• f7d4a1110dda209244b257e1d3143ffdab447b8c
• 402e8cdb8d24bf2fddf7fdfe0402d730cbaffb19
• a117660624ec1117687f8922ee8f0b84361654f9 (This is the parent; add link to Treeherder push)

Comment 1

[shavindra]

Attachment: fuzzy matches.PNG

Is it a question of filtering to only return commits without Fuzzy query? i.e. ?if(!revisions.comment.includes('Fuzzy query'))

or need to modify/pass param to API call?

Can I pick this up?

Comment 2

[Cameron Dawson [:camd]]

I think we would have to call a different API for this, just like Push Health does.

The API Push Health uses to get commits back to the parent is:

'https://hg.mozilla.org/{}/json-automationrelevance/{}'.format(repository.name, revision)

If that API returns a 404, then we fallback to:

'{}/json-pushes?version=2&full=1&changeset={}'.format(repository.url, revision)

This would likely need to happen during ingestion in the /etl/push_loader.py file.

I think Armen would be a better person to mentor this bug than I am. He has more recent context into Push ingestion.

Comment 3

[Armen [:armenzg]]

The ingestion code is around here:
https://github.com/mozilla/treeherder/blob/master/treeherder/etl/push_loader.py#L267-L289

You will need to modify this mock:
https://github.com/mozilla/treeherder/blob/master/tests/etl/test_push_loader.py#L89-L100

You might also need to determine which fixtures/mocks need adjusting:
https://github.com/mozilla/treeherder/tree/master/tests/sample_data

You can run the tests related to Hg ingestion like this:
pytest tests/etl/test_push_loader.py -k test_ingest_hg_push

Comment 4

[shavindra]

Ahh this is a python change? I thought it would be react. I just started learning python so would be bit slow. I will try to do this within next couple of weeks.

Comment 5

[Armen [:armenzg]]

The UI shows what the backend stores. Perhaps we should move this Data Ingestion instead.

Comment 6

[Armen [:armenzg]]

@shavindra, we will leave the bug unassigned until you tell us you're comfortable to tackle it. Thanks though!