1127676 - Handle empty pushes in pushlog web API

Status: RESOLVED FIXED

(Developer Services :: Mercurial: Pushlog, defect, P1)

Description

[Gregory Szorc [:gps]]

This seems to be currently biting production.

Comment 1

[Gregory Szorc [:gps]]

Attachment: MozReview Request: bz://1127676/gps

/r/3157 - pushlog: handle empty pushes from web API (bug 1127676)

Pull down this commit:

hg pull review -r 8bc919f1fc458c8d068ae7a73e0bf6d86a3dae07

Comment 2

[Monty the Python Review Bot]

https://reviewboard.mozilla.org/r/3157/#review2529

And now for something completely different.

Congratulations, there we no Python static analysis issues with this patch!

The following files were examined:

  hgext/pushlog-legacy/pushlog-feed.py

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8556840 [details]
MozReview Request: bz://1127676/gps

/r/3157 - pushlog: handle empty pushes from web API (bug 1127676)

Pull down this commit:

hg pull review -r 7f2af0b3458df95b6614af16164cd2c69cc572fb

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8556840 [details]
MozReview Request: bz://1127676/gps

/r/3157 - pushlog: handle empty pushes from web API (bug 1127676)

Pull down this commit:

hg pull review -r 2af84325df2a22da24d816ee491785bd57a6ec5c

Comment 5

[Monty the Python Review Bot]

https://reviewboard.mozilla.org/r/3157/#review2533

And now for something completely different.

Congratulations, there we no Python static analysis issues with this patch!

The following files were examined:

  hgext/pushlog-legacy/pushlog-feed.py

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8556840 [details]
MozReview Request: bz://1127676/gps

/r/3157 - pushlog: handle empty pushes from web API (bug 1127676)

Pull down this commit:

hg pull review -r 2af84325df2a22da24d816ee491785bd57a6ec5c

Comment 7

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/3155/#review2535

edmorley: as a Treeherder, I think you get to make the call as to how you want it. Version 2 of the diff displays empty arrays of changesets. Version 3 filters empty pushes from output.

Comment 8

[Ed Morley [:emorley]]

(In reply to Gregory Szorc [:gps] from comment #7)
> edmorley: as a Treeherder, I think you get to make the call as to how you
> want it. Version 2 of the diff displays empty arrays of changesets. Version
> 3 filters empty pushes from output.

I'm slightly confused - the output in the tests in the version up for review seems to omit the push entirely, not just display an empty array of changesets. And there's no additional version 3 added?

Comment 9

[Ed Morley [:emorley]]

Treeherder is currently using version 1 of the API, bug 1076826 will make us switch to the new API and using the lastpushid.

As such, at the moment we just do &fromchange=rev, and if the response is a 404, we ditch the fromchange param, and use bare json-pushes, which gets us the last 10 pushes (and then the last rev is cached from that response).

The issue last night was that instead of a 404 (or a valid response containing some pushes, but no changesets for the deleted one), we got a 500, so just kept on trying with the original:
https://hg.mozilla.org/releases/mozilla-aurora/json-pushes?full=1&fromchange=d4d912d789021abef9a94af7929a10c9f0ecc539 

Treeherder code:
https://github.com/mozilla/treeherder-service/blob/45824d8507d3cdc154ff452b1abefa3b6f3d75d7/treeherder/etl/pushlog.py#L96

Comment 10

[Ed Morley [:emorley]]

(In reply to Ed Morley [:edmorley] from comment #8)
> I'm slightly confused - the output in the tests in the version up for review
> seems to omit the push entirely, not just display an empty array of
> changesets. And there's no additional version 3 added?

Ah, I read "Version 2 of the diff" as "Version 2 of the API" lol. All clear now.

Comment 11

[Ed Morley [:emorley]]

Comment on attachment 8556840 [details]
MozReview Request: bz://1127676/gps

https://reviewboard.mozilla.org/r/3155/#review2565

Ship It!

Comment 12

[Gregory Szorc [:gps]]

https://hg.mozilla.org/hgcustom/version-control-tools/rev/080fee669cb5

Comment 13

[Gregory Szorc [:gps]]

This is deployed. Spot checking mozilla-aurora reveals that empty pushes are now properly filtered from the output. e.g. https://hg.mozilla.org/releases/mozilla-aurora/json-pushes?startID=7528

Comment 14

[Ed Morley [:emorley]]

And the fromchange example in comment 9 no longer fails.
Thank you for sorting this :-)

Comment 16

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: handle empty pushes from web API (bug 1127676)