918632 - Create a task which can verify that pull request status is tested on top of latest tip of $BRANCH

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::GithubBot, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

We should have a node module to determine whether a revision passed CI tests.  We should abstract current CI.

Comment 1

[James Lal [:lightsofapollo]]

OK so my initial idea worked like this:

(steps to verify)

1. get commit status from github
2. verify its passed
3. get latest tip sha
4. get parent sha from travis test run and verify its the same as tip

The last part is the problem (4) as far as I can tell this information is not stored anywhere so we cannot determine which "tip" is the right one... I have a less optimal approach which is:

1. verify the pull request is "mergable" (github gives us this for free)
2. get last "updated_at" time on the PR
3. get commit status from github
4. verify its passed
5. verify the commit status was created (and passing) after the last updated_at
6. verify $BRANCH tip commit date is before "updated_at"

This approach is much less correct and is hinging on the fact that the "tip" date is correct. That said steps 1-4 give us quite a bit of protection so this might be good enough for a start.

Comment 2

[James Lal [:lightsofapollo]]

An alternative approach is to post the parent information from our travis jobs but this seems sub-optimal... Ideally our system is a "drop-in" kind of thing and just works out of the box... I will see if there is some way we can store this state in travis.

Comment 3

[Kevin Grandon :kgrandon]

Mass modify: Since we are going to be using Autolander now, mark old github bot bugs as resolved wontfix.