Verify that treeherder is correctly sending classification data to Elasticsearch via TBPL's starcomment.php

RESOLVED FIXED

Status

P1
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: emorley, Assigned: jgriffin)

Tracking

Details

(Reporter)

Description

4 years ago
Mauro, I don't suppose you could capture one of the payloads going to starcomment.php from the treeherder API, when someone classifies a failure - so I can check that it's being sent through correctly?

It's just I don't have any other way to ensure we're all set for OrangeFactor still working, when we switch over otherwise - since the ES db is somewhat of a black box.
Flags: needinfo?(mdoglio)
(Reporter)

Updated

4 years ago
Priority: -- → P1
I can log the request towards tbpl and attach it to this bug
Flags: needinfo?(mdoglio)
Assignee: nobody → mdoglio
Here is a sample taken from the log:
{'comment': 'Bug 1018943', 'buildtype': u'opt', 'os': u'linux64', 'tree': u'try', 'who': u'mdoglio', 'buildname': u'Ubuntu VM 12.04 x64 try opt test mochitest-4', 'date': '2014-08-04', 'logfile': '00000000', 'machinename': u'tst-linux64-spot-1033', 'rev': u'c8e70dd51a1a', 'starttime': 1407167477, 'timestamp': 1407340091, 'type': u'Mochitest'}
(Reporter)

Comment 3

4 years ago
Thank you - I'll compare against TBPL shortly.
Status: NEW → ASSIGNED
Flags: needinfo?(emorley)
(Reporter)

Comment 4

4 years ago
I'll take a look at this next week; switching assignee so this appears on http://harthur.github.io/bugzilla-todos/
Assignee: mdoglio → emorley
Flags: needinfo?(emorley)
(Reporter)

Updated

4 years ago
Priority: P1 → P2
(Reporter)

Updated

4 years ago
Priority: P2 → P1
(Reporter)

Comment 5

4 years ago
On second thoughts I think we need to actually access the ES DB to confirm this, and seed it with some treeherder-submitted entries, which is going to take some work. Unassigning for now, since I think we'll need to play around with this during the work week.
Assignee: emorley → nobody
Status: ASSIGNED → NEW
(Assignee)

Comment 6

4 years ago
> since the ES db is somewhat of a black box.

If you have ssh access to brasstacks, it's possible to set up ssh forwarding for ES through brasstacks to your local machine:

ssh -L 9200:elasticsearch-zlb.webapp.scl3.mozilla.com:9200 brasstacks1.dmz.scl3.mozilla.com

This will make the ES cluster available as localhost:9200 on your machine.

If you don't have ssh access to brasstacks, let's file a bug and get that.
(Assignee)

Comment 7

4 years ago
I went ahead and verified this is working correctly.  I starred a failure on try using Treeherder, and then verified in appeared in ES:

    "_source": {
        "buildtype": "opt",
        "os": "linux32",
        "tree": "try",
        "who": "jgriffin@mozilla.com",
        "buildname": "Ubuntu VM 12.04 try opt test mochitest-e10s-browser-chrome-2",
        "date": "2014-09-17",
        "logfile": "00000000",
        "machinename": "tst-linux32-spot-334",
        "rev": "e2d5f87ddcfc",
        "starttime": "1410994799",
        "timestamp": "1410996378",
        "type": "Mochitest e10s Browser Chrome",
        "bug": "1066474"
    }

I then compared this entry with one generated by TBPL for a failure in the same suite/platform on m-c:

  "_source": {
    "buildname": "Ubuntu VM 12.04 mozilla-central pgo test mochitest-e10s-browser-chrome-2",
    "machinename": "tst-linux32-spot-1053",
    "os": "linux32",
    "date": "2014-09-17",
    "type": "Mochitest e10s Browser Chrome",
    "buildtype": "pgo",
    "starttime": "1410986640",
    "logfile": "48311349",
    "tree": "mozilla-central",
    "rev": "d2c01d77b9d0",
    "who": "RyanVM",
    "timestamp": "1410988439",
    "bug": "1067953"
  }

The only difference is the 'logfile' field, which is an artifact of TBPL and is not used by OrangeFactor for anything.

I'm going to go ahead and close this; Ed, reopen if you think I've missed something.
Assignee: nobody → jgriffin
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Reporter)

Comment 8

4 years ago
That looks great - thank you for doing this :-)
You need to log in before you can comment on or make changes to this bug.