[gaia-taskcluster] Tests on taskcluster have not run since November 3rd

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: dylan, Assigned: garndt)

Tracking

Details

Attachments

(1 attachment)

(Reporter)

Description

2 years ago
Created attachment 8808605 [details]
Screen Shot 2016-11-08 at 08.48.38.png

For some reason, tests have not run.
(Reporter)

Comment 1

2 years ago
Running them locally, I get this far:

ok 449 - wait_for_page_to_load, 60000
ok 450 - get_title, 'Logged Out'
1..450
ok
Error requesting http://localhost:4444/selenium-server/driver/:
500 read timeout

handing this over to the expert.
Assignee: dylan → dkl
Nothing has changes wit regard to our taskgraph.json file so something has changed on the receiving end. Note that this repo/branch is still using the old method of kicking off builds from github -> taskcluster -> treeherder.

https://gaia-taskcluster.herokuapp.com/github

The webhook logs from github just show:

Request:
Request URL: https://gaia-taskcluster.herokuapp.com/github
Request method: POST
content-type: application/json
Expect: 
User-Agent: GitHub-Hookshot/d83765d
X-GitHub-Delivery: ce5fec00-a548-11e6-82ad-9c0a54abd834
X-GitHub-Event: push

Response (code 500):
Connection: keep-alive
Content-Length: 21
Content-Type: text/plain; charset=utf-8
Date: Tue, 08 Nov 2016 00:18:01 GMT
Server: Cowboy
Via: 1.1 vegur

Internal Server Error
Severity: normal → critical
Component: Testing Scripts and Infrastructure → Github
Product: bugzilla.mozilla.org → Taskcluster
Version: Production → unspecified
(Assignee)

Updated

2 years ago
Component: Github → Integration
Summary: Tests on taskcluster have not run since November 3rd → [gaia-taskcluster] Tests on taskcluster have not run since November 3rd
(Assignee)

Updated

2 years ago
Assignee: dkl → garndt
(Assignee)

Comment 3

2 years ago
Looking over the logs it appears that a 500 error was being returned by treeherder when attempting to create the resultset. There could be a result between gaia-taskclsuter trying to create the resultset and treeherder creating it, but camd is digging in.

I've remove all code for creating the resultset in hopes that's what it is.  :dkl, could you try again?
dkl: please give me a heads up when you are able to try again, so I can observe locally, too.  It's possible that it was ONLY failing with the API submission (as stated above) which caused the tests not to get kicked off.  

A local test with a dummy github repo worked swimmingly locally.  But I can ingest locally with a little extra debugging code while you try on your end and hopefully track down what's choking.  Or, perhaps it's not choking at all anymore.  Wouldn't that be nice?  :)
It looks like it finally worked Yesterday:  https://treeherder.mozilla.org/#/jobs?repo=bmo-master&revision=b18ca285bf12d1351c69d7033a24bb8f75c6f8f9

There must be something broken in our resultset API.  I can investigate a little later.
(Assignee)

Comment 6

2 years ago
Looks like the original issue might be resolved for bmo.  I think that this bug can be resolved, but I'll let dkl make that call.
Talked to Will who recently made some changes to this code.  He said he'd take a look.

Updated

2 years ago
Flags: needinfo?(wlachance)
Yeah looks like we were trying to insert a resultset without a revision here:

https://rpm.newrelic.com/accounts/677903/applications/14179757/traced_errors/032063bc-a5fd-11e6-9c7c-b82a72d22a14_5689_9924

If you've turned off the service, I think we should be good.
Flags: needinfo?(wlachance)
This is working again now. Thanks.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.