1276742 - Failing to post job to local Treeherder instance using HTTPS

Status: RESOLVED INVALID

(Tree Management :: Treeherder, defect)

Description

[Armen [:armenzg]]

I've been trying to push a job to a local instance of treeherder with this script [1]
Once I switched the client from 'https' to 'http' it stopped failling in this manner [2]

Any ideas on how to prevent this? or improve the message?

[1] https://github.com/armenzg/TC_developer_scheduling_experiments/blob/treeherder/treeherder_hello_world.py
[2] 
(TC_scheduling) armenzg@armenzg-thinkpad:~/repos/TC_developer_scheduling_experiments$ python treeherder_hello_world.py 
Traceback (most recent call last):
  File "treeherder_hello_world.py", line 85, in <module>
    client.post_collection('try', tjc)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/thclient/client.py", line 929, in post_collection
    collection_inst.get_collection_data(), timeout)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/thclient/client.py", line 709, in _post_json
    timeout=timeout, auth=self.auth)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/requests/api.py", line 111, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/requests/adapters.py", line 467, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='local.treeherder.mozilla.org', port=443): Max retries exceeded with url: /api/project/try/jobs/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f66efe20590>: Failed to establish a new connection: [Errno 111] Connection refused',))

Comment 1

[Armen [:armenzg]]

How can I know if the script I wrote works? I tried reloading the revision I'm posting to and I don't see any jobs.

When I reload the revision I see the following message [1]
The revision is _not_ 'da39a3ee5e6b4b0d3255bfef95601890afd80709' and the guid was '48c218c0-5ec7-4550-ad98-9b553d5b4469'

[1]
[2016-05-30 14:16:38,041] WARNING [django.request:170] Not Found: /js/config/local.conf.js
datasource.hubs.MySQL.MySQL debug message:
	host:localhost db:try host_type:read_host proc:jobs.selects.get_result_set_list
	Executing SQL:SELECT DISTINCT rs.id, rs.author, rs.long_revision, rs.revision_hash, rs.push_timestamp FROM result_set AS rs INNER JOIN revision_map ON rs.id = revision_map.result_set_id INNER JOIN revision ON revision_map.revision_id = revision.id WHERE rs.active_status = 'active' AND revision.long_revision = %s ORDER BY rs.push_timestamp DESC LIMIT 10
	Execution Time:3.8290e-04 sec

datasource.hubs.MySQL.MySQL debug message:
	host:localhost db:try host_type:read_host proc:jobs.selects.get_result_set_details
	Executing SQL:SELECT rm.result_set_id, r.repository_id, r.revision, r.author, r.comments FROM revision_map AS rm LEFT JOIN revision AS r ON rm.revision_id = r.id WHERE r.active_status = 'active' AND rm.result_set_id IN (%s) ORDER BY r.id DESC
	Execution Time:9.0003e-04 sec

datasource.hubs.MySQL.MySQL debug message:
	host:localhost db:try host_type:read_host proc:jobs.selects.get_job_list
	Executing SQL:SELECT j.id, j.`job_guid`, j.`signature`, j.`job_coalesced_to_guid`, j.`build_platform_id`, j.`option_collection_hash`, j.failure_classification_id, m.`name` as machine_name, mp.`platform` as platform, mp.`os_name` as machine_platform_os, mp.`architecture` as machine_platform_architecture, bp.`platform` as build_platform, bp.`os_name` as build_os, bp.`architecture` as build_architecture, j.`job_type_id` as job_type_id, jt.`name` as job_type_name, jt.`symbol` as job_type_symbol, jt.`description` as job_type_description, jg.`name` as job_group_name, jt.`job_group_id` as job_group_id, jg.`symbol` as job_group_symbol, jg.`description` as job_group_description, j.`who`, j.`result_set_id`, j.`result`, j.`state`, j.`reason`, j.`start_timestamp`, j.`end_timestamp`, j.`submit_timestamp`, j.`running_eta`, j.`last_modified`, j.`tier`, rds.`name` as ref_data_name, rds.`build_system_type` as build_system_type FROM `job` as j LEFT JOIN `treeherder`.`machine` as m ON j.`machine_id` = m.id LEFT JOIN `treeherder`.`machine_platform` as mp ON j.`machine_platform_id` = mp.id LEFT JOIN `treeherder`.`build_platform` as bp ON j.`build_platform_id` = bp.id LEFT JOIN `treeherder`.`job_type` as jt ON j.`job_type_id` = jt.id LEFT JOIN `treeherder`.`job_group` as jg ON jt.`job_group_id` = jg.id LEFT JOIN `treeherder`.reference_data_signatures rds ON j.signature = rds.signature WHERE 1 AND j.result_set_id = %s GROUP BY j.id LIMIT 2000 OFFSET 0
	Execution Time:7.1170e-03 sec

[2016-05-30 14:16:39,449] ERROR [treeherder.webapp.api.exceptions:24] u'da39a3ee5e6b4b0d3255bfef95601890afd80709'
[2016-05-30 14:16:39,451] ERROR [treeherder.webapp.api.exceptions:25] Traceback (most recent call last):
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/vagrant/treeherder/treeherder/webapp/api/utils.py", line 105, in use_jobs_model
    return model_func(*args, jm=jm, **kwargs)
  File "/home/vagrant/treeherder/treeherder/webapp/api/jobs.py", line 105, in list
    job["platform_option"] = option_collection_map[option_hash]
KeyError: u'da39a3ee5e6b4b0d3255bfef95601890afd80709'

[2016-05-30 14:16:39,453] ERROR [django.request:256] Internal Server Error: /api/project/try/jobs/
Traceback (most recent call last):
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/rest_framework/viewsets.py", line 87, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch
    response = self.handle_exception(exc)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/vagrant/treeherder/treeherder/webapp/api/utils.py", line 105, in use_jobs_model
    return model_func(*args, jm=jm, **kwargs)
  File "/home/vagrant/treeherder/treeherder/webapp/api/jobs.py", line 105, in list
    job["platform_option"] = option_collection_map[option_hash]
KeyError: u'da39a3ee5e6b4b0d3255bfef95601890afd80709'

Comment 2

[William Lachance (:wlach)]

I don't think we support https for the local vagrant instance. We should document this.

As far as this error is concerned, I believe the problem is that you need to specify an "option" as part of your submission script via `add_option_collection`. I know this is pretty lame, but I generally just recommend that people look at the autophone code when writing treeherder submission stuff:

https://github.com/mozilla/autophone/blob/master/autophonetreeherder.py

Again, it would be nice if the documentation here were improved. Also, of course, we could do a better job of helping people to validate their submissions with jsonschema, etc. Oh, and better error messages on the server side when something is missing, of course.

I suspect bugzilla isn't the best places to debug these problems. Could you post in mozilla.tools.treeherder? That way we'll have a record of these conversations for others to learn from, which would be good as you're definitely not the first to run into these issues. :)

Comment 3

[Ed Morley [:emorley]]

Perhaps the client would be less foot-gun-y if the protocol wasn't a separate argument? ie make them pass 'http://local.treeherder.mozilla.org' or 'https://treeherder.mozilla.org'.

The problem at the moment is that the default value for the protocol argument is 'https', which means:

foo = TreeherderClient(host='local.treeherder.mozilla.org')

...will fail.

Comment 4

[William Lachance (:wlach)]

(In reply to Ed Morley [:emorley] from comment #3)
> Perhaps the client would be less foot-gun-y if the protocol wasn't a
> separate argument? ie make them pass 'http://local.treeherder.mozilla.org'
> or 'https://treeherder.mozilla.org'.
> 
> The problem at the moment is that the default value for the protocol
> argument is 'https', which means:
> 
> foo = TreeherderClient(host='local.treeherder.mozilla.org')
> 
> ...will fail.

This sounds great, can we do it while preserving backwards compat? If so, let's file a bug and do it (or get a contributor to). :)

Comment 5

[Ed Morley [:emorley]]

(In reply to William Lachance (:wlach) from comment #4)
> This sounds great, can we do it while preserving backwards compat? If so,
> let's file a bug and do it (or get a contributor to). :)

I've filed bug 1279213 with some ideas :-)