Got 403 FORBIDDEN when posting resultset to treeherder API

RESOLVED INVALID

Status

RESOLVED INVALID
3 years ago
2 years ago

People

(Reporter: shinglyu, Unassigned)

Tracking

Details

(Reporter)

Description

3 years ago
When I attempted to push data to treeherder.allizom.org with this script (https://github.com/shinglyu/servo-perf/blob/master/submit_to_perfherder.py), I got a 403 forbidden. 

The client ID is "servo". 

This error starts at my May 16 01:05:50 (GMT+8) build. The previous build (May 14) was fine.

Is there any security policy update to the server or is my client secret invalidated? 

Full error message:
===============================================
(venv) vagrant@vagrant-ubuntu-trusty-64:~/buildbot/slave/runtests/build/servo-perf$ python submit_to_perfherder.py output/perf-1463503799.json  servo/revision.json 
[DEBUG] performance data:
{'performance_data': {'framework': {'name': 'servo-perf'}, 'suites': [{'subtests': [{'name': u'56.com', 'value': 3700}, {'name': u'amazon.com', 'value': 4893}, {'name': u'bbc.co.uk', 'value': 1433}, {'name': u'beatonna.livejournal.com', 'value': 1006}, {'name': u'cgi.ebay.com', 'value': 2623}, {'name': u'chemistry.about.com', 'value': 462}, {'name': u'dailymail.co.uk', 'value': 5801}, {'name': u'dailymotion.com', 'value': 1789}, {'name': u'ezinearticles.com', 'value': 602}, {'name': u'globo.com', 'value': 7069}, {'name': u'google.com', 'value': 859}, {'name': u'goo.ne.jp', 'value': 1337}, {'name': u'huffingtonpost.com', 'value': 2442}, {'name': u'imgur.com', 'value': 3659}, {'name': u'mail.ru', 'value': 1851}, {'name': u'myspace.com', 'value': 3086}, {'name': u'page.renren.com', 'value': 1664}, {'name': u'rakuten.co.jp', 'value': 6923}, {'name': u'reddit.com', 'value': 2195}, {'name': u'spiegel.de', 'value': 3923}, {'name': u'stackoverflow.com', 'value': 3038}, {'name': u'tudou.com', 'value': 2531}, {'name': u'w3.org', 'value': 855}, {'name': u'xunlei.com', 'value': 3510.0}, {'name': u'yelp.com', 'value': 2287}, {'name': u'youku.com', 'value': 3468}, {'name': u'youtube.com', 'value': 1565}], 'name': 'domComplete', 'value': 2222.8695297694426}]}}
/home/vagrant/buildbot/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/vagrant/buildbot/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
No handlers could be found for logger "thclient.client"
Traceback (most recent call last):
  File "submit_to_perfherder.py", line 322, in <module>
    main()
  File "submit_to_perfherder.py", line 317, in main
    submit(perf_data, revision)
  File "submit_to_perfherder.py", line 293, in submit
    client.post_collection('servo', trsc)
  File "/home/vagrant/buildbot/venv/local/lib/python2.7/site-packages/thclient/client.py", line 929, in post_collection
    collection_inst.get_collection_data(), timeout)
  File "/home/vagrant/buildbot/venv/local/lib/python2.7/site-packages/thclient/client.py", line 712, in _post_json
    resp.raise_for_status()
  File "/home/vagrant/buildbot/venv/local/lib/python2.7/site-packages/requests/models.py", line 844, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: FORBIDDEN for url: https://treeherder.allizom.org/api/project/servo/resultset/
(Reporter)

Comment 1

3 years ago
@wlach: Sorry to bother you again.
Flags: needinfo?(wlachance)

Comment 2

3 years ago
Looking at the logs our side I see:

[2016-05-18 00:01:14,361] ERROR [treeherder.webapp.api.exceptions:25] Traceback (most recent call last):
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/views.py", line 454, in dispatch
    self.initial(request, *args, **kwargs)
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/views.py", line 376, in initial
    self.perform_authentication(request)
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/views.py", line 310, in perform_authentication
    request.user
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/request.py", line 353, in __getattribute__
    return super(Request, self).__getattribute__(attr)
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/request.py", line 193, in user
    self._authenticate()
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/rest_framework/request.py", line 316, in _authenticate
    user_auth_tuple = authenticator.authenticate(self)
  File "/data/www/treeherder.allizom.org/venv/lib/python2.7/site-packages/hawkrest/__init__.py", line 90, in authenticate
    raise AuthenticationFailed(msg)
AuthenticationFailed: Hawk authentication failed: The token has expired. Is your system clock correct?

The string "Hawk authentication failed: The token has expired. Is your system clock correct?" would have been shown in your logs, however it looks like you don't have logging set up:
`No handlers could be found for logger "thclient.client"`
(see eg http://stackoverflow.com/a/346501)

Bug 1229025 is filed to add some logging tips to the docs, but for now add something like this to submit_to_perfherder.py:

```
import logging

logging.basicConfig(level=logging.DEBUG)

# ...
```

As for the root cause:
"Note: The system clock on the machines making submissions must be correct (or more specifically, within 60 seconds of the Treeherder server time), otherwise authentication will fail."
(from http://treeherder.readthedocs.io/submitting_data.html#authentication)
Flags: needinfo?(wlachance)

Comment 3

3 years ago
Ah it's 9 mins out:

[2016-05-18 00:01:14,361] WARNING [hawkrest:81] access denied: TokenExpired: token with UTC timestamp 1463554379 has expired; it was compared to 1463554874

Updated

3 years ago
Flags: needinfo?(slyu)
(Reporter)

Comment 4

3 years ago
Geee! Let me sync my PC's time.
Flags: needinfo?(slyu)

Comment 5

2 years ago
A reminder about logging is being added to the docs in:
https://github.com/mozilla/treeherder/pull/1535
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.