864451 - Re-enable product icon API

Status: RESOLVED FIXED

(Marketplace Graveyard :: Payments/Refunds, defect, P2)

Description

[Kumar McMillan [:kumar]]

The product icon API in webpay (bug 848675) is disabled because of authentication issues. This is a placeholder to re-enable it.

Comment 1

[Kumar McMillan [:kumar]]

Disabled: https://github.com/mozilla/webpay/commit/70f34db8b86cc5c7da2e01963302f76dbe5e93b0

Comment 2

[Kumar McMillan [:kumar]]

We're trying to get more logging to figure this out. The error you see in webpay is:

Apr 22 11:58:19 celery1.dev.addons.phx1.mozilla.com: [] z.celery:ERROR Celery TASK exception: HttpClientError: Client Error 401: https://marketplace-dev.allizom.org/api/webpay/product/icon/ ...

Comment 3

[Andy McKay]

Added in lots and lots of logging.

Got to this. The headers being sent are:

Apr 22 16:23:18 dev2.addons.phx1.mozilla.com: [<anon>][63.245.216.220] z.api:INFO Headers: OAuth realm="" :/data/www/addons-dev.allizom.org/zamboni/mkt/api/authentication.py:85

Just a bit missing then.

Apr 22 16:23:18 dev2.addons.phx1.mozilla.com: [<anon>][63.245.216.220] z.api:ERROR ValueError on verifying_request :/data/www/addons-dev.allizom.org/zamboni/mkt/api/authentication.py:117#012Traceback (most recent call last):#012  File "/data/www/addons-dev.allizom.org/zamboni/mkt/api/authentication.py", line 115, in is_authenticated#012    require_resource_owner=False)#012  File "/data/www/addons-dev.allizom.org/venv/lib/python2.6/site-packages/oauthlib/oauth1/rfc5849/__init__.py", line 849, in verify_request#012    signature_type, params, oauth_params = self._get_signature_type_and_params(request)#012  File "/data/www/addons-dev.allizom.org/venv/lib/python2.6/site-packages/oauthlib/oauth1/rfc5849/__init__.py", line 605, in _get_signature_type_and_params#012    raise ValueError('oauth_ params are missing. Could not determine signature type.')#012ValueError: oauth_ params are missing. Could not determine signature type.

Is then the error. Next figuring out why.

Comment 4

[Andy McKay]

I'd like to have proof if possible from jason or oremj that something wierd isn't happening with requests here. From everything I can tell they are being sent correctly, just not being received correctly. 

Works locally. 

Sending the same kind of wrong header locally produces the same kind of error.

I note the same error happens with other webpay OAuth api, such as failure, so it's not limited to this API.

Jason or oremj, can you spend some time in a shell with TCPDump or something with me please?

Comment 5

[Andy McKay]

If you change webpay to connect to marketplace without HTTPS it works just fine:

[[e-mail]1 webpay]$ ../venv/bin/python manage.py shell
Python 2.6.6 (r266:84292, Oct 12 2012, 14:23:48) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.conf import settings
>>> settings.MARKETPLACE_URL = 'http://127.0.0.1:10002'
>>> from lib.marketplace.api import client
>>> client.slumber.api.v1.webpay.product.icon.get(data={'url':'http://test'})
{'meta': {'previous': None, 'total_count': 4, 'offset': 0...}

The logs:

[[e-mail]1 ~]$ tail -f logs/http_app_addons_marketplacedev/2013/04/2013-04-23 | grep api
Apr 23 12:00:09 dev1.addons.phx1.mozilla.com: [<anon>][127.0.0.1] z.api:INFO Request: /api/v1/webpay/product/icon/ :/data/www/addons-dev.allizom.org/zamboni/mkt/api/base.py:74
[snipped]
Apr 23 12:00:09 dev1.addons.phx1.mozilla.com: [<anon>][127.0.0.1] z.api:INFO Headers: OAuth realm="", oauth_body_hash="2jmj7l5rSw0yVb%2FvlWAYkK%2FYBwk%3D", oauth_nonce="65933570", oauth  ..[snipped]
[snipped]
Apr 23 12:00:09 dev1.addons.phx1.mozilla.com: [<anon>][127.0.0.1] z.api:INFO Authorizer PermissionAuthorization returned: True :/data/www/addons-dev.allizom.org/zamboni/mkt/api/authorization.py:39

Comment 6

[Andy McKay]

Jason disabled a "addons-dev-fix-auth" script in Zeus and things started to work again.

Comment 7

[Jason Thomas [:jason]]

We added 'addons-dev-fix-auth' Zeus traffic script to fix a issue with github web hooks to addons-updater service. github was sending double Authentication headers which Zeus combined and caused nginx HTTP basic auth to fail. 

It looks like github fixed this on their end now and we no longer need this traffic script.

Comment 8

[Andy McKay]

Kumar, I think you can re-enable now.

Comment 9

[Kumar McMillan [:kumar]]

I re-enabled it on dev to test as of https://github.com/mozilla/webpay/commit/3ac3198e05a8f5a75ae57e631d6799272ed1f19b but bug 863487 is making it hard to test. I think because I got that far that it's working though.

Comment 10

[Kumar McMillan [:kumar]]

re-enabled on stage https://github.com/mozilla/webpay/commit/337a8d937e13f1c1c17c0dda3683637396a9e138

Comment 11

[Kumar McMillan [:kumar]]

I got a 401 on stage:

HttpClientError: Client Error 401: https://marketplace.allizom.org/api/v1/webpay/product/icon/
Content: 

http://sentry.dmz.phx1.mozilla.com/addons/marketplace-stage-webpay/group/13461/

so I disabled it again. I can't seem to find what the actual problem is though. Can anyone tell if this is still the same header issue? Maybe stage just needs the right oauth credentials?

Comment 12

[Kumar McMillan [:kumar]]

Is everything from bug 860113 configured right?

Comment 13

[Kumar McMillan [:kumar]]

Now icons seem to be broken everywhere (not just stage) so I disabled them everywhere in https://github.com/mozilla/webpay/commit/a352322a6e7018f3bf9f6a675b2eeadf8c4dc849

The error on dev is the same: http://sentry.dmz.phx1.mozilla.com/addons/marketplace-dev-webpay/group/13719/

Client Error 401: https://marketplace-dev.allizom.org/api/v1/webpay/product/icon/
Content:

Comment 14

[Andy McKay]

https://github.com/mozilla/webpay/commit/e4fa98

Comment 15

[Kumar McMillan [:kumar]]

Thanks for the fix, Andy! It's working in dev. We will enable it Monday morning on stage so we don't ruin the CET morning if something goes wrong.

Comment 16

[Kumar McMillan [:kumar]]

product icons are enabled on stage and working https://github.com/mozilla/webpay/commit/ef7602696fa5e84d79931f145e74343fd5065391