1067390 - Restoring users products from receipt does not work (ERROR_DECODING)

Status: VERIFIED FIXED

(Marketplace Graveyard :: Payments/Refunds, defect, P1)

Description

[[:AndreiH]]

#STR:

1. Use application with FF OS https://marketplace.allizom.org/app/in-app-payments-test/
2. Buy a real product https://marketplace.allizom.org/developers/app/in-app-payments-test/in-app-payments/in-app-products/
3. Close application 
4. Reopen application

#Expected results
Products bought should be restored from receipts

#Actual results
Products couldn't be restored from receipts:

"xhr load: GOT" "{"status": "invalid", "reason": "ERROR_DECODING"}" fxpay.js:768
"verification result:" Object { status: "invalid", reason: "ERROR_DECODING" } fxpay.js:268
"receipt" "eyJhbGciOi" "is invalid; service returned:" "invalid" "ERROR_DECODING" fxpay.js:288
"error restoring product" "fd60244c-22b0-4814-95fe-80ac43cb0196" "message:" "INVALID_RECEIPT"

Comment 1

[[:AndreiH]]

Attachment: Screenshot.png

Comment 2

[[:AndreiH]]

Attachment: Screenshot2.png

Second screenshot with "derived base API URL from receipt:" "https://marketplace.allizom.org"

In this one I used a fake product

Comment 3

[Kumar McMillan [:kumar]]

I can reproduce this as well. Here is my purchase receipt that resulted in {"status": "invalid", "reason": "ERROR_DECODING"}

eyJhbGciOiAiUlMyNTYiLCAidHlwIjogIkpXVCIsICJqa3UiOiAiaHR0cHM6Ly9tYXJrZXRwbGFjZS1kZXYtY2RuLmFsbGl6b20ub3JnL3B1YmxpY19rZXlzL3Rlc3Rfcm9vdF9wdWIuandrIn0.eyJpc3MiOiAiaHR0cHM6Ly9tYXJrZXRwbGFjZS1kZXYtY2RuLmFsbGl6b20ub3JnL3B1YmxpY19rZXlzL3Rlc3Rfcm9vdF9wdWIuandrIiwgInByaWNlX2xpbWl0IjogIjEwMCIsICJqd2siOiBbeyJhbGciOiAiUlNBIiwgIm1vZCI6ICJBTVZnck9VWkIxeXQwMmxld1F4MHJjLTM5dEZkRkVfLW1GX0oxSG1NZ2ktR2xQOEMxTWJqY212WWwwZFZXeHRvNlZPYnBncWo5QVM5NVZJQmdMZ0tmSXVUQkV3S2kzT3FrNEp2aDI5ZjF6VWNYQ3lfeXV5d09WX1gzNWxPaWRRYUdmaU1KaVhXT0FVZngzYnNMZEw4Mk0wRjU0cGRmN2N3bGlQaUFGYkNhM0hMOGNaQ3pIeU9sSmhBY3lWSElOd0xabE02SElad1JLeG1VdzhuRlpJVWREb2VjWUVEREc0SkJqLXN1MmtwVzRYTW15RHNHTGdNZ290Si15Z0lxbnduWFkwcFRVUTVWM245aHE3YzZzSW5TNk51dXZhUWhUX3I1dmFBS3VycHdSOS0zejRpbXNRV2FTVm1tMzZrMnZ3Y1ZEWldHdkR6UG1xdW9lSFZzekE0dXI4IiwgImV4cCI6ICJBUUFCIiwgImtpZCI6ICJzaWduZXIuc3RhZ2UuYWRkb25zLnBoeDEubW96aWxsYS5jb20ifV0sICJleHAiOiAxNDE2MzU4NzY4LCAiaWF0IjogMTM4NDgyMjc2OCwgInR5cCI6ICJjZXJ0aWZpZWQta2V5IiwgIm5iZiI6IDEzODQ4MjI3Njh9.nJ1qnaEaXhnTevXDC8b1FKQeWU4iQ3Qld44Oohx9IwSr7LkI5uGpAlCilYoumtKf0GiNYHiB_IAXnT_Pez15CtE_tsu9Xy4fl0X5pll5hh9wKo7Dnkxu9uJwNiNE8vuotvKR7SfYukeMEwE3nHfEREFU87Frs8wUgauxWo880G88lkaT20AArebmpg_I_iH8ldlbSj05iAocDbzjKSHsDmryqZzqLFFV5qwXmOCtTRGWNnhug-eiWmZnDqukA41tWF_OD_paP9EM8iP2vmpSNVhavkrQKk2v6-U5VYzueSMXqvk964yHunrwUCktFWIys1ItNpwDE9Fv5_36IPkHAg~eyJqa3UiOiAiaHR0cHM6Ly9tYXJrZXRwbGFjZS1kZXYtY2RuLmFsbGl6b20ub3JnL3B1YmxpY19rZXlzL3Rlc3Rfcm9vdF9wdWIuandrIiwgInR5cCI6ICJKV1QiLCAiYWxnIjogIlJTMjU2In0.eyJwcm9kdWN0IjogeyJ1cmwiOiAiYXBwOi8vbXktYXBwLmNvbSIsICJzdG9yZWRhdGEiOiAiY29udHJpYj0xMTkxJmlkPTUwNjA2MSZpbmFwcF9pZD1mZDYwMjQ0Yy0yMmIwLTQ4MTQtOTVmZS04MGFjNDNjYjAxOTYifSwgImlzcyI6ICJodHRwczovL21hcmtldHBsYWNlLmFsbGl6b20ub3JnIiwgInZlcmlmeSI6ICJodHRwczovL3JlY2VpcHRjaGVjay1tYXJrZXRwbGFjZS5hbGxpem9tLm9yZy92ZXJpZnkvIiwgImRldGFpbCI6ICJodHRwczovL21hcmtldHBsYWNlLmFsbGl6b20ub3JnL2FwaS92MS9yZWNlaXB0cy9yZWlzc3VlLyIsICJyZWlzc3VlIjogImh0dHBzOi8vbWFya2V0cGxhY2UuYWxsaXpvbS5vcmcvYXBpL3YxL3JlY2VpcHRzL3JlaXNzdWUvIiwgInVzZXIiOiB7InR5cGUiOiAiZGlyZWN0ZWQtaWRlbnRpZmllciIsICJ2YWx1ZSI6ICJhbm9ueW1vdXMtdXNlciJ9LCAiZXhwIjogMTQyNjUyNjU1OCwgImlhdCI6IDE0MTA4MDE3NTgsICJ0eXAiOiAicHVyY2hhc2UtcmVjZWlwdCIsICJuYmYiOiAxNDEwODAxNzU4fQ.AGSzb4arD9kjFSOB-f6xPMHlYhMlWR-YxGeXHyB2Zbws211HRD_KSaOq3leEJzlrlzelJZsY7u7yWHOtMIus7yTScPKqW7PyC4BeZF-zSDfxFMb74A91OMlsVajMXavn4e0SrUnIw5KvczdpnDgiXNIrNSbZ41QfOjufU5sZGLuH2WHyHyE7mnxJDU39KJklPxyx23HPv2kjr2NmX7Vns9VO7aoKVYH0bxBavTdWCdwVzGjEQouQlKOZiQOI1PXBgsEtv_dRNH8drVqvxaxX5N1rIcrWAsgtq8NbKrvqV8SftYr-eZjVtWZhyNRCSESH_gBWhKPFhYys6CqyGkovdg

Comment 4

[Kumar McMillan [:kumar]]

Looks like we might have a couple errors here:

Sep 15 17:23:31 web1.stage.addons.phx1.mozilla.com: [][] z.receipt:ERROR Type: <type 'exceptions.ValueError'>, Certificate issuer invalid: marketplace-dev-cdn.allizom.org. Data: {'app': None, 'receipt': 'eyJhbGciOi...'} :/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/utils.py:62#012Traceback (most recent call last):#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 118, in decode#012    receipt = decode_receipt(self.receipt)#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 375, in decode_receipt#012    result = verifier.verify(receipt)#012  File "/data/addons-stage/www/marketplace.allizom.org/current/venv/lib/python2.6/site-packages/receipts/certs.py", line 130, in verify#012    cert = self.verify_certificate_chain(certificates, now=now)#012  File "/data/addons-stage/www/marketplace.allizom.org/current/venv/lib/python2.6/site-packages/receipts/certs.py", line 167, in verify_certificate_chain#012    self.check_certificate_issuer(root_issuer)#012  File "/data/addons-stage/www/marketplace.allizom.org/current/venv/lib/python2.6/site-packages/receipts/certs.py", line 146, in check_certificate_issuer#012    raise ValueError('Certificate issuer invalid: %s' % domain)#012ValueError: Certificate issuer invalid: marketplace-dev-cdn.allizom.org
Sep 15 17:23:31 web1.stage.addons.phx1.mozilla.com: [][] z.receipt:INFO Error decoding receipt :/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/utils.py:67
Sep 15 17:23:31 web1.stage.addons.phx1.mozilla.com: [][] z.receipt:INFO Invalid store data: Verify instance has no attribute 'decoded' :/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/utils.py:67

Comment 5

[Kumar McMillan [:kumar]]

manually decoded version of receipt from comment #3:

{u'product': {u'url': u'app://my-app.com', u'storedata': u'contrib=1191&id=506061&inapp_id=fd60244c-22b0-4814-95fe-80ac43cb0196'}, u'iss': u'https://marketplace.allizom.org', u'verify': u'https://receiptcheck-marketplace.allizom.org/verify/', u'detail': u'https://marketplace.allizom.org/api/v1/receipts/reissue/', u'reissue': u'https://marketplace.allizom.org/api/v1/receipts/reissue/', u'user': {u'type': u'directed-identifier', u'value': u'anonymous-user'}, u'exp': 1426526558, u'iat': 1410801758, u'typ': u'purchase-receipt', u'nbf': 1410801758}

Comment 6

[Kumar McMillan [:kumar]]

and more importantly, here's the key segment from comment #3:

{u'iss': u'https://marketplace-dev-cdn.allizom.org/public_keys/test_root_pub.jwk', u'typ': u'certified-key', u'jwk': [{u'alg': u'RSA', u'kid': u'signer.stage.addons.phx1.mozilla.com', u'exp': u'AQAB', u'mod': u'AMVgrOUZB1yt02lewQx0rc-39tFdFE_-mF_J1HmMgi-GlP8C1MbjcmvYl0dVWxto6VObpgqj9AS95VIBgLgKfIuTBEwKi3Oqk4Jvh29f1zUcXCy_yuywOV_X35lOidQaGfiMJiXWOAUfx3bsLdL82M0F54pdf7cwliPiAFbCa3HL8cZCzHyOlJhAcyVHINwLZlM6HIZwRKxmUw8nFZIUdDoecYEDDG4JBj-su2kpW4XMmyDsGLgMgotJ-ygIqnwnXY0pTUQ5V3n9hq7c6sInS6NuuvaQhT_r5vaAKurpwR9-3z4imsQWaSVmm36k2vwcVDZWGvDzPmquoeHVszA4ur8'}], u'exp': 1416358768, u'iat': 1384822768, u'price_limit': u'100', u'nbf': 1384822768}

Comment 7

[Kumar McMillan [:kumar]]

This is how the signer is configured in Zamboni on stage:

SIGNED_APPS_SERVER = 'http://app-signer.marketplace.allizom.org'
SIGNING_SERVER = 'http://signer.stage.addons.phx1.mozilla.com'

Comment 8

[Kumar McMillan [:kumar]]

The issuer problem seems to be resolved.

jason: kumar: I regenerated the root and root key + key and jwk.
jason: kumar: https://marketplace-cdn.allizom.org/public_keys/root_pub.jwk
jason: >>> jwt.decode(a,verify=False)
jason: {u'iss': u'https://marketplace-cdn.allizom.org/public_keys/root_pub.jwk', u'typ': u'certified-key', u'jwk'

However, I see this error now which is causing a 500 response to the validator:

Sep 15 23:37:32 web1.stage.addons.phx1.mozilla.com: [][] z.receipt:ERROR Type: <class '_mysql_exceptions.ProgrammingError'>, (1146, "Table 'addons_allizom_org_new.inapp_products' doesn't exist"). Data: <none> :/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/utils.py:62#012Traceback (most recent call last):#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 412, in receipt_check#012    return 200, json.dumps(verify.check_full())#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 71, in check_full#012    self.check_purchase()#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 232, in check_purchase#012    self.check_purchase_inapp()#012  File "/data/addons-stage/www/marketplace.allizom.org/deploy-zamboni-stage-20140912191240-356168a415/zamboni/services/verify.py", line 246, in check_purchase_inapp#012    {'contribution_id': self.get_contribution_id()}#012  File "/data/addons-stage/www/marketplace.allizom.org/current/venv/lib/python2.6/site-packages/MySQLdb/cursors.py", line 205, in execute#012    self.errorhandler(self, exc, value)#012  File "/data/addons-stage/www/marketplace.allizom.org/current/venv/lib/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler#012    raise errorclass, errorvalue#012ProgrammingError: (1146, "Table 'addons_allizom_org_new.inapp_products' doesn't exist")

Comment 9

[Kumar McMillan [:kumar]]

Jason, I think the SERVICES_DATABASE_URL settings on zamboni stage needs updating. The MySQL error is:

"Table 'addons_allizom_org_new.inapp_products' doesn't exist"

I think it should be connecting to the same DB as in DATABASES_DEFAULT_URL

Comment 10

[Jason Thomas [:jason]]

I've updated SERVICES_DATABASE_URL and pushed the changes out to stage.

Comment 11

[[:AndreiH]]

Attachment: Screenshot from 2014-09-16.png

Verified this and it seems that fxPay is now restoring products without any errors (see screenshot).
So devs should be able now to use the restored products as they please.
Is this fully fixed?

Comment 12

[[:AndreiH]]

I just modified the app and the "onRestore" function works now.
My bought apps are restored from the receipt and shown into the app when I close and reopen the app.

Thanks!

Comment 13

[Kumar McMillan [:kumar]]

Yes, this is fixed for me too. Thanks jason! It was just a settings error on our stage server so production should not be affected. I filed bug 1068070 so we can catch these misconfigurations sooner.

Comment 14

[[:AndreiH]]

This is now verified fixed. Thank!