734445 - Design project plan for receipt generation service

Status: RESOLVED WONTFIX

(Core Graveyard :: DOM: Apps, defect)

Description

[Michael Hanson]

Receipt Generation Service
Project Plan

The Receipt Generation Service is a high-availability service that generates digitally-signed receipts for objects vended by the Marketplace.
The receipts are intended to be a complete proof of purchase that can be independently verified by third parties.

System Overview

Receipts are essentially an RSA 2048-bit signed triple of (user ID, store ID, product URL/ID).  They contain other useful metadata, but from a trust standpoint those are the important pieces.

The receipt trust system is rooted in a small number (1, or 2-4; see appendix A) of asymmetric keypairs.  The private keys are held in HSMs, and require k-of-n procedures to change.  The root receipt trust nodes are responsible for generating daily receipt signing keys, and certifying them.  Within a given site, only one of these nodes is active; the other is on hot standby.  Each site is responsible for generating its daily keys, since a receipt generation cluster could become active on short notice.

The public keys are carefully exported from the HSMs and conveyed to a distribution point that provides 3rd parties with confidence (see Appendix B).
A horizontally-scalable cluster of receipt signing nodes receives one of these keys from the root nodes every day.  These signing nodes are responsible for receiving unsigned receipts from the Marketplace, signing them, logging the fact of signing, and returning the signed receipt as the response.

When an authenticated user session makes a receipt request, the Marketplace issues a request-to-sign to the receipt signing cluster, logs the request, and returns the signed receipt to the user.

The user agent makes requests for receipts when Marketplace content uses the Application DOM API to request an install.  The receipts are stored, along with application metadata, in the user agent, where they are made available to the developer's domain through the Application DOM API.  The developer is then expected to upload the receipt to their own server, where they can perform validation.

Validation of the receipt can be done by the developer's server, by verifying the signature, and then verifying the chain of certificates.  The root of trust of the chain will be one of the public certificates previously advertised by the Marketplace in ##.

Alternatively, the developer can POST the receipt to a URL specified in the receipt (the verify_url).  This is a service, hosted by Marketplace, which will:

1. Verify the receipt and certificate chain
2. If verification succeeds, retrieve the business state of the receipt (one of okay, refunded, rejected, invalid) (XX language check here)
3. Report back the verification of the receipt and the business state of the receipt.

(NB to Marketplace team: If there is going to be a developer-facing API to query the state of individual receipts by user ID, it will need to have a developer-facing authentication piece.  This system does not require authentication; possession of the receipt is considered sufficient to find out the user's status).

Software Components

The root signing nodes are responsible for generating a daily list of keys, certifying them with a hardware-protected private key.  They push these certified keys to the signing nodes through a secure, intranet-only link.  There is no internet access to the root signing nodes.  The public half of these keys is exported and delivered very carefully to the advertising point.
The receipt signing nodes are responsible for signing receipt-requests that comes form Marketplace nodes.  They receive new private keys once a day.  XXX Best case would be to hold the key in memory only - but how do we handle unexpected process termination and restarts?  If there's no way to request a new key from the root nodes we're stuck. 

The Marketplace Verification service needs to receive the public keys, and to perform verification of the receipt and the certificate chain.

Threat Model

An attacker could capture a signing key and export it from the signing node.  They could make as many nodes as they like.
Countermeasure: The certification of the signing key would expire after a short number of days; from that point onward the attacker could not mint new receipts.  (XXX wait, that's not right, you could mint new receipts for the old time range, but they're still acceptable.)

An attacker could compromise a signing node and create as many receipts as they like.
Countermeasure: Infrasec will correlate the signing activity log with actual requests from the Marketplace.

An attacker could create their own receipt, sign it with their own key, certify that key with their own private key, and attach that.
Countermeasure: The developer must verify that the chain is based on a public key that comes from the store the receipt claims to be from.

An attacker could compromise the public key distribution point of Marketplace, place their own key there, and produce their own receipts rooted in that key.
Or: Attacker could compromise the SSL chain leading to the public key distribution point and MITM their own key.
Countermeasure: Changes to the public key should be accompanied by out-of-band signaling from Marketplace; any other change is an attack.  Geographic tripwiring would be needed to catch the MITM.

An attacker could compromise k-of-n Mozilla employees, and export the private key.

An attacker could compromise the channel through which the public keys are relayed from the root trust nodes to the distribution point, and place their own public key in the list.

Operational Procedures

* The public keys need to be advertised very securely; any tampering with this trust root enables receipt-impersonation attacks.
* A receipt-generation site would need to detect if a root node has stopped generating private keys, and failover to the backup node.
* Marketplace would need to detect if a receipt signing site has stopped responding to requests from Marketplace, and fail over to a backup site.

Appendix A: One key or 2-4?

The HSMs have mechanisms to sync a private key pair across all nodes; if we do this we have only one private key.  If not, we have four, 

Appendix B: How to distribute the public keys?

The straightforward approach is to place them on an SSL-protected website.  We would then be vulnerable to any attack which can compromise SSL.

Comment 1

[Daniel Veditz [:dveditz]]

Why is this a hidden bug? Seems more like a design document that would go on wiki.mozilla.org. Is this a review request bug?

Comment 2

[Michael Hanson]

I defer to mcoates to decide whether it should be hidden, since it includes discussion of security implications of internal infrastructure.  'twas originally on an etherpad and email thread; trying to get it in the right place for actual feedback.

Comment 3

[Michael Coates [:mcoates] (acct no longer active)]

I'm ok with this being open. I'm also ok with moving it to a feature page or somewhere else.  Before it was in a etherpad where people didn't have accounts.

A bug is better, perhaps the next step is a feature page with design specs. 

Copying in Kumar who will be doing lots of the dev here.

Comment 4

[Kumar McMillan [:kumar]]

andym is out next week but also cc'ing him as he'll be focusing on receipts

Comment 5

[Jason Smith [:jsmith]]

Why is this bug marked under infrastructure? I thought bugs marked under infrastructure have to do with setting up servers, staging environment setup, build environment setup, etc. Is David Clarke the correct assignee on this bug?

Comment 6

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

schedule for 23-Mar

Comment 7

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

Review Notes:

;Items to be reviewed:
(this bug)
https://wiki.mozilla.org/Apps/WebApplicationReceipt

== Conclusions / Action Items ==
* Who :: What :: By when
* Bill to verify with Justin about plan for receipt revocation
* Need to design and implement a receipt reissue system
* Review to verify daily keys are correctly destroyed each day
* Need to alter receipt verification to cope with the proposed signing chain
* need to define process for recovation/re-issue of root key

Comment 8

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

[More notes]
How we are going to authenticate the signers.  
authenticate server (marketplace) requesting signing.
Operations Security Tasks
1. write ocs/acs (hsm smartcards) operational procedures and policy
2. program ocs/asc (hsm smartcards)
3. publish system and network security requirements for these servers and hosts
4. verify that CEF logging in place for each receipt signing operation. <joes><ray>
risk
-----
server ip compromise could allow push of signing cert from root cert to malicious server.
multiple refunds against non-valid transactions or just too many refunds.

Comment 9

[Michael Coates [:mcoates] (acct no longer active)]

(In reply to Curtis Koenig [:curtisk] from comment #8)
> [More notes]
> How we are going to authenticate the signers.  
> authenticate server (marketplace) requesting signing.
> Operations Security Tasks
> 1. write ocs/acs (hsm smartcards) operational procedures and policy
> 2. program ocs/asc (hsm smartcards)
> 3. publish system and network security requirements for these servers and
> hosts
> 4. verify that CEF logging in place for each receipt signing operation.
> <joes><ray>
> risk
> -----
> server ip compromise could allow push of signing cert from root cert to
> malicious server.
> multiple refunds against non-valid transactions or just too many refunds.

Curtis - please move discussions / comments to the wiki page https://wiki.mozilla.org/Apps/WebApplicationReceipt/GenerationService

Comment 10

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

Notes moved

Comment 11

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

since we did a review I am marking this sec-review+ I know we have other bugs ongoing but I think rforbes is working those issues

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Core Graveyard / DOM: Apps is inactive. Closing all bugs in this component.