Prior to a future 1.1 release of the SDK (likely in mid or late Q2) AMO must have the ability to repackage add-ons when new versions are released so add-ons continue to be compatible. In talking to Justin Scott (Fligtar), the most attractive solution would be for FlightDeck to provide and API that is able to receive a package or XPI, and create a new XPI from it using a version of the SDK specified by the caller. In observing the performance of the current XPI creation service, such an offering would likely require more hardware. I would like you to include in this investigation a rough estimate for any additional infra you feel you would need (if possible). The end of March is when I'd like to have initial feedback on this item. If any further clarification on the goals of this ticket are required, feel free to ping me or schedule a call.
Assignee: nobody → zaloon
Priority: -- → P4
Target Milestone: --- → Builder 0.9.2
I think it's an equivalent of 1. upload XPI from Dashboard 2. click on Download 3. delete Add-on Fairly simple as we already have all functionalities The only problem is to fix upload to support libraries as well
Target Milestone: Builder 0.9.2 → Builder 0.9.3
Problem with private key ... http://flightdeck.zalewa.info/xpi/repackage/123/awesome/ "Your package.json says our ID is: jid0-GXjLLfbCoAx0LcltEdFrEkQdQPI@jetpack But I don't have a corresponding private key in: /tmp/ePEmWNaWpp/keydir/jid0-GXjLLfbCoAx0LcltEdFrEkQdQPI@jetpack If you are the original developer of this package and have recently copied the source code from a different machine to this one, you should copy the private key into the file named above"
(In reply to comment #2) > Problem with private key ... > > http://flightdeck.zalewa.info/xpi/repackage/123/awesome/ > > "Your package.json says our ID is: jid0-GXjLLfbCoAx0LcltEdFrEkQdQPI@jetpack But > I don't have a corresponding private key in: > /tmp/ePEmWNaWpp/keydir/jid0-GXjLLfbCoAx0LcltEdFrEkQdQPI@jetpack If you are the > original developer of this package and have recently copied the source code > from a different machine to this one, you should copy the private key into the > file named above" Hmm? Can you explain what steps led you to that message?
Priority: P4 → P1
Summary: Investigate offering CFX XPI creation as an API service → Develop a CFX XPI repackaging service w/ external API
Target Milestone: Builder 0.9.3 → Builder 0.9.4
I extract data from install.rdf and package dir, in install.rdf there is an id field which contains a string paired with private key. To create a xpi with given id one has to provide private key as well. I could generate one for each xpi created, but as I understand it is the same as changing the owner of the add-on. "[...] Otherwise, if you are a new developer who has made a copy of an existing package to use as a starting point, you need to remove the 'id' property from package.json, so that we can generate a new id and keypair. This will disassociate our new package from the old one."
Brian/Daniel: Apparently you guys came up with a solution for the signing problem. Can you please post an update here?
Signing is a separate matter from generating the ID keys for each XPI. Signing will be a decision that occurs later, but either way will very closely mirror the way we will do ID key generation as described below: The service shall require the caller to submit with the post data their private key which will be use to generate the XPI that includes an ID key derived from that private key. If no private key is passed, the service will assume the add-on is from the Add-on Builder and look for the private key there. Additionally, this requires that we let users upload private keys to the Builder, or have it generate one for them and store it. A benefit sending the private key over the wire as a API parameter is that external applications can use the packaging feature even if they do not use AMO or the Builder for any add-on related services (think an Eclipse or TextMate plugin that creates add-ons with your data). Please post any questions or requests for clarifications here and Brian or I will be sure to answer them promptly.
You're going to require a private key from anyone that is hosting an add-on on AMO? That's a big change on the AMO side that no one was expecting. Additionally, you're talking about uploading private keys. Private keys are one of the most protected resources by people who know what they are doing - I don't think it's good practice to ask people to upload them: not good for us to be responsible for storing them and not good for end users to be in the habit of giving their private keys to anyone.
[summary: For now, let's change the JID when transferring an addon between Flightdeck and the SDK, and not attempt to transfer keys at all.] So.. We don't have a story yet for signatures in add-ons. (We've tried to get the discussion started (https://forums.mozilla.org/addons/viewtopic.php?f=27&t=2939&sid=674a50a331b533d1403c978eee25d481) but so far nobody's participating. Please jump in there, or in the corresponding post on the addon-sdk mailing list, and let us know what you think.) A year ago, I added some key-handling code to the SDK so that we'd have some tools available for when we finally do figure out the story: 1: the SDK creates a keypair for each addon, the first time you do 'cfx xpi' 2: it derives the JID and addon ID from the public verifying key, writes it into package.json 3: it stores the private signing key in ~/.jetpack/keys/ 4: it complains at appropriate times to teach people to copy or replace keys when appropriate, so they don't accidentally delete private keys or leave them behind on an old computer In particular, deriving the JID from the single per-addon key completely eliminates the common "which key should I use" question, which (in my experience) is what enables safe design of things like upgrade mechanisms and secure data storage. With this approach, our future story could include statements like: * XPIs presented to the AMO reviewers include components signed by the addon author, not relying upon transport-layer security between the author and the AMO upload server * updated versions of the .xpi will be signed, and the Add-On Manager can check these signatures before installing the update * the SDK provides an API module for each addon to publish data in an unspoofable way, through a URL that includes the JID This approach requires that the per-addon private key remain available to the add-on developer, hence the error message when you run 'cfx xpi' and don't have the private key. If we were actually using that key to e.g. sign XPIs during 'cfx xpi', then XPI generation on Flightdeck would also require the privkey (to maintain the same JID), and we'd need some mechanism to transfer them from the SDK's ~/.jetpack/keys/ into the flightdeck server. Likewise, if a developer starts with flightdeck and "goes unplugged" to develop locally with the SDK, they'd need to be able to download the automatically-generated privkey. But, in either case, we could just change the JID instead of transferring the privkey. Especially now, when we don't really have a signing story, it's probably easiest to declare that the JID must change when you transfer a project between the SDK and Flightdeck, and not try to build any sort of key-transfer mechanism. > Additionally, you're talking about uploading private keys. Private keys are > one of the most protected resources by people who know what they are doing > - I don't think it's good practice to ask people to upload them: not good > for us to be responsible for storing them and not good for end users to be > in the habit of giving their private keys to anyone. Yeah, I understand. In this case, the private key is narrowly scoped: its possessor has the authority to define what qualifies as valid addon code for one specific JID, e.g. anyone who knows the privkey "pwltj4d3pp23ybkjwmqsfbausxaukvcljnwbwq3f5r3mtcxwnfga" gets to control an (imaginary) addon I wrote named "MySuperDuperAddon" with a JID of "jid0-aWkr4xfT5JCADAg2LXVzPWbzUx4". Depending upon where our signing story goes, that may mean the right to access stored data under that JID, or the right to provide upgrades for old versions of that specific addon, or the right to submit code to AMO review for consideration of replacing a specific addon. By uploading the MySuperDuperAddon privkey to Flightdeck, I'd be delegating those rights to Flightdeck. This is a necessary step if I want Flightdeck to create XPIs for me (and I want them to have those signature-based security properties). I fully agree that we need to teach users to be thoughtful about delegating authority, and make it clear what sort of power they're giving away (just like with OAuth tokens). It'd be nice if they didn't have to give up raw key material to do it, but if "no client-side code" is Flightdeck's mission, then I don't see any other way to do it. Other options: I could have Flightdeck create an unsigned XPI and give it to me for signing, but then I'd have to run SDK code locally, which defeats some of the purpose of Flightdeck. I could withhold the key and ship unsigned XPIs, but then they couldn't have those signature-based rights. We could not use signatures in/on XPIs at all, and not bother with key-derived-JIDs (and I'm not ruling this out, mind you), but that could tie our hands later, leaving us with a big design gap as we try to add non-JID-based keys later and have to answer the question of whether a given signature is by the "right" key or not.
Also, see Bug 654588 where I propose that we remove the JID keys from the SDK, which probably moots all of this.
Current repackage service is working as follows (all in celery): 1. download xpi 2. copy SDK to some temp directory (tSDK) 3. unpack xpi into tSDK/packages/ 4. generate package.json for all packages 5. generate keypair (by calling cfx xpi) 6. build xpi 7. hack in the right guid a. unpack fresh xpi b. create new temp zip with content from xpi c. remove xpi d. rename temp zip as xpi the xpi is available to download in using the standard download XPI method I'm not closing the bug as I will need to implement the 1.0b5 features, which will speed up the process - it currently takes 7s to repackage a xpi including download time.
Keypair requirement removed from SDK 1.0b5. repackaging service is now working
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
Comment on attachment 8519064 [details] 1415384496855.jpg zx.1416.zx, please stop messing with flags & attachments on random bugs. If you want to experiment, use http://landfill.bugzilla.org/ , not this production bugzilla instance.
You need to log in before you can comment on or make changes to this bug.