Closed Bug 1157578 Opened 9 years ago Closed 9 years ago

create production FxA OAuth client credentials for Pocket

Categories

(Cloud Services :: Operations: Miscellaneous, task, P1)

Product:
Cloud Services
Component:
Operations: Miscellaneous
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rfkelly, Unassigned)

References

Details

(I'm creating this bug for tracking purposes; we're not yet ready to actually provide production credentials to Pocket.)

Prior to Firefox 38.0.5 release, we'll need to provision production oauth client credentials for the pocket integration.  There will be two distinct sets of credentials, one for use in mobile apps and one for the web.  We have the following set up in our development environment:

Web credentials:

  name: "Pocket web dev"
  client_id: "749818d3f2e7857f"
  redirect_uri: "https://firefoxauth.dev.readitlater.com/ff_auth"

Mobile credentials:

  name: "Pocket mobile dev"
  client_id: "7377719276ad44ee"
  redirect_uri: "pocket://oauth/fxa"

As with previous integrations, let's use the same client_id for the production credentials but change the secret etc.  Items we'll need to confirm before provisioning include:

* Final name, redirect_uri, and image_uri for each client
* GPG keys or similar to securely provide client secrets to the pocket devs
ni? :edwong for linking to an appropriate tracking meta-bug
Flags: needinfo?(edwong)
Blocks: Pocket
Flags: needinfo?(edwong)
FYI - I am the contact person on the Pocket side for these credentials.
from IRC - rfkelly to coordinate Oauth creds handoff.  We'd like to get Pocket using the PROD OAuth server ASAP as the longer we wait to switch, the more risk of late issues.  There are two changes requested that will not hit PROD server till 5/11 but they are not core functionality so they shouldn't block the switch over.
Flags: needinfo?(rfkelly)
I've emailed matt requesting his gpg fingerprint for secure transmittal of the credentials when they are created.
For FxA we have three levels of deployment - "dev" which you've already got running, "stage" for testing releases in a very-close-to-production mannger, and obviously "prod" for production.

Matt, do you have a staging environment on the pocket side that would need additional credentials, or are you happy with credentials just for dev and prod environments?

:ckolos, to follow up on your question from IRC, are you suggesting we provision two independent sets of credentials for our prod environment?  Like this:

    pocket dev env     ==>  production FxA stack
    production pocket  ==>  production FxA stack

I'm not sure I see the value in using separate-but-equally-powerful credentials in this case, but could easily be convinced.  Sorry if I misunderstood your question.
Flags: needinfo?(rfkelly)
Flags: needinfo?(matt)
Flags: needinfo?(ckolos)
Ryan -

Currently, just dev and production is sufficient for us. With that said, for load testing purposes it might make sense to have a "staging" set of credentials.

So if it isn't too much work or hassle, would be great to have the three sets of credentials to mirror your setup and I can configure things on our side appropriately.
Flags: needinfo?(matt)
OK, it sounds like we should go ahead with both stage and prod credentials, for both "Pocket web dev" and "Pocket mobile dev" client ids.  So we'll need the following two entries in both stage and prod (sorry if I'm belabouring the details, but over-communication is better than under- at this stage):

  // web credentials
  {
    name: "Pocket",
    client_id: "749818d3f2e7857f",    <-- this will be the same on both stage and prod
    secret: TODO,                     <-- but this will be different, obviously 
    image_uri: "",                    <-- empty for now; in future could link to pocket logo
    redirect_uri: TODO,               <-- Matt to provide required prod web redirect
    canGrant: false,
    whitelisted: false,               <-- this part is super important, but see discussion below...
  }

  // mobile credentials
  {
    name: "Pocket Mobile",               <-- this will appear on our permission screen; text ok?
    client_id: "7377719276ad44ee",
    secret: TODO,
    redirect_uri: "pocket://oauth/fxa",
    image_uri: "",
    canGrant: false,
    whitelisted: false,
  }


There's a slight hitch here w.r.t the "whitelisted" field.  We need this to distinguish between trusted Mozilla-hosted reliers and third-party reliers on the web.  It will trigger (among other things) display of a secondary permission screen and terms-of-service links.

Problem is, it currently doesn't work in prod.  Untrusted reliers currently get a 500 error on all requests.  We're adding this support into our oauth server right now but it's not deployed yet:

  https://github.com/mozilla/fxa-oauth-server/pull/248

We have two ways to proceed:

1)  Provision pocket as whitelisted=true for now, and revisit this when the above hits production.  This would give pocket the ability to silently request additional scopes in our production environment, and would mean that we're not really testing in production configuration.

2)  Wait for the above to deploy before provisioning the credentials.  This introduces delay in integration testing.  It's currently scheduled to be deployed week of May 18, and we'll probably want to bake production URLs into Firefox config before then.

I'm leaning towards (1) because IIRC we don't have any high-value additional scopes that could be requested anyway, and because we actually *do* trust pocket to do the right thing here :-)

Thoughts anyone?

ni? Matt to provide the production redirect_uri
Flags: needinfo?(matt)
Ryan -

(1) redirect_uri

For production use: getpocket.com/ff_auth
For staging use: staging.getpocket.com/ff_auth

NOTE: Staging server is not live yet.

(2) Can I provide you a link to a logo for the image_uri? If yes, any specifics on specs required?

(3) Re: the name for the mobile credentials - any reason we can't just call it Pocket? Does it need to be different from the web-based?
Flags: needinfo?(matt) → needinfo?(rfkelly)
> (3) Re: the name for the mobile credentials - any reason we can't just call it Pocket?
> Does it need to be different from the web-based?

I don't see any reason they can't both be called "Pocket".  Sean, you're the most familiar with the code details there, can you please confirm?
Flags: needinfo?(smcarthur)
> (2) Can I provide you a link to a logo for the image_uri? If yes, any specifics on specs required?

I don't believe we're actually displaying this anywhere yet, but it's here as a placeholder for future use.
Flags: needinfo?(rfkelly)
There's no code restrictions on the name (other than a max size). The name's purpose is to be displayed in the login/permission UI, such as "Sign in to continue to {client.name}".

It's also shown in the oauth-console, but so is the client_id, so it's possible to distinguish via id.
Flags: needinfo?(smcarthur)
I just want to summarize what I'm seeing here so I have a clear understanding.

Credentials for Pocket's stage browser/mobile integrations are needed in our stage fxa-oauth db only.

Stage client ID's are not needed in Mozilla's prod Oauth DB b/c Pocket will not be testing their staging code against our prod db.

Credentials for Pocket's production browser/mobuile integrations are needed.

These production credentials will only be used for the final shipping software/integration.

Do I have that correctly?
Flags: needinfo?(ckolos)
In discussion with :ckolos, he convinced me there's security value in provisioning separate pairs of "production only" and "dev production" credentials.  The former would only ever be used for production pocket deploys to talk to the production fxa stack, never anywhere else.  Which gives us a total of six new credentials pairs to create:


  // web credentials in stage fxa db
  {
    name: "Pocket",
    client_id: "749818d3f2e7857f",
    secret: XXX,
    image_uri: ""
    redirect_uri: "https://staging.getpocket.com/ff_auth",
    canGrant: false,
    whitelisted: true,                   <-- let's push ahead with this for now, change to false later
  }

  // mobile credentials in stage fxa db
  {
    name: "Pocket",
    client_id: "7377719276ad44ee",
    secret: XXX,
    redirect_uri: "pocket://oauth/fxa",
    image_uri: "",
    canGrant: false,
    whitelisted: true,
  }

  // web *dev* credentials in prod fxa db
  {
    name: "Pocket (dev)",
    client_id: "749818d3f2e7857f",
    secret: XXX,
    image_uri: ""
    redirect_uri: "https://firefoxauth.dev.readitlater.com/ff_auth",
    canGrant: false,
    whitelisted: true,
  }

  // mobile *dev* credentials in prod fxa db
  {
    name: "Pocket (dev)",
    client_id: "7377719276ad44ee",
    secret: XXX,
    redirect_uri: "pocket://oauth/fxa",
    image_uri: "",
    canGrant: false,
    whitelisted: true,
  }

  // web *production* credentials in prod fxa db
  {
    name: "Pocket",
    client_id: TODO,                      <-- brand new id for production use only
    secret: XXX,
    image_uri: ""
    redirect_uri: "https://getpocket.com/ff_auth",
    canGrant: false,
    whitelisted: true,
  }

  // mobile *production* credentials in prod fxa db
  {
    name: "Pocket",                       <-- both of these have production name field "Pocket"
    client_id: TODO,
    secret: XXX,
    redirect_uri: "pocket://oauth/fxa",
    image_uri: "",
    canGrant: false,
    whitelisted: true,
  }


This will allow pocket dev => fxa prod testing during initial development, with a secure final cutover to exclusively-prod-use-only credentials for final release.

:ckolos, please sanity-check and if this matches your description, make it so.
Ryan -

Please change "web *dev* credentials in prod fxa db" redirect to:

https://firefox.dev.readitlater.com/ff_auth

?
Thanks Matt, :ckolos please make the above adjustment
Pocket dev for FxA Prod creds sent
pocket Dev browser (to be deleted when Pocket goes live)
    - {
      id: "749818d3f2e7857f",
      name: "Pocket (dev)",
      hashedSecret: "XXX",
      imageUri: "",
      redirectUri: "https://firefox.dev.readitlater.com/ff_auth",
      whitelisted: false,
      canGrant: false
    }

# Pocket Dev mobile (to be deleted when Pocket goes live)
    - {
      id: "7377719276ad44ee",
      name: "Pocket (dev)",
      hashedSecret: "XXX",
      redirectUri: "https://firefox.dev.readitlater.com/ff_auth",
      imageUri: "",
      whitelisted: false,
      canGrant: false
    }
Ryan / Chris -

Apologize if this isn't the right place to ask. 

Next week, we are going to "Pref on the Pocket toolbar button in Nightly".

The week after we are going to uplift to beta.

Which fxa servers should I be hitting for each of these? I just want to make sure I have the right credentials setup.

For the "uplift to beta" - we will want the redirect_uri to be to getpocket.com - there is enough load there that I won't be comfortable having that all go to our dev environment.
Flags: needinfo?(rfkelly)
Flags: needinfo?(ckolos)
(Matt and I chatted about this on IRC; summary below) 

> Next week, we are going to "Pref on the Pocket toolbar button in Nightly".

Since this will be seen by end-users, I think it has to talk to the production FxA endpoint.  Anything else would be too confusing.  So it will depend on whether we want to use readitlater.com or getpocket.com on the pocket side, and that will settle the choice of whether to use the "dev-to-prod" or "production only" credentials.  Matt will follow up on the desired outcome here.

> For the "uplift to beta" - we will want the redirect_uri to be to getpocket.com

In which case, the only worthwhile option is to start using the production-only credentials at this point, with production pocket talking to production fxa.
Flags: needinfo?(rfkelly)
Flags: needinfo?(ckolos)
Ryan - I confirmed this evening that the "Pocket Dev to Prod FxA" credentials I have work properly.

I am waiting on Chris to provide "Pocket Prod to Prod FxA" credentials so haven't tested those yet. I sent him an email.

Based on the successful test to Production FxA, I would like to move forward with using "Pocket Prod to Prod FxA" when we launch "pref on Nightly" next week.
OK, great!  I also spotted this in Bryan's latest status update:

> Wed 6: Pref on the Pocket toolbar button in Nightly
> -- This step requires a Pocket server to replace the current firefox.dev.readitlater.com

Which matches what you suggest above about doing this on production getpocket.com in Nightly.

:ckolos, do you think we'll have FxA train-36 in production before this date?  ISTM we can proceed either way, but worth sanity-checking how the dates line up.
:rfkelly - If I can get a go ahead on train-36 by Tuesday morning, I can have train-36 live by Wednesday.

That said, I need an exact time/date to "cut over" the credentials used in the prod db for "release"; right now, only developer credentials are available in prod.
Flags: needinfo?(rfkelly)
:ckolos :rfkelly My plan is to move the Pocket backend code that supports everything to our production servers on Monday.

I can test from there with the dev credentials. Once I confirm everything is working perhaps we can define a cut over time for Tuesday?
> Once I confirm everything is working perhaps we can define a cut over time for Tuesday?

Yes, let's aim to do this on the Tuesday, so it's in place before things pref on in Nightly on the Wednesday.

:ckolos if I'm interpreting you correctly, does this men the exiting 'dev to prod' credentials will stop working after the cutover?
Flags: needinfo?(rfkelly)
:rfkelly - yes, that's correct; the "Prod-dev" and "Prod-release" creds share the same client ID and they can't co-exist in the database at the same time.
:rfkelly - do we need a client bug to land the prod URL in nightly?
Flags: needinfo?(rfkelly)
:rfkelly :ckolos

How hard is it for you to do the switch over form "Prod-dev" to "Prod-release"? I'm getting ready to release our backend code to getpocket.com and I'm realizing I don't have a getpocket.com -> FxA Dev set of keys. I only have getpocket.com -> FxA Prod - and those aren't live yet as only the firefox.dev -> FxA Prod are live.

Can we:

(a) Do the cut over on the keys now?

(b) Provide me with keys for getpocket.com -> FxA Dev?

After that I can have:

firefox.dev to FxA Dev for testing anything new internally / outside of builds

getpocket.com to FxA Prod for Nightly + Beta + Prod browser builds
Flags: needinfo?(ckolos)
from IRC - mattk is going to move over the PROD OAuth server at 10am PDT.
Update:

- Ryan setup Pocket Prod to Firefox Dev certs for me to use.
- Our backend code has been pushed to getpocket.com and I've confirmed that Firefox Auth is working using the Firefox Dev endpoints.
- Per conversation with Edwin - if you turn on the Pocket Prod to Firefox Prod certificates tomorrow morning, I will then update getpocket.com to use those certs at 10 AM PT.

None of our servers are currently using the Pocket Dev to Firefox Prod certs, so timing isn't critical - just let me know when the switch is made on your end.
Flags: needinfo?(ckolos)
:mattk - will do. Please plan on this being active @10:01 AM PDT.
> do we need a client bug to land the prod URL in nightly?

Per IRL discussion, yes, :edwong to file.
Flags: needinfo?(rfkelly)
As of 10:00 PDT the Production Oauth creds for Pocket are live.
And Pocket servers are correctly pointing to FxA Prod.
I'm going to close this out - we've confirmed the flow is working with prod servers, (though not directly from the panel sign in with firefox button)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Discovered an issue this AM where the mobile credentials were pointing to the wrong redirect_url. They were hitting, getpocket.com/ff_auth instead of pocket://oauth/fxa .

All mobile certificates should always point to: pocket://oauth/fxa 

:ckolos and I discussed in IRC and he is updating.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Priority: -- → P1
:ckolos is it safe to re-close this?
Flags: needinfo?(ckolos)
I think so; I believe mattk has not had any futher issues and his email seems to indicate they are working now.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Flags: needinfo?(ckolos)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.