1140889 - Decide on an HTTPS hosting solution for mcMerge

Status: RESOLVED FIXED

(Tree Management :: Bugherder, defect)

Description

[Ed Morley [:emorley]]

mcMerge (a client-side tool that uses Bugzilla's REST API to make update bugs after a merge from one repository to another; used by sheriffs and others) is currently in the TBPL repo and hosted with TBPL.

TBPL is due for EOL imminently (ideally end of month), so I need to move mcMerge somewhere else. It's a 100% static app - I just need HTTPS so people cannot MITM the JS libs and thereby steal people's Bugzilla credentials.

I have the (currently empty) github repo set up - I just need some hosting for it, and I'll do the rest over in bug 1050477.

Ideally the deployment would just pull from the Github repo on a cron, so I don't have to manually deploy. (I'd use Github pages for this to achieve the same result with no additional hosting, but there's no way to force HTTPS).

Many thanks :-)

Comment 1

[Ed Morley [:emorley]]

Current code location: https://hg.mozilla.org/webtools/tbpl/file/default/mcmerge
New code location (Note: I'll do the conversion/move): https://github.com/mozilla/mcMerge

Current webapp location: https://tbpl.mozilla.org/mcmerge/

Comment 2

[Ed Morley [:emorley]]

Actually, would https://wiki.mozilla.org/Paas_Apps / https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453 be an acceptable solution for this?

Comment 3

[Shyam Mani [:fox2mike]]

(In reply to Ed Morley [:edmorley] from comment #2)
> Actually, would https://wiki.mozilla.org/Paas_Apps /
> https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453 be an
> acceptable solution for this?

That would probably be best.

Comment 4

[Ed Morley [:emorley]]

I've read through https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453 and also http://docs.stackato.com/user/quick-start/index.html however the more I read the more questions I'm finding than answers.

In no particular order:
* Many of the docs.stackato.com references on the mana page above 404.
* Should I be using paas.allizom.org or pass.mozilla.org? I'm presuming the latter since we need to serve over HTTPS and presumably shouldn't use the wildcard cert of allizom?
* How do I go about just deploying static pages? The filesystem service? Do I still need a manifest? The list of example Mozilla apps (https://wiki.mozilla.org/Paas_Apps) includes one that is supposedly static content only... so I thought I could steal their manifest.. except they actually run node after all.
* The docs at http://docs.stackato.com/user/quick-start/index.html mention org and space names. Are we using those? What should I set them to?
* How do I get my repo to auto deploy?

All I want to do is:
* Deploy a static site to eg https://mcmerge.pass.mozilla.org from a github repo
* Have that site update automatically as people commit to the repo.

Thanks :-)

Comment 5

[Ed Morley [:emorley]]

Also https://mail.mozilla.org/pipermail/paas-users/2014-April/000242.html hints that I should be creating apps under a group, not my own user. Is this correct?

The docs on the Mana page don't mention this at all.

Comment 6

[Shyam Mani [:fox2mike]]

I'm going to let gozer help out with those questions :)

Comment 7

[Shyam Mani [:fox2mike]]

(In reply to Ed Morley [:edmorley] from comment #4)
> * Many of the docs.stackato.com references on the mana page above 404.

Try http://docs.stackato.com/user/deploy/index.html#stackato-push 

> * Should I be using paas.allizom.org or pass.mozilla.org? I'm presuming the
> latter since we need to serve over HTTPS and presumably shouldn't use the
> wildcard cert of allizom?

Ideally, you should try allizom first before going to production mozilla.org

> * How do I go about just deploying static pages? The filesystem service? Do
> I still need a manifest? The list of example Mozilla apps
> (https://wiki.mozilla.org/Paas_Apps) includes one that is supposedly static
> content only... so I thought I could steal their manifest.. except they
> actually run node after all.

Not really sure here, :gozer might have a better answer. 

> * The docs at http://docs.stackato.com/user/quick-start/index.html mention
> org and space names. Are we using those? What should I set them to?
I believe setting these up, will take care of all that :

https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-ConfiguretheClient

> * How do I get my repo to auto deploy?

Another one I'll leave for :gozer

Comment 8

[Ed Morley [:emorley]]

(In reply to Shyam Mani [:fox2mike] from comment #7)
> > * How do I go about just deploying static pages? The filesystem service? Do
> > I still need a manifest? The list of example Mozilla apps
> > (https://wiki.mozilla.org/Paas_Apps) includes one that is supposedly static
> > content only... so I thought I could steal their manifest.. except they
> > actually run node after all.
> 
> Not really sure here, :gozer might have a better answer. 

Thank you :-)

Comment 9

[Ed Morley [:emorley]]

gozer, ping?

To summarise the remaining questions:
1) mcMerge is just static content only; what manifest settings do I use for that? The Stackato & mana docs really aren't clear on this. They only have examples for more complicated stacks. eg http://docs.stackato.com/user/deploy/buildpack.html 
2) How do I set it up so that it will pull from a github repo on a cron, so it will auto-update?
3) How do I disable HTTP access (we only want it available over HTTPS)? Do I have to just use an .htaccess or can I get Stackato to deny access to port 80?
4) Given it's static content only, and the project is already used (and so not brand new and untested), do I really need an allizom instance as well as a mozilla one? We won't be using staging at all once it's deployed.
5) The mana page links to the official client download page (http://www.activestate.com/stackato/download_client). However they only serve the client over HTTP and don't have any hashes available for verification. Undocumented is the download link from the Mozilla Stackato instance (https://api.paas.mozilla.org/console/client/) however that is a really old version of the client. So either I risk a MITM or get an old client. Could you suggest where I could download it over HTTPS? :-(

Since writing the above, I have just found https://github.com/ActiveState/staticfile-buildpack , which isn't documented anywhere. It also doesn't support forcing HTTPS, though the repo it forked from has just added support yesterday amusingly. As I understand it, I can either reference those buildpacks from my manifest, or else ask you to upload it to the Mozilla Stackato instance & then the auto-detect will do it's thing? (ie the readme for that buildpack says I just add a file named "Staticfile" to the app root).

Eugh, I just want this to be hosted somewhere so I can forget about it and move onto something else! :-)

Comment 10

[Ed Morley [:emorley]]

Oh and:

(In reply to Ed Morley [:edmorley] from comment #5)
> Also https://mail.mozilla.org/pipermail/paas-users/2014-April/000242.html
> hints that I should be creating apps under a group, not my own user. Is this
> correct?
> 
> The docs on the Mana page don't mention this at all.

Comment 11

[Philippe M. Chiasson (:gozer)]

(In reply to Ed Morley [:edmorley] from comment #9)
> gozer, ping?
> 
> To summarise the remaining questions:
> 1) mcMerge is just static content only; what manifest settings do I use for
> that? The Stackato & mana docs really aren't clear on this. They only have
> examples for more complicated stacks. eg
> http://docs.stackato.com/user/deploy/buildpack.html 

What I've done in the past is just pick PHP as the framework and push a bunch
of static files. After all, the PHP stuff is Apache under the hood, so it serves
static files just fine.

> 2) How do I set it up so that it will pull from a github repo on a cron, so
> it will auto-update?

That's a bit trickier, since updating applications from *inside* their stackato
deployment breaks the auto-scaling model and all.

What I would look into is to hook something external to stackato that updates
from github and pushes to stackato if changes were pulled in.

> 3) How do I disable HTTP access (we only want it available over HTTPS)? Do I
> have to just use an .htaccess or can I get Stackato to deny access to port
> 80?

Not quite possible from Stackato's POV. but once your application is all good and deployed,
we will need to make a final production DNS entry for it, and there, you can request SSL-only.

> 4) Given it's static content only, and the project is already used (and so
> not brand new and untested), do I really need an allizom instance as well as
> a mozilla one? We won't be using staging at all once it's deployed.

That's entirely up to you, actually. You can just create a single paas.mozilla.org instance
if you want.

> 5) The mana page links to the official client download page
> (http://www.activestate.com/stackato/download_client). However they only
> serve the client over HTTP and don't have any hashes available for
> verification.


> Undocumented is the download link from the Mozilla Stackato
> instance (https://api.paas.mozilla.org/console/client/) however that is a
> really old version of the client. So either I risk a MITM or get an old
> client. Could you suggest where I could download it over HTTPS? :-(

If you loook here:

http://downloads.activestate.com/stackato/client/v3.2.0/

you'll find SHA1 checksums for the clients.

But for what it's worth, you are better off using the client that matches the version
we have deployed.


> Since writing the above, I have just found
> https://github.com/ActiveState/staticfile-buildpack , which isn't documented
> anywhere. It also doesn't support forcing HTTPS, though the repo it forked
> from has just added support yesterday amusingly. As I understand it, I can
> either reference those buildpacks from my manifest, or else ask you to
> upload it to the Mozilla Stackato instance & then the auto-detect will do
> it's thing? (ie the readme for that buildpack says I just add a file named
> "Staticfile" to the app root).

Not going to happen, our current version of Stackato is too old for this. We have
an upgrade path planned, but it's not going to be anytime soon.

Comment 12

[Philippe M. Chiasson (:gozer)]

(In reply to Ed Morley [:edmorley] from comment #5)
> Also https://mail.mozilla.org/pipermail/paas-users/2014-April/000242.html
> hints that I should be creating apps under a group, not my own user. Is this
> correct?

If you create the app as yourself, you'll be the only person ever able to push updates to it.

If you create a group first, and deploy under that group, then you can add folks to the group
and they will be able to push updates as well.

Comment 13

[Ed Morley [:emorley]]

(In reply to Philippe M. Chiasson (:gozer) from comment #11)
> > 2) How do I set it up so that it will pull from a github repo on a cron, so
> > it will auto-update?
> 
> That's a bit trickier, since updating applications from *inside* their
> stackato
> deployment breaks the auto-scaling model and all.
> 
> What I would look into is to hook something external to stackato that updates
> from github and pushes to stackato if changes were pulled in.

This would be a blocker for me; I don't want to have to set up something external as well and have credentials say on people.m.o (at that point I might as well just host this from people.m.o).

However I believe I have found a solution:
* Push to stackato from a directory that is otherwise empty apart from the manifest (ie no mcMerge assets).
* Add a pre-staging hook to the manifest that clones the mcmerge repo.
* Add a cron entry that git pulls that repo.

That way:
a) There are no stale files in the actual app I push to stackato.
b) If stackato kills/recreates/scales the app, it re-clones the repo from fresh.
c) When the app is running, it auto updates thanks to the cron.

Sound ok?

Comment 14

[Philippe M. Chiasson (:gozer)]

I didn't think of that. But it should indeed work just fine!

Comment 15

[Ed Morley [:emorley]]

Great, thank you - looks like we have a winner :-)

Comment 16

[Ed Morley [:emorley]]

(In reply to Philippe M. Chiasson (:gozer) from comment #11)
> > Since writing the above, I have just found
> > https://github.com/ActiveState/staticfile-buildpack , which isn't documented
> > anywhere. It also doesn't support forcing HTTPS, though the repo it forked
> > from has just added support yesterday amusingly. As I understand it, I can
> > either reference those buildpacks from my manifest, or else ask you to
> > upload it to the Mozilla Stackato instance & then the auto-detect will do
> > it's thing? (ie the readme for that buildpack says I just add a file named
> > "Staticfile" to the app root).
> 
> Not going to happen, our current version of Stackato is too old for this. We
> have
> an upgrade path planned, but it's not going to be anytime soon.

I unfortunately read this as "unable to upload custom buildpack" rather than "buildpacks not supported at all".

In fact many of the options specified on the docs pages linked both in this bug and Mana don't even exist for the version of Stackato deployed at Mozilla.

Please can someone make this clearer on our Mana page? ie: link to our own v2 instances' docs at https://api.paas.mozilla.org/docs/index.html - rather than the v3 on stackato.com?

Comment 17

[Ed Morley [:emorley]]

This also explains:

(In reply to Ed Morley [:edmorley] from comment #4)
> * The docs at http://docs.stackato.com/user/quick-start/index.html mention
> org and space names. Are we using those? What should I set them to?

Stackato v2 didn't have this concept.

This is pretty frustrating; so much of the back and forth could have been avoided here were the docs clearer :-(

Comment 18

[Shyam Mani [:fox2mike]]

If you have recommendations on the doc changes, I'm happy to make them for you. Sorry, the only person who managed this full time isn't here anymore and we're all trying to fill in :)

Comment 19

[Ed Morley [:emorley]]

(In reply to Shyam Mani [:fox2mike] from comment #18)
> If you have recommendations on the doc changes, I'm happy to make them for
> you. Sorry, the only person who managed this full time isn't here anymore
> and we're all trying to fill in :)

Filed bug 1183414 for me to make some changes.