1248762 - Define infrastructure

Status: RESOLVED FIXED

(Release Engineering Graveyard :: Applications: Balrog (backend), defect)

Description

[Benson Wong [:mostlygeek]]

Infrastructure should include: 

- diagram of components (ELBs, ASGs, one off servers, RDS, etc)
  - includes traffic flow
- sizing of EC2 servers for web nodes
- sizing of RDS database
- should we have a read-only second region?
- security around assets (security groups, vpn, etc)

Comment 1

[[e-mail] (:bhearsum)]

(In reply to Benson Wong [:mostlygeek] from comment #0)
> Infrastructure should include: 
> 
> - diagram of components (ELBs, ASGs, one off servers, RDS, etc)
>   - includes traffic flow

Is this something for you to do, or for me?

> - should we have a read-only second region?

It would be wonderful to have a second region for the public facing app (aus[2-5].mozilla.org). Multihoming has been on our want list for a long time. There's probably no point in multihoming the admin app unless it's possible to do failover rw as the vast majority of requests to it are submitting new data, which would fail if it were in ro mode. We can survive short outages of the admin app pretty easily anyways.

Comment 2

[Benson Wong [:mostlygeek]]

I started documenting things here: 

https://mana.mozilla.org/wiki/display/SVCOPS/Firefox+Update+Service

Very standard deployment. 
Only notable is that public web servers only have READONLY access to the database with WRITE access only available on the ADMIN server.

Comment 3

[Benson Wong [:mostlygeek]]

After some thought the read-only second region doesn't add any value. The service is behind the scenes and both latency and unavailability does not affect a user's browsing experience.

Comment 4

[Julien Vehent]

There may be a situation where all slaves are up to date, master is down, and 90% of Firefox users still have to pull down their updates (I believe updating everyone takes a few days).

Comment 5

[Benson Wong [:mostlygeek]]

A new bug will be created to support multi-region reads.