Closed Bug 721743 Opened 12 years ago Closed 12 years ago

Request Linux VM with access to breakpad symbol mount

Categories

(Infrastructure & Operations :: Virtualization, task)

Product:
Infrastructure & Operations
Component:
Virtualization
Platform:
x86_64
All
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: benjamin, Assigned: dustin)

References

Details

I am working on a profiling project which will use the breakpad symbols to do local profiling of nightly profiling builds. This is described at the feature page https://wiki.mozilla.org/Snappy_Symbolication_Server

For the first phase of this project I would like a Linux VM with the following characteristics:

* access to the breakpad symbol data readonly (currently available at dm-symbolpush01:/mnt/netapp/breakpad
* either a modern version of node.js or python 2.6+ so that I can compile node myself
* ability to run a webservice based on node.js

Currently this can live behind firewalls: eventually when development is complete we'd like to expose a public webservice, but don't need that for the initial development phase.

Assuming that setting up a new development VM is not a big project, I'd like to have this early next week. If this is a bigger project for some reason please let me know and we may try to set up local development environments and deal remotely with symbols for now.
What are the specs for this VM? RAM/disk/preferred OS?
Other than a standard OS and the breakpad mount and maybe some logging, it doesn't need any significant storage. Benwa has apparently already written a version of this code in Java, so I'll let him comment on RAM. Linux, definitely Linux.
The memory requirement is ~50MB per builds' symbol we want to cache. More is needed as well if we want to cache symbols for system libraries. I don't have access to them so I can't estimate.

We could get by with less memory but we're going to require a lot of IO to process a request (20 MB disk read per requests).

Ideally 1 GB of RAM would let us cache several recent builds.
Here is the code we're hoping to deploy. It not finished ATM, once we know more about the environment we can make the correct optimization better:
https://github.com/bgirard/ProfilerSymbolServer
Is it possible to get an ETA for this? I'd like to know if I should use an external VPS while waiting for this to be setup.
Depends on: 688186
Triage : Moving to the right component. Dan, you can work on this once the dependant bug is FIXED. Thanks!
Component: Server Operations → Server Operations: Virtualization
QA Contact: cshields → dparsons
Actually, this isn't dependent - symbols are in phx1 already (10.8.75.254:/vol/pio_symbols).
No longer depends on: 688186
(1) What should the fqdn of this VM be?

(2) 20GB HD, 1GB RAM, 1 CPU is good enough?

(3) Which usernames will need ssh / sudo access to this VM?
1) You mean the internal one, or the eventual public one? The public one I think should eventually be

symbolapi.mozilla.org

I don't think we care at all what the internal FQDN is.

2) yes

3) myself, benwa, ehsan for now
Agreed. If RAM is cheap more would be great, but 1GB will be sufficient.
The internal one. And we do care very much, especially me, because I still have very little understanding of what people or VMs belong to which Mozilla group. I barely even know what breakpad is or what group it's under. So any clues you can give me along those lines will help me give the VM a proper name. It seems kind of silly but it's actually very important, because (a) I can't create the VM without a name; (b) we need to name it properly so we know how to manage it in the future.

:dustin, any suggestions on a name given the above?
breakpad-symbolapi1

I think this should go in the DMZ as well.

Breakpad is part of socorro, fwiw.

And yes, this is a general problem, as my stall-mucking in sjc1 is demonstrating.  I'm not sure what a good solution is.
OK, so this is what I plan to make. Please, everyone review the specs and tell me if you see anything you want changed:

Hostname: breakpad-symbolapi1.dmz.phx1.mozilla.com
Specs: 2GB RAM, 20GB HD, 1 CPU

:bsmedberg, please translate this string "myself, benwa, ehsan for now" into a list of valid LDAP usernames :/
Okay, so the username I have so far is "bgirard", how about "myself" and "ehsan"?

I can't build this until I have all this information :{
Could you add me too? 

mail=vdjeric@mozilla.com,o=com,dc=mozilla
Usernames with ssh / sudo access: bgirard, eakhgari, bsmedberg, vdjeric. I'll get building now.
:BenWa, I need your SSH public key in order to get you access to this VM. Can you paste it here?
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA1UQSfiFafyB/WwGMKufVEP5LwzMlEpsZfvM2Y8bXwu41+IWMecpCKNYlJpCYM9WP6nl8I7OBIp2kF1hB1PmPX57BX/feRWCUBLQ0mtmxeDaoVv30cH/aDLYvLUy5PpBCLEJvBvasiBHqxj+Ma466VPJHKhyZaHrXElkLF6bmO1+CfA4/cjhbhMvIQ4afY+b9BtOdUYm1h+kZ7lvMFEJQw5CbO+4Wps7uv3/+g+ZWV+TkOTncMP7FDAIT1azPFpAVQbaXbNEWTUgYRIBYFN4/TvhnVvaMeOjQvEtC32peya+D/uaeY4idBJg58zRwXVmX39P3vOHJaEMpO786IMI4uQ== mozilla@wavicle-2.local
OK, I believe this is done. Please test it ASAP and let me know. Try to ssh in, try to use sudo, etc.

hostname:
breakpad-symbolapi1.dmz.phx1.mozilla.org
I filed bug 728239 to make this host accessible from mpt-vpn so that we can access it.
Depends on: 728239
Assignee: server-ops → dparsons
I can access the machine and sudo (installed Java), but I don't see the breakpad symbol mount.
as you probably noticed, this is breakpad-symbolapi1.dmz.phx1.mozilla.com :)

I'll fix up the mountpoint in a moment.
Assignee: dparsons → dustin
I'm unable to connect to the machine:
ssh: connect to host breakpad-symbolapi1.dmz.phx1.mozilla.com port 22: Connection refused
I'm trying to connect from the mozilla toronto office network. Do I need to tunnel via mpt-vpn?
:BenWa - you need to use the MPT VPN for this (even though it's in Phoenix..).

:bsmedberg - the mountpoint is added in puppet, but it doesn't look like the netapp perms are set up yet.  Dan?  Jake?
Dan fixed the mountpoint up, and it's mounted now in the usual location.  You should be good to go!
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
What is the publicly accessible IP or domain name of this host? I can only connect to it on port 80 and only from MPT VPN
We are not making a public DNS or firewall entry until the code and testing are complete and security-reviewed.
Blocks: 753588
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.