Need a new home for brasstacks results Db and webserver

RESOLVED WORKSFORME

Status

mozilla.org Graveyard
Server Operations
RESOLVED WORKSFORME
8 years ago
3 years ago

People

(Reporter: cmtalbert, Assigned: fox2mike)

Tracking

Details

(Reporter)

Description

8 years ago
Our current sm-brasstacks VM is running beyond capacity. It doesn't look like the ESXi machine is overloaded per se, but the experience of the users of the web tools and databases housed on that machine are reporting serious problems getting to the data.

I think it may be time to split this guy off and make him a real machine.  This machine (and the tools on it) are becoming our results reporting server for many of the tools that we are developing and completing.  Currently it houses:
* Grafx Bot test results
* Topfails web tool
* Automated Mozmill QA Results

We are expanding each of these tools as well as creating new ones which will be talking to the systems on this machine.  We need this machine to be able to scale better to meet the needs of a web/database server.

It should be:
* 64 bit Linux 
* Quad core
* 4Gb RAM
* 1T disk

It should still be available publicly as brasstacks.mozilla.com.

We're looking for IT help here on these points:
* Should it be a VM or a stand-alone box?  It doesn't seem like it'd have to be an amazing box, but it definitely needs much more resources than it currently has.
* What OS should it use?  I'm doing some experiments with my own Vms trying to answer that question myself: I'm currently looking at Ubunutu Server 64 bit 9.10 and RHEL 64 bit, 5.3.  But, I'd cede to your expertise on this point.
(In reply to comment #0)
> machine (and the tools on it) are becoming our results reporting server for
> many of the tools that we are developing and completing.  Currently it houses:
> * Automated Mozmill QA Results

Regarding the Mozmill tests I have to add that we are creating a cloud testing extension right now, which will allow any user to run Mozmill tests by a single click and it will send the report to brasstacks too. That means we will have to cover a lot more results per day as we currently do. Further we also plan to extend our in-house Mozmill tests by running tests against add-ons.

Updated

8 years ago
Assignee: server-ops → shyam
(Assignee)

Comment 2

8 years ago
(In reply to comment #0)
> We're looking for IT help here on these points:
> * Should it be a VM or a stand-alone box?  It doesn't seem like it'd have to be
> an amazing box, but it definitely needs much more resources than it currently
> has.

We could try bumping up the resources on the current VM and see if that helps? If it doesn't, I guess we can be sure we'd have to go to hardware. 

> * What OS should it use?  I'm doing some experiments with my own Vms trying to
> answer that question myself: I'm currently looking at Ubunutu Server 64 bit
> 9.10 and RHEL 64 bit, 5.3.  But, I'd cede to your expertise on this point.

IT usually goes with RHEL 5.x but we just got our hands on 6 beta ;) That'll be interesting too.
(In reply to comment #2)
> We could try bumping up the resources on the current VM and see if that helps?
> If it doesn't, I guess we can be sure we'd have to go to hardware. 

For Mozmill we will have a lot more traffic soon. I would feel better when we could get our own dedicated machine and can also get rid of CentOS in the same step.
(Assignee)

Comment 4

8 years ago
mrz,

Can I bring this up in PHX? I see a bunch of spare machines on thermo-b10 with 6 and 12 GB of RAM, could put one of the sixers on this?

Comment 5

8 years ago
Depends what sort of access this needs.  There isn't any user VPN in Phoenix and no QA environment there.  I'd lean towards SJC.
(Reporter)

Comment 6

8 years ago
(In reply to comment #5)
> Depends what sort of access this needs.  There isn't any user VPN in Phoenix
> and no QA environment there.  I'd lean towards SJC.
As we're still developing many of the applications on this box, I'd prefer to keep it in SJC in order to maintain our ability to VPN/ssh into it.  I used VIC to bump the ram up to 2Gb (was 768Mb), so that does seem to helping a bit.  I didn't go too much higher because I'm not sure how much I can safely overload qm-vmware02's memory before it becomes a problem with the other machines.

Comment 7

8 years ago
> I
> didn't go too much higher because I'm not sure how much I can safely overload
> qm-vmware02's memory before it becomes a problem with the other machines.

IIRC, you can over subscribe memory.  If it becomes an issue, we'll add memory to that  ESX host.
(Assignee)

Comment 8

8 years ago
Clint, have you tried boosted RAM further? Still need physical hardware for this?
(Reporter)

Comment 9

8 years ago
(In reply to comment #8)
> Clint, have you tried boosted RAM further? Still need physical hardware for
> this?
Sorry, forgot to reply to mrz.  I boosted the memory and it is running ok.  Diskspace use will be an issue eventually, but I think with us moving the sm-purify box to physical hardware, we can increase the diskspace here as needed so we should be ok.

I'll close this as wfm now.  Thanks for following up w/me.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → WORKSFORME
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.