I'd like to increase the memory in the VM running the Wowza Streaming Engine to 16 GB if possible. The machine is wowza1.corpdmz.scl3.mozilla.com.
Let's talk - back in bug 1102493 - we talked about what options there were for increase. Can you provide any documentation as to why 4x the amount of RAM is needed? Current state, it's only using a little over 3G of the 4G assigned to it. Looking over the past little while using SAR, I see that the average memory usage is closer to 90% - and I don't see a concomitant jump in swap usage, so it doesn't seem that the system is desperately hurting for memory starvation. (of course, this could be because it's a very well behaved application) So, Given all that, and barring some obvious issues from the user side of the fence, I think I could see bumping to 8G, but from under the hood here, I can't see problems indicating that 16G are needed. Of course, I'm just looking at the VM, you're the one dealing with the applications, so this is all negotiable. In order to bump the RAM, we'll need an outage of about 5-10 minutes for me to power it down, increase the ram and power it back on.
OK, I'm happy with trying 8G to see if there's a change. The problem I'm chasing is intermittent stalling of HLS output streams. I suspect the problem may have been a bug in the mid-March releases of Firefox Developer Edition, but I did get one report of stalling on Chrome canary. It was watching the memory indicator bounce around 90% that made me think insufficient memory might be a cause. So far we've only been feeding it one input stream. We're going to want to test multiple streams, but I have no real data on what that does to memory requirements. If possible, I'd like to test the larger memory during next week's Monday Project Meeting. Feel free to take the VM down anytime that's convenient, other than during the Project Meeting. This is a test instance of the Streaming Engine and we're not dependent on it for any production service yet.
OK, it's been bumped to 8G. As a note, if the thing were critically needing more memory, I'd expect it to eat into swap more, which makes me think that it's in "use whatever is there" mode - which, if too greedy, isn't ideal for VMs. (VMs are best with applications that use what they need when they need it, and release it when they don't.) In any case, you have the extra RAM, let's keep an eye on things and make sure that suffices. Also, wanted to mention - when I was getting ready to power down, noticed that it doesn't have nagios, which is appropriate for a test/dev box, but monitoring might be a good thing if/when this goes prod.
Assignee: server-ops-virtualization → cknowles
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
On further consultation with team, and re-reading - if we're stalling out on 1 stream at 4G and want to go to multiple streams, then I can totally see how 16G is a place where we could need to be. Bumped it again. Let me know if you have any concerns.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.