Pushing java process onto a different core.

[Djinnii]

Hardware: VPS

OS: CentOS

Okay, so my Host (CreeperHost) has been doing some experimentation with better CPU Load balancing to try and get some better performance out of their virtualisation solutions. And I was recently told about this by them:

taskset -c 1,2 -p PIDofjava

The problem is that most Virtualisation solutions are quite ... "dumb" and invariably the first 2 cores of any system will get hammered by everyone while the rest are barely used.

(I have a VPS, so I currently get a 3 Core allocation out of a 16 core system.) The general idea is to "manually" force the java process to use different core(s) to the OS/other users.

Anyone have any thoughts or suggestions on this?

-- For the uninitiated CPU cores are numbered 0 through 15 (or higher), computer math always starts with a 0, not a 1.

So Core 0 would be the first core, Core 1 would be the second core and so on....

[The_DarthMoogle]

Well, you can set affinity, but then you'd get the problem you outlined. So you could multithread.

But I see two inevitable problems if you get this working.

Disk I/O will slow horribly when you've got threads running in parallel.

And when it's trying to run more trivial tasks, the cost of getting other cores to work at them may actually be negated.

Although this is true of any concurrency. The fact minecraft doesn't even have an actual engine, and just tacks things on to the architecture will just make CreeperHost cry.

I wish I could be of some help, but I do not know how to computer enough.

[MrKekson]

spigot mcpc are multicore optimized a bit, you can try it, if im right atm just the plugins/mods, and later every world will have 4 thread + 1 assync thread, is works like a champ for me with 3.1.2 will try it with 3.1.3 later

[D3matt]

The problem is that most Virtualisation solutions are quite ... "dumb" and invariably the first 2 cores of any system will get hammered by everyone while the rest are barely used

I take it they're not using VMWare, then.

[Djinnii]

spigot mcpc are multicore optimized a bit, you can try it, if im right atm just the plugins/mods, and later every world will have 4 thread + 1 assync thread, is works like a champ for me with 3.1.2 will try it with 3.1.3 later

Don't suppose you've got links? (Unless you mean CB++ build 75, which I'm already using)

I take it they're not using VMWare, then.

No, OpenVZ

[pherce]

I'm thinking your host wants to do that so they can just virtualize more servers on one machine and make more money! If you're allowed to run MCPC (CraftBukkit --) or CB++ (Build 75) then you will see a healthy load balance across all cores on htop, but only if you don't invoke with the taskset. If they push you down that road though (and it would be worse), then at least get them to push MySQL on another thread. Essentially, you want to run those CB forks without taskset or it defeats the purpose of the jar altogether - that is, if you're hosting 50 users or so. You will always be 100% cpu.

[Djinnii]

I'm thinking your host wants to do that so they can just virtualize more servers on one machine and make more money! If you're allowed to run MCPC (CraftBukkit --) or CB++ (Build 75) then you will see a healthy load balance across all cores on htop, but only if you don't invoke with the taskset. If they push you down that road though (and it would be worse), then at least get them to push MySQL on another thread. Essentially, you want to run those CB forks without taskset or it defeats the purpose of the jar altogether - that is, if you're hosting 50 users or so. You will always be 100% cpu.

Thankfully they don't work like that. It's not a shared environment like redstone host or anything like that. It is a VPS and each VPS is "assigned" a specific number of cores, if there aren't enough cores on a given machine at a given location they actually build another machine.

For the most part most VPS' have 3 or 4 cores, on a 16core machine.

[D3matt]

If each host is assigned dedicated cores, what's the problem? Or do they just assign the number of cores and not the actual core affinity? I learned virtualization on Hpyer-V and vSphere, so I kind of just assumed processor affinity was a "standard" feature of server virtualization software.

[Djinnii]

Well from the sounds of what the guy was telling me, just the number of cores, they can't actually assign each VPS a specific set of cores. Hence how this whole thing came about.

[D3matt]

Well then it doesn't really matter what you do in your VM. You can assign it to a certain virtual processor, but you'll still be running on whichever processor the hypervisor shuffles you to. Unless somebody with more experience with OpenVZ can come up with something, their only option is to either just trust the hypervisor to balance the load, or use a better hypervisor.

I'm guessing since they're using a free hypervisor that they're a fairly small company?

[Djinnii]

Why would you say that?

Plenty of OpenSource/Free software is actually better than many of their proprietary counterparts.

Also I'm not so sure they are so small... They have multiple racks in multiple locations round the world, (Mostly Europe and US) They host a fairly large number of MineCraft community projects.

I guess they are small enough that they can still offer great personalized customer service, and haven't been tainted by the attitude "we don't really care about the small fish" But large enough that they can get physcial access to each of their machines within the datacenter.

Since it's now clear there's not much I can do from within the VM, just going to have to wait for them to come to a solution. (The only reason I know about it, is he was working on it.)