Issues with HugePages

Discussion in 'Performance Tweaking' started by Kainzo, Jan 2, 2013.

  1. OS: Debian 7
    Java: 7u9
    Ram: 32GB

    Server Start Script:
    The error I'm getting at the moment.
    Anyone have this issue out of box?
    I've have pretty bad luck with HugePages and was wanting to hear from someone a little more experienced what I should be setting up for how many pages to dedicate, etc.
  2. errno = 22 means the user is unable to make use of the HugePages. Odd, I didn't run into this when I was trying it out on a test VPS. In this case, try setting up the user with access.

    Code (Text):

    # This creates the user group "hugepager"
    groupadd hugepager
    # This adds an additional group for the user "minecraft"
    usermod -a -G hugepager minecraft
    # This tells you which group to add to your sysctl
    cat /etc/group | grep hugepager
    Looking at the result of the last one, it will be in form of this:
    Code (Text):
    In my case, the group id is 1008. So then go to /etc/sysctl.d/hugepage.conf, and fill in like this:
    Code (Text):

    kernel.shmmax = 34359738368
    vm.hugetlb_shm_group = 1008
    AnonHugePage support should be available in Wheezy, so you probably won't need the vm.nr_hugepages line. The shm line is always nice to have, and is just the amount of memory you have in bytes.

    Apply the settings by running:
    Code (Text):
    sysctl -p /etc/sysctl.d/hugepage.conf
    I don't think you'd need to reboot your machine after making these change. Try to start spigot again and java shouldn't give you that errno 22 thing again.
  3. andyhuang ... yah
    didn't exist
    only README.sysctl existed.
  4. Yeah, just create the file like what readme says.
  5. I'm out right now, but it should be fairly straight forward. Pls post back if it doesn't work out and I'll try to reply when I get a better keyboard than two thumbs on a phone :)
  6. Yep, created it with
    vm.hugetlb_shm_group = 1002

    I did sysctl -p /etc/sysctl.d/hugepage.conf and attempted to run but it still gave me the error (two this timef or some reason but it was the same as before)
  7. PhanaticD


  8. It definitely supports it - just isnt doing it.... like it should.
  9. Testing this on the test minecraft server. (Same machine different instance) ... Put XMX to 4G now getting this heh

    Welcome to [DH]
    >[email protected]:~/dynmap$ ./
    Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 22).
    Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 22).
    Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 22).
    208 recipes
  10. Same machine different instance? Are you running this in some virtual machine container? If so, you need to make sure the kernel supports it properly, as VM often uses slightly modified kernel. Check the Debian wiki's Hugepages page for more details.
  11. I didnt think I was running this inside of a VM. It's just on my production server, basically out of the box of Debian 7.0 - I'm going to try and move this to the test server (different machine) to see.

    How can I tell if its running in a VM or not?
  12. Well, what did you ordered? If you ordered a dedi, you probably have a dedi, if you ordered a VPS, you probably have a VPS.

    The reason I asked if you're running a VM because you said "different instance" which suggested you have several instances running, and that was a bit confusing :)
  13. andyhuang Oh... I built this machine with my bare hands and its at a collohost. its an i7 3960k with 32gb of ram.
    When I said instance, I meant... different minecraft server processes on the same machine :)
  14. Kainzo So jelly of your CPU :p Or rather, the fact that it uses DDR3 RAM. My server is dual Xeon E5450 (also own hardware), so it uses DDR2 RAM, and 8GB DDR2 sticks are stupidly expensive so I'm stuck at 8x4GB for a while (even though the machine can handle 8x8GB instead).

    If you're not using a VM, then I really don't know why it is not working without the pre-allocation -sadface- I guess you will still need to set the "vm.nr_hugepages = ####" line to your /etc/sysctl.d/hugepage.conf. To get the value to fill there, do:
    Code (Text):

    grep Hugepagesize /proc/meminfo
    It will output something like: "Hugepagesize: 2048 kB"
    Take desired GB and divide by that, and fill there. So for example I want 24GB, I do 24GB * 1024MB/GB * 1024KB/MB / 2048KB/Page = 12288 Pages, and fill in 12288 like so:
    Code (Text):

    vm.nr_hugepages = 12288
    As previously noted, though, this will make the kernel pre-allocate the memory at boot (or after you do sysctl -p /etc/sysctl.d/hugepage.conf. So the resource will appear "used" even if your server is not running. If you don't want this anymore, either set it to 0, and do sysctl -p /etc/sysctl.d/hugepage.conf again. Theoretically, you do not need to reboot, but you want to make sure your minecraft server is not running when you set it back to 0, else it might lose information that's in there (I have no idea what would happen at this point, actually).
  15. andyhuang - yah I was hoping to not have to go through this approach to get this effect. My question is... do I really need to dedicate 24GB ? if the server happens to go over the limit of say 16GB - would it start using the normal page method?
  16. Kainzo If you don't want to put aside 24GB (rightfully so), you don't have to. In fact, you only need to have a little bit more than your Xmx value so you don't run into errno 28 (out of space). I suspect if your Xmx is larger, it just won't use of it at all, and just proceed per normal, but I don't really know for fact.
    • Optimistic Optimistic x 1