Why Monster hardware lags @ 50 players like hell

Discussion in 'Performance Tweaking' started by UltraMC, Mar 2, 2013.

  1. Why my server lags? Please, help me.
    unsolved in half
    Sympthoms:
    When more than 30-40 players are online they all experience `common lag` which is: pvp is impossible due other players movement is `slideshow` instead of regular motion. Block when dig tend to go back to their place after a while. Teleportation commands flash player to destination and one is rapidly backed to his oryginal location getting `success` status for his teleportation. When players leave and only a few is online after a while this lag dissapears. Manual reset helps as well for a short time.

    What I did:
    - tried several startup scripts (if you want to see them, tell me)
    - tried several servers (Kimsufi, Sprit Data Center (as deamon) now Hetzner)
    - replaced java open JDK with Oracle's 7
    - installed SSD
    - tried RAMDISK (huge fail, no change to SSD really, when it got full - disaster, so I stick with SSD)
    - hopped from Bukkit to Spigot ;-)
    - trying recomended or newest Spigot versions (same before with Bukkit)
    - changed several plugins with their replacements
    - posted on http://www.minecraftforum.net/topic/1634331-monster-hardware-lags-50-players/ (solved tag means I require any more assistance, I hoped I will not need it, as I do now)

    Server IP:
    5.9.154.116 (http://www.hetzner.de/en/hosting/produkte_rootserver/ex10)

    server.log http://hastebin.com/gigepinaya.vhdl

    bukkit.yml http://hastebin.com/moyadomeji.vhdl

    Timings: (to be viewed by http://aikar.co/timings.php)
    Most recent http://hastebin.com/goqirapepi.vhdl
    2 months ago http://hastebin.com/yerogesovo.vhdl

    Server spec:
    Intel Core i7-3930K Hexacore incl. Hyper-Threading Technology
    64GB DDR3 RAM
    120GB SSD
    1Gbit connected at 100Mbit
    Unlimited traffic

    Tops and dpkg -l
    http://5.9.154.116/tops_and_dpkg/

    uname -a
    Code (Text):
    Linux Debian-60-squeeze-64-minimal 2.6.32-5-amd64 #1 SMP Sun Sep 23 10:07:46 UTC 2012 x86_64 GNU/Linux
    java -version
    Code (Text):
    java version "1.7.0_10"
    Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
    Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)
    Startup line:
    Code (Text):
    java -server -Xincgc -Xmx40900M -XXarallelGCThreads=12 -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -jar spigot.jar nogui
    OS and MySQL DB are on the 2 x HDD 3 TB SATA 6GB/s 7200 rpm (Software RAID 1)
    Minecraft server is on 1 x SDD 120GB

    MySQL DB plugins:
    - AuthMe (passwords),
    - PreciousStones (cuboids),
    - BanReport (bans)
    - SMSshop & SimpleWebAuctions - my plugins for give/take with MySQL support

    Plugins (26):
    lshw
    Code (Text):
    *-disk:2
    description: EXT3 volume
    product: INTEL SSDSC2CT12
    vendor: Linux
    physical id: 0.0.0
    bus info: [email protected]:0.0.0
    logical name: /dev/sdc
    logical name: /home/ultra/ssd
    version: 1.0
    serial: 5600ae9a-63d5-41ac-b582-d08a5a42e287
    size: 111GiB
    capabilities: journaled extended_attributes large_files recover ext3 ext2 initialized
    configuration: ansiversion=5 created=2013-01-06 11:23:42 filesystem=ext3 label=/myssd modified=2013-02-25 18:38:47 mount.fstype=ext3 mount.options=rw,relatime,errors=continue,data=ordered mounted=2013-02-25 18:38:47 state=mounted
     

    Attached Files:

    #1 UltraMC, Mar 2, 2013
    Last edited: Mar 3, 2013
  2. YoFuzzy3

    Supporter

    Just going to throw this out to you, but it may not be the problem.

    You're running a 6core/12thread CPU. Minecraft isn't multi-threaded and prefers to suck all the power of one main core/thread. This means that if you don't have a high power to core ratio you'll get bad TPS.
     
  3. What is and how to check my
     
  4. YoFuzzy3

    Supporter

    It's the amount of voltage allocated to each core/thread. I can't really help you with anything further than that unfortunately.
     
  5. Even I I could directly increase voltage in 1 core, how I would know which one to set as Miencraft picks random one. I should than increase V on every core and that in my pure electronics opinion isn't save - so machine replacement needed. Correct me if wrong and to your tip be considered further provide more information or let someone else point it out.
     
  6. Bestle

    Patron

    Your CPU should easily handle that amount of players - That isn't the issue here.

    You have some plugins doing some fairly hefty tasks on player login (such as authme).
     
  7. Bestle thats the only plugin I havn't disabled for testing as it required for login purposes. Most of people us it and only I have that kind of problem. Got any replacement?

    Any other plugins that are crazy? I dont really know how to read this timings.
     
  8. SpaZMonKeY777

    Wiki Team

    When looking at your timings report, I see 2 things taking a hefty load. Entity ticks, and PreciousStones. One in specific is Dynmap-PreciousStones. I recommend disabling that or dynmap all together for a bit and see if the lag lessens on that front.

    And can you post your bukkit.yml? Reducing your entity-activation-range really low will help. My community likes massive animal farms, and I reducded animal activation to 5 and monsters to 10 and it works great! Post your bukkit.yml though and we can look more into it.
     
  9. SpaZMonKeY777

    Wiki Team

    Well I reduced the 'Entity-Activation-range-animals" to 5 because that was the bulk of entities on our server. Dozens of our players have sheep farms with 500+ sheep, and with the server ticking each one, especially when multiple of them were at their farms, it was causing a large tick load.

    I have had it set to 5 for a month now, and nobody has even noticed lol. I moved monsters to 15 and even that could go as low as 10. Misc can be like 10, and that will make a drastic change. Just give it a whirl and see where your TPS sits with 40-50+ online. If you can, when it's lagging, take screenshots of your in game TickRate/MemUsage/Entities/Etc.

    ALSO: When your servers getting lag, type '/worldguard report' and paste the report for us as well. It makes the report in your worldguard directory in your plugins folder.
     
  10. Um... How to do that? :p (not screes obv.)
     
  11. WorldGuard: http://hastebin.com/kimobuhata.vala
    I've did /worldguard reload x3 times, than /worldguard report (as you not stated this, had to Google it)

    Memory etc: in the attachment

    Timings: in the attachment - 1st timing is from server start till lag, 2nd is reseted from lag for 30s.
     

    Attached Files:

  12. SpaZMonKeY777

    Wiki Team

    Sorry I meant report lol. Was leaving for work so I was typing fast. Looking at your worldguard report, at the time of that report you had 1700ish chickens loaded and over 1000 sheep alone. Tells me you have players with large farms. Edit your bukkit.yml so the entity-activation-range-animals is set to 5, and monsters to 15. Upon restart, test your settings by finding some animals and see if they move very much from far away, and when you get within 5 blocks, they should start ticking normally.

    With all that set, see how your TPS is throughout.
     
  13. I did! I have changed values in bukkit, double checked it, restarted server and reported again: 2000 sheeeps! 1000 Chickens!

    Now the question is how to cut now and limit those numbers in future?

    I am wondering if that may be the main issue...
     
  14. http://dev.bukkit.org/server-mods/chunkspawnerlimiter/
     
  15. #16 UltraMC, Mar 2, 2013
    Last edited: Mar 2, 2013
  16. SpaZMonKeY777

    Wiki Team

    This will help with natural spawns, but not when people are breeding large amounts of animals. I tried it a while ago, but dumped it.

    Let me give my stats: I run a 20k x 20k square world, our server caps at 80 players and is full almost all day everyday. We average 8k-10k entities when the servers full, and used to get low TPS and my timings reports were showing really high EntityTick. Once I got my hands on the entity-activation commit, it pulled us to 20 TPS all day, maybe dropping to 17-18 once in a while, but it jumps back to 20.

    Please try changing those activation ranges and see if it improves. Then let us know. If it doesn't change, then we can move from there.
     
  17. I've already did, no effect:
    Code (Text):
        entity-activation-range-animals: 5
        entity-activation-range-monsters: 15
        entity-activation-range-misc: 10
     
  18. SpaZMonKeY777

    Wiki Team

    Well the only plugins I saw giving bad timing reports were the dynmap-PreciousStones and PreciousStone itself. I would disable the dynmap addon first and see if anything improves. If nothing improves, then your only best option will be to try disabling plugins one-by-one, and it won't be easy. Start with plugins you can live without for a period of time, and see if anything improves then.

    It's a pain and time consuming, but eventually you will find the solution. With the hardware your listing, nothing on your hardware should be limiting you to run what your running.