My server problems

Discussion in 'Performance Tweaking' started by Qentus, Dec 31, 2012.

  1. Hi everybody,
    I have my own server since September 2011.
    Everything was ok until 1.4

    I have 4GB RAM for server, but it is never fully used.
    I have 4-core CPU, but usage is always under 50% (each core).

    I am using a few days old Spigot 1.4.6 build

    Server is running in RAMdisk.

    And my problem:
    Every time server is saved with /save-on , there is 1sec lag. (save interval is I think 30 seconds)
    It is really annoying. TPS is always 17-20 with 20-30 players.
    I don´t know why I have theese lags.

    Solutions that I have tried:
    /save-off and save server every X minutes with /save-all
    With /save-off theese 1s lagz were gone but /save-all doesnt unload chunks so number of chunks were only growing ale laggz were bigger and bigger.

    You can say that 1s lag isnt that bad but if you are playing for an hour then you get 120 theese lagz.

    I can post you any additional information you want.

    P.S.: Sorry for my English
     
  2. From the looks of things, you have two problems, one is clear and easy to rectify, the other we will need more information to know.

    First, for the saving lag; Bukkit.yml have a setting for auto saving, which is defaulted to 6000 [ticks] (5 minutes). Every time you issue a save-all command, server must copy chunk data from memory to 'storage' (ramdisk). Because of this copy process, regardless of how fast your memory is, there will always be a tiny bit of delay.

    Second, for the chunks not being unloaded; chunks will be automatically unloaded whenever players are further than the configured view distance. This can be set in server.properties as "view-distance" in number of chunks, and should be default to 10 (9 chunks away from the one player's standing on). This means the server will load 9 chunks infront of the player, the chunk the player is on, and 9 chunks behind the player, or, the player can see minimum of 9*16 = 144 tiles away from where they are standing. If you've got this set correctly, chunks should unload. What makes you certain that chunks are not being unloaded? What plugins do you have running that may potentially deal with chunk data?
     
  3. Thanks for you reply.
    But I think that it isnt that easy.

    I have autosave: 900 set by default (45 seconds).
    And there is a 1 second lag every 45 seconds.

    And view-distance is set to 5 for a long time.

    And unloading of chunks?
    Step 1:
    /save-on - lag every 45 seconds

    Step 2:
    /save-off - lagz are gone, NoLagg or Essentials /elag says that forexample 3000 chunks are loaded

    Step 3:
    type /save-all as many times as you can :D

    Step 4:
    after X minutes trying to /save-all, /elag says that 5000 chunks are loaded
    (lagz are still gone, because our server can handle 5000 chunks with 30 players)

    Step 5:
    /save-on - /elag says that there are 3000 loaded chunks and server is now lagging again
     
  4. Qentus - Without actually looking at the system, it is hard to say definitively what is going on. However, several things spring to mind about this... I'll list them here in no particular order, and you can check into them in any order.

    If I remember correctly, view distance of 5 means 5 chunks ahead, chunk player is on, 5 chunks behind, so (5+1+5)^2 per player (just check the wiki, I was wrong to think it was n-1 instead of n), or 121 chunks per player; assuming zero overlap (i.e.: everyone is more than 6 chunks away from each other), with 30 players online, that should go to only 3.6K chunks loaded, plus a couple for spawn chunks in the different worlds. Chunks should (theoretically) be unloaded after they move out of range, so if you are seeing 5000 loaded chunks, there is probably a bad plugin keeping the chunks loaded. Disable plugins one at a time, and monitor the performance to see if you can find the culprit. If you have multiverse, and a large amount of worlds set up, it may also be the source of the problem.

    You mentioned you are using RAM Disk, but also that you only have 4GB RAM. What are the distribution setup right now between RAM Disk (the 'disk' capacity), java heap (XMX parameter), and OS (unallocated to the previous two). Please also double check the command `free -m` (without the quotes) in terminal, and see how much free memory you have available. Keep in mind that your OS (and other applications such as ssh server, ftp server, kernel logger, mysql database, etc. etc. etc.) also need some memory set aside, and you should try to avoid swapping if at all possible.

    You are saving too often. I understand you've done it as a temporary work-around to the problem, but once you have the issue resolved, you should consider reducing the save interval. Saving that frequently basically forces your system to move large amount of memory back and forth, even if they are all in RAM (I really question whether or not this is actually the case), which may come at expense of a little bit of CPU power. If you are using physical disk or SSD, you will make the disk work extra hard, and may cause it to fail sooner than designed.
     
  5. andyhuang

    On my server there is 16GB avaliable RAM. 4GB are for java, 4GB are for RAMdisk and I have 8GB RAM free for my second server in the future.

    I´ll try to increase the interval. But if the interval is 5 times bigger, does it mean that the lag caused by the saving will be 5 times longer?

    And multiverse. Yes I am using multiverse. There are 4 small worlds and one main world.


    P.S.: Do you think that server cant handle forexample 30 worlds, each 300x300 blocks? It is actually smaller than current main world. I have one idea for my second server so I am asking.
     
  6. PhanaticD

    Patron

    dont use the auto-save thing in bukkit.yml its crap use a cron or plugin
     
  7. I know that it´s crap but I cant save my worlds without /save-on as I described earlier in this post.

     
  8. I'm guessing you meant it doesn't do it at proper interval? If so, that's because t is not a time based setting but a ticks based setting; so if your TPS is less than optimal, the timings will be off. If you mean it in a different way, please elaborate further.


    Qentus

    Looks like the RAM is okay for now then. Just to be on the safe side, can you please do this in terminal and copy and paste the result back:
    Code (Text):
    free -m
    I do not know for certain if increasing it will help reduce the lag or not, as I have no way to test that easily. At the peak, my server reached 10 concurrency, and now I am re-designing the world and game mode, I don't have a busy enough server to test the results. However, I see saving over-frequently as a very easy way to hurt your server hardware, and should be avoided if at all possible.

    As for worlds, I do not think you should do small worlds, as players might not like it (this really depends on what kind of game mode you are running). However, if you think that might work better for you, you can consider what I've posted in the thread: Multiple instances on single box to run one server?
     
  9. andyhuang

    ram.PNG

    I want to make small worlds as planets with different types of generated terrain.
     
  10. Qentus

    Sorry about late reply, got caught up replying to something else on minecraft related subreddits :)

    It looks like you are running into Swap. I don't know how much this would help (if any), but give this a try.

    Warning: Sysctl sets settings to tell your kernel what to do. Some settings are safer than others. Incorrectly configuring something may result in your system not able to boot. Please practice extreme caution before continuing. Swappiness is generally a fairly safe thing to change. It tells the kernel how often it should try to use swap. A value of 0 says never use it unless you are fully out of Ram, value of 100 means to always prefer swap over actual memory. Default on my Debian 6.0.6 was set to 60. I do not know what distro you use, and even if I do know, I do not know their default values. You can find out what the default is before you make the change with the command:
    Code (Text):
    sysctl vm.swappiness

    Once you've read the warning note, and understand what you are changing, I recommend you do some reading online before going any further. I am not responsible if your system refuse to boot after doing this.

    Create the file: /etc/sysctl.d/swappiness.conf
    Code (Text):
    vm.swappiness = 0
    Run this command:
    Code (Text):
    sysctl -p /etc/sysctl.d/swappiness.conf
    Verify that the setting took with the check command again:
    Code (Text):
    sysctl vm.swappiness

    If it doesn't (this really shouldn't happen), you will need to reboot the server to have it load automatically.

    Basically, what we are doing is avoiding swap whenever possible. This will reduce the situation where if you are doing a save, it still hits up the disk because kernel feel it is better to use swap instead of ram for your tmpfs partition acting as ramdisk.