Performance tweaking and optimisation server settings.

Discussion in 'Performance Tweaking' started by United Factions, Apr 6, 2013.

  1. Dear Spigot community, first off I want to give you all a big thank you for the support that you offer to me, and to everyone else here. It's just amazing. I try to do my part on it as well every now and then. However right now, I am trying to get the best optimisation I can do ;)

    But for that I will need your guys help. - md_5 could you please take a look as well for me.

    Okay so this is the situation:

    April 15 - 2013. - My grand re-opening of United Factions is going to happen. ;) The server will be open that date, and the plan is to not have any lag or try to minimize it.

    Goals and Mission:
    - The goal is to make the server handle 200 players without any noticeable lag or TPS drop. In the utopia situation I would like to achieve a 200 players online, with 20 TPS of course, but I don't expect anything to happen like that, but at least I try the best I can okay? Together with your helps I hope I can get far.

    - The mission is to give everyone room on the server and to let them enjoy it as much as possible. ;) Therefor of course we need tools, and time. Below here are the tools that will be used to achieve this 20 TPS - 200 players thing currently:

    Server specifications:
    Intel Xeon E3-1240 V2 @ 3.40GHz
    16 GB RAM
    128 GB SSD Crucial
    Datacenter at the London Internet Exchange. ;) Awesome. -> Thanks Eoin. (cookednoodles)

    - SQL server for SQL storage of Minecraft data. (used by a few plugins, explained later on)

    Server software:
    - Linux x64, Ubuntu Server. -> with Java 7
    - Mark2, as startup script and use for the start, backup, saving and more. (Thanks to mcdevs, especially storm2x)

    Server plugins:
    Code (Text):

    AntiFarm, latest version
    ArenaSpleef, latest version
    AutoAnnouncer, latest version
    BattleArena, latest version
    BookShop, latest version
    Buycraft, latest version
    ChestShop, latest version
    Citizens, latest version
    CleanChat, latest version
    CombatTag, latest version
    CompatNoCheatPlus, latest version
    Essentials, latest version
    EssentialsAntiBuild, latest version
    EssentialsChat, latest version
    EssentialsProtect, latest version
    EssentialsSpawn, latest version
    Factions, latest version
    GroupManager, latest version
    IPLog, latest version
    LinksOnSigns, latest version
    Lottery, latest version
    mcMMO, latest version
    Multiverse-Core, latest dev version
    Multiverse-Portals, latest version
    NoCheatPlus, latest version
    NoPortals, latest version
    ObsidianDestroyer, latest version
    ObsidianFlow, latest version
    Orebfuscator, latest version
    Permissions, latest version
    PlayerHeads, latest version
    ProtocolLib, latest version
    PvPTimer, latest version
    Shopkeepers, latest version
    SimpleRegionMarket, latest version
    Stats, latest version
    StreetLights, latest tweaked version. Thanks Devix!
    UnderWaterTNT, latest version
    Vault, latest version
    Vote, latest version
    Votifier, latest version
    Weed, latest version
    WorldBorder, latest version
    WorldEdit, one of latests dev versions
    WorldGuard, one of latests dev versions
     
    Spigot bukkit.yml settings


    Code (Text):

    # This is the main configuration file for Bukkit.
    # As you can see, there's actually not that much to configure without any plugins.
    # For a reference for any variable inside this file, check out the bukkit wiki at
    # http://wiki.bukkit.org/Bukkit.yml
    settings:
      allow-end: true
      warn-on-overload: true
      permissions-file: permissions.yml
      update-folder: update
      ping-packet-limit: 100
      use-exact-login-location: true
      plugin-profiling: true
      connection-throttle: 4000
      query-plugins: false
      deprecated-verbose: default
      shutdown-message: Server has closed, restarting ASAP
      filter-unsafe-ips: true
      whitelist-message: www.unitedfactions.com new website! Read the News!
      log-commands: true
      command-complete: true
      spam-exclusions:
      - /skill
      tab-ping: false
      timeout-time: 60
      restart-on-crash: false
      restart-script-location: start.bat
      bungee-proxies:
      - 127.0.0.1
      texture-resolution: 16
      map-send-interval: 10000
      stop-message: Server restarting. We are right back! Hold on
    world-settings:
      default:
        growth-chunks-per-tick: 650
        mob-spawn-range: 4
        random-light-updates: false
        aggregate-chunkticks: 4
        item-merge-radius: 3.5
        exp-merge-radius: 3.5
        wheat-growth-modifier: 100
        cactus-growth-modifier: 100
        melon-growth-modifier: 100
        pumpkin-growth-modifier: 100
        sugar-growth-modifier: 100
        tree-growth-modifier: 100
        mushroom-growth-modifier: 100
        entity-activation-range-animals: 48
        entity-activation-range-monsters: 48
        entity-activation-range-misc: 16
        entity-tracking-range-players: 48
        entity-tracking-range-animals: 48
        entity-tracking-range-monsters: 48
        entity-tracking-range-misc: 32
        entity-tracking-range-max: 64
        info: true
        monsters-per-chunk: 70
        land-creatures-per-chunk: 15
        water-creatures-per-chunk: 5
        chunky-freeze-temperature: true
        vine-growth-modifier: 100
      world:
        growth-chunks-per-tick: 1000
      world_nether:
        view-distance: 5
        growth-chunks-per-tick: 0
        random-light-updates: true
        water-creatures-per-chunk: 0
    spawn-limits:
      monsters: 70
      animals: 15
      water-animals: 5
      ambient: 15
    chunk-gc:
      period-in-ticks: 600
      load-threshold: 0
    ticks-per:
      animal-spawns: 400
      monster-spawns: 1
      autosave: 900
    auto-updater:
      enabled: true
      on-broken:
      - warn-console
      - warn-ops
      on-update:
      - warn-console
      - warn-ops
      preferred-channel: rb
      host: dl.bukkit.org
      suggest-channels: true
    database:
      username: -
      isolation: SERIALIZABLE
      driver: com.mysql.jdbc.Driver
      password: -
      url: jdbc:mysql://IP:3306/database_name
    orebfuscator:
      enable: false
      engine-mode: 2
      update-radius: 2
      disabled-worlds:
      - UnitedFactions
      - UnitedFactions_nether
      - UnitedFactions_the_end
      - Shopworld
      blocks:
      - 1
      - 5
      - 14
      - 15
      - 16
      - 21
      - 48
      - 49
      - 54
      - 56
      - 73
      - 74
      - 82
      - 129
      - 130
    storm-settings:
      strong-electrical-storm:
        chance: 0
        lightning-delay: 10
        lightning-random-delay: 20
      electrical-storm:
        chance: 0
        lightning-delay: 40
        lightning-random-delay: 150
      strong-thunderstorm:
        chance: 0
        lightning-delay: 60
        lightning-random-delay: 250
      thunderstorm:
        chance: 0
        lightning-delay: 0
        lightning-random-delay: 500
      weak-thunderstorm:
        chance: 0
        lightning-delay: 300
        lightning-random-delay: 1000
      rainstorm:
        chance: 0
        lightning-delay: 500
        lightning-random-delay: 2000
    config-version: 2
     
     
    Mark2 default properties:
    Code (Text):

     
    ### ###
    ### ### This file defines the default property set for mark2. When mark2 is started,
    ### ### it is copied to /path/to/mark2/config/mark2.properties (if it doesn't exist)
    ### ### and never used again.
    ### ###
    ### ### Please DO NOT edit this file! If you wish to upgrade mark2 in the future,
    ### ### your changes may conflict with git's. DO NOT remove this file.
    ### ###
     
    ###
    ### This file defines the configuration that all your servers share. You can do fine-
    ### tuning on a server-by-server basic by dropping a file called 'mark2.properties'
    ### in the same directory as your 'minecraft_server.jar'. There's an example in the
    ### 'samples' directory.
    ###
     
    ###
    ### mark2 options
    ###
     
    #Semicolon-separated list of jar file patterns to try. For example, if you're
    #running a Feed The Beast server you probably want something like FTB-*.jar
    mark2.jar-path=craftbukkit-*.jar;craftbukkit.jar;minecraft_server.jar
     
    #How long to wait for the server to shut down after issuing `stop` before we kill it.
    mark2.shutdown-timeout=60
     
    ###
    ### mark2 services
    ###
     
    #mark2 provides a few services that provide statistics on the running server.
    #it's recommended you enable all of them, as this gives you the best idea of
    #how the server's running and there isn't any performance impact
     
    #ping: sends a 'multiplayer server list ping'
    mark2.service.ping.enabled=true
    mark2.service.ping.interval=5
     
    #query: connect periodically to the server's G4S server
    mark2.service.query.enabled=true
    mark2.service.query.interval=5
     
    #top: calls `top` to get a thread-by-thread view of cpu/mem
    mark2.service.top.enabled=true
    mark2.service.top.interval=5
     
    ###
    ### JVM options
    ###
     
    #memory allocation pool initial size
    java.cli.X.ms=2800M
    #memory allocation pool max size
    java.cli.X.mx=2800M
     
    #Possible speed-ups - use only if you know what you're doing
    #http://stas-blogspot.blogspot.co.uk/2011/07/most-complete-list-of-xx-options-for.html
     
    #java.cli.X.incgc=
    #java.cli.XX.AggressiveOpts=true
    #java.cli.XX.CompileThreshold=1500
    #java.cli.XX.MaxGCPauseMillis=50
    #java.cli.XX.MaxPermSize=256M
    #java.cli.XX.MaxTenuringThreshold=15
    #java.cli.XX.OptimizeStringConcat=true
    #java.cli.XX.ParallelGCThreads=4
    #java.cli.XX.PermSize=64M
    #java.cli.XX.TargetSurvivorRatio=90
    #java.cli.XX.ThreadPriorityPolicy=42
    #java.cli.XX.TieredCompilation=true
    #java.cli.XX.UnlockExperimentalVMOptions=true
    #java.cli.XX.UseAdaptiveGCBoundary=true
    #java.cli.XX.UseCompressedOops=true
    #java.cli.XX.UseConcMarkSweepGC=true
    #java.cli.XX.UseFastAccessorMethods=true
    #java.cli.XX.UseG1GC=true
    #java.cli.XX.UseGCOverheadLimit=true
    #java.cli.XX.UseLargePages=true
    #java.cli.XX.UseSSE=3
    #java.cli.XX.UseStringCache=true
    #java.cli.XX.UseThreadPriorities=true
    ###
    ### Plugins
    ###
     
    #ALERT: prints messages on an interval
    plugin.alert.enabled=true
    plugin.alert.path=alerts.txt
    plugin.alert.interval=200
    plugin.alert.command=say {message}
     
    #BACKUP: compresses the world folders to a .tar.gz
    plugin.backup.enabled=true
    plugin.backup.path=backups/{timestamp}.tar.gz
    plugin.backup.mode=include
    #semicolon separated paths to include or exclude, depending on mode
    plugin.backup.spec=worlds*
     
    #HANG: restarts the server if it crashes
    plugin.hang.enabled=true
     
    #crash: checks the server console is still accepting commands
    plugin.hang.crash-enabled=true
    plugin.hang.crash-interval=60
    plugin.hang.crash-fail-limit=10
    #oom: restarts the server if it runs out of memory
    plugin.hang.oom-enabled=true
    #ping: periodically make sure the server is still accepting connections
    plugin.hang.ping-enabled=true
    plugin.hang.ping-fail-limit=10
    #pcount: *if* you run a big server where having 0 players for an extended period
    #means there's something wrong, enable this option
    plugin.hang.pcount-enabled=false
    plugin.hang.pcount-fail-limit=10
     
    #IRC: relay in-game chat
     
    #connection
    plugin.irc.enabled=false
    plugin.irc.host=
    plugin.irc.port=
    plugin.irc.ssl=false
    plugin.irc.channel=
     
    #user
    plugin.irc.nickname=RelayBot
    plugin.irc.realname=RelayBot
    plugin.irc.username=RelayBot
    plugin.irc.password=
     
    #game -> irc settings
     
    #whether to right-align usernames into a column
    plugins.irc.game-columns=true
     
    #announces server start/stop
    plugins.irc.game-status-enabled=true
    plugins.irc.game-status-left=!
    plugins.irc.game-status-right= | server {what}.
     
    #chat
    plugins.irc.game-chat-enabled=true
    plugins.irc.game-chat-left={username}
    plugins.irc.game-chat-right= | {message}
     
    #join
    plugins.irc.game-join-enabled=true
    plugins.irc.game-join-left=*
    plugins.irc.game-join-right= | --> {username}
     
    #part
    plugins.irc.game-quit-enabled=true
    plugins.irc.game-quit-left=*
    plugins.irc.game-quit-right= | <-- {username}
     
    #[SERVER] messages (sent with /say)
    plugins.irc.game-server-message-enabled=true
    plugins.irc.game-server-message-left=*
    plugins.irc.game-server-message-right= | [server] {message}
     
    #/me (bukkit only)
    plugins.irc.game-me-enabled=true
    plugins.irc.game-me-left=*
    plugins.irc.game-me-right= | {username} {message}
     
    #irc -> game settings
    plugins.irc.irc-chat-enabled=true
    plugins.irc.irc-chat-command=say [IRC] <{nickname}> {message}"
     
     
    #LOG: exports server.log to a directory
    plugin.log.enabled=true
    plugin.log.gzip=true
    plugin.log.path=logs/server-{timestamp}-{status}.log.gz
    #whether to log to the default server.log format
    plugin.log.vanilla=false
     
    #MCBOUNCER: simple mcbouncer plugin for vanilla servers
    plugin.mcbouncer.enabled=false
    plugin.mcbouncer.api-key=
     
    #MUMBLE: provides a user count/details to a user when they say something like !mumble
    plugin.mumble.enabled=false
    plugin.mumble.host=
    plugin.mumble.port=64738
    plugin.mumble.timeout=10
    plugin.mumble.trigger=!mumble
    plugin.mumble.command-up=\
        msg {username} &2host: &a{host}\n\
        msg {username} &2port: &a{port}\n\
        msg {username} &2status: &aup! users: {users_current}/{users_max}
     
    plugin.mumble.command-down=\
        msg {username} &2host: &a{host}\n\
        msg {username} &2port: &a{port}\n\
        msg {username} &2status: &adown.
     
    #RSS
    plugin.rss.enabled=false
    plugin.rss.url=
    plugin.rss.check-interval=60
    plugin.rss.command=say {link} - {title}
     
    #SAVE: regularly save the game
    plugin.save.enabled=true
    plugin.save.warn-message=WARNING: saving map in {delay}.
    plugin.save.message=MAP IS SAVING.
     
    #SCRIPT: run a program or command on a given event
    #see resources/scripts.example.txt for details
    plugin.script.enabled=true
    plugin.script.path=scripts.txt
    plugin.script.user=
    plugin.script.shell=/bin/sh
     
    #SU: https://github.com/edk141/bukkit-sudo/ integration
    plugin.su.enabled=true
    plugin.su.command=sudo -su {user} -- {command}
     
    #SHUTDOWN: cleanly stop/restart the server, or do so on an interval
    plugin.shutdown.enabled=true
    plugin.shutdown.stop-warn-message=WARNING: server going down for planned maintainence in {delay}.
    plugin.shutdown.stop-message=Server going down for maintainence.
    plugin.shutdown.restart-warn-message=WARNING: planned restart in {delay}.
    plugin.shutdown.restart-message=Server restarting we will be right back!
     
    #TRIGGER:
    plugin.trigger.enabled=true
    plugin.trigger.filename=triggers.txt
    plugin.trigger.command=msg {user} {message}
     
    NOTE: Everything with # is commented out! Including at these JVM flags!

    Plugins using SQL and why:
    - mcMMO, large player database. According to #mcmmo, GJ, everyone there SQL is faster for my purpose and playerbase.
    - Stats, for statistics of players, cannot get rid of this. Very efficient plugin! Thanks and credits to lolmewn and cynexit, and Jack Bentley for all the great help!
    *MAYBE: Chestshops, if I can afford that, since it works with the JBMC database connection, it might be laggy.



    Timings and current load report:
    What I have here too is a timings report with a runtime of 300 seconds, with 1 player.
    http://aikar.co/timings.php?url=5683948

    Also current load: cpu: 8.10% // mem: 0.13% // players: 0 of 200 -> Thanks to mcdevs! mark2

    Current issues that are being worked on:
    - mcMMO is informed on the long take of PlayerLoginEvent. t00thpick1 is working on this soon.
    - Simple-Region-Market has an issue that it keeps all chunks loaded, that needs to be changed. Ticket can be found here: http://dev.bukkit.org/server-mods/simple-region-market/tickets/407-task-taking-too-long/
    - Factions is working on a few improvements: https://github.com/MassiveCraft/Factions/issues/225
    Especially on performance issues. Would be pretty awesome if they are able to optimise that, not only for my server.


    Current questions by myself:
    - Groupmanager unfortunately has issues at the PlayerJoinEvent and PlayerQuitEvent, of course this could be related to mcMMO. Not sure, if it can though of course not tested without mcMMO yet.

    - All these things that are in the mark2 properties file, I am really really bad with all of the java arguments. I have not modified any of these, these are mcdevs (mark2 devs) default!
    as #java.cli.XX.MaxPermSize=256M, #java.cli.XX.MaxTenuringThreshold=15, #java.cli.XX.OptimizeStringConcat=true, #java.cli.XX.ParallelGCThreads=4, #java.cli.XX.PermSize=64M, #java.cli.XX.TargetSurvivorRatio=90, #java.cli.XX.ThreadPriorityPolicy=42. ETC. ETC. ETC. I tried to understand the guides that TNT and md_5 have been making, but I really do not know what to do or what not to do.

    The big question is:

    - What would I change in the Java arguments in the mark2 properties according to you guys, what should I remove, add or change. And why should I do that?
    -> The ultimate goal is to run 20 TPS with 200 players stable. If that can be achieved, that be awesome! Thanks for all your help already.

    Things I do not want to do:
    - Change my render distance to below 10.
    - Run orebfuscator on engine mode 1 instead of 2.


    Final note:
    - All plugins we use are public, and can be downloaded through bukkitdev, only 1 plugin is customized a bit. If you have a question about a certain plugin, be not afraid to ask! But please keep this on topic and try to help me. ;)

    Everyone thanks for your feedback, your thoughts, questions, answers and anything more that can get me wiser and give me more input. ;) Special thanks to all the great developers that make these things possible.
     
    #1 United Factions, Apr 6, 2013
    Last edited: Apr 6, 2013
  2. Okay, here some answers from IRC @ #spigotmc:
    ---------------------------------------------------------------------------------------------------
    VeMacs: " SupaYoshi: don't use any Java flags, simple" -> No reason, most expected reason:
    Because flags usually do more damage then good, they cause more load and work for the CPU and memory usage.
    Source:
    http://forums.bukkit.org/threads/how-to-improve-minecraft-server-performance.662/
    http://www.spigotmc.org/threads/spigot-configuration-guide-last-updated-3-24-2013.802/

    - Statement by EvilSeph regarding to JVM flags: (thanks vemacs)
    http://forums.bukkit.org/threads/op...edback-thx-i-am-back.25994/page-3#post-493421

    Note I cannot find the current real guide that told you to remove all the JVM flags but it is true i suppose, so I might be editing this.
    ---------------------------------------------------------------------------------------------------
    Z750: " JVM is usually pretty good at figuring out what it needs to do on its own, so forcing it to do things may not always be the most efficient way to run things - thats the logic most servers use about running w/out lots of flags "
     
  3. Some tips for NoCheatPlus:
    You can mess around with the data expiration section:
    Code (Text):
    data:
      expiration:
        active: false
        duration: 60
        history: false
    With this option you can force NC+ to clean all data based on whatever you set to duration. Be aware that some checks need more data than other, so cleaning data too fast could make some checks inaccurate in detecting cheats.

    You can also disable some harmless cheats such as NoSwing

    More infos here: http://dev.bukkit.org/server-mods/nocheatplus/pages/documentation/configuration/#w-data-section

    PermissionsEx will probably be faster than GroupManager and yea if 1 plugin causes high load than other plugins will get slowed down also.
     
  4. MyPictures, what is the default duration 60 meaning? seconds, days, hours? Would 24 hours be enough? / to much? please help ;)

    About PermissionsEX, I am quite frequent in IRC. Everyone there hangs on with this Google Docs file, that PermissionsEX is broken and should be fixed... or move to GM. (Which I am using already) I optimised my GM files. Though Essentials devs tell me this is nothing bad really ;) So they say that this is not theyre fault. -> I need to test without mcMMO or with a optimised mcMMO.

    Source:
    PermissionsEx Google Docs:
    https://docs.google.com/document/d/1BIgAU7senSG1hY0BjQAQFAnbuOqFhpln7ffN6pkOp4s/edit?pli=1
     
  5. One thing that can help a ton, get rid of the Orebfuscator plugin, and use the build in Orebfuscator in native spigot. Much more resource friendly.
     
  6. "Duration in minutes after logout, after which data gets erased." So minutes :).

    About PEX: Thats just the opinion about others, for me it works fine and all my installed plugins register their permissions as should. I would give it a try, on high amounts of players its wonderful!
    And also regex for the win!:
    - essentials.(balancetop|balance|eco|pay)
    - essentials.(afk|help|getpos|list|me|msg|motd|rules)
    - essentials.(spawn|delhome|home|sethome|tpa|tpaccept|tpdeny|warp|warps|back)

    This document shows some bad points about PEX but where are the good ones? Only because you bypass the BukkitPermissions system does not mean that you automatically are bad. But I don't want to discuss that here any further, I just like to say: Try it out and see how it fits your needs. I used it since 1 year and never had any big issues with it.
     
  7. Seriously? I never said md_5 made it. If you look here you can read up on his idea of that if it is put directly in to the launcher it will run much more efficiently due to threading and more direct access to the network. And it does. I run it on engine 1 perfectly. If you have a problem with installing ProtocolLib, (which you already do have installed), then so be it. I am just giving a suggestion to help you out.
     
  8. Hai, I didnt say you said that md_5 made it either ;) I know i know :) Orebfuscator does threading as well as a stand alone plugin, only thing that orebfuscator standalone has more is proximyhider, which is removed i believe. Orebfuscator actually threads really well. Second, it has engine mode 1 as well ;) And engine mode 1 is much much less CPU intensive, but also makes ores exposed to air x-ray-able. Which i don't want that is why I wrote in "things I do not want to do":
    - run orebfuscator in engine mode = 1. ;) Anyway, no hard feelings.

    With source I ment that spigot handles threading better @ antixray then lishid does in his own stand alone plugin. Which isn't the case currently. Anyway ontopic and back to timings please.