Server Monitoring

Discussion in 'Server & Community Management' started by graywolf336, Jul 28, 2013.

  1. graywolf336

    Supporter

    I've been running a server for a little over two years now and it hasn't been anything fancy, nothing big, just a good ole Towny server with a good community. I can't complain about donations as we've received plenty to take care of our monthly bills and such, but during these years of running a server the area that I've always lacked and am looking to pick up my slack in is server monitoring and so I'm curious what everyone else uses or if anyone else uses anything.

    Right now I use two things, the first is a simple alerting deal that my hosting provider provides which is port checking and bandwidth monitoring. I've setup the port monitor to monitor the default Minecraft port (we don't use bungee since we're a smaller server of ~30-50 people) and then I have it send me hourly emails on the bandwidth usage. I've found that the bandwidth is nice to have and monitor as I can usually tell when a lot of people have been on because the used bandwidth goes up a lot and the simple port monitor isn't too bad but it's just that - simple.

    Recently though I've started looking around at monitoring services and found a somewhat decent one but it has it's flaws. The one I've found is Scout App. It has some pretty good monitoring and if you purchase the $15 one or higher you will get SMS alerts, which are really nice. I've setup some disk space monitors there, port monitors, memory, and several others and use the SMS alerts for the critical monitors and it has helped me keep my server up more than when I wasn't using it (yes, we do use RToolkit as our wrapper). I've noticed though that they allow you to create your own "plugins" for their system that let you monitor custom things about your server and that has a spark an interest in me to see what it takes to get a custom one to monitor things about Minecraft such as the tps, players, can players log in, etc and after asking them if it was possible they replied back with something around "if you can do it with Ruby, then you can make your own". My deal is I don't know Ruby and honestly don't know if it is possible to monitor something Minecraft related using Ruby.

    So, I said all this to ask this question. Do you use any type of monitoring software/website/service? If so, what is it and would you recommend it to someone? If you do recommend it, why?

    As a side note too, if anyone knows Ruby and is interested in making a custom plugin for the Scout monitoring let me know as I would be willing to pay for one.
     
    • Like Like x 1
    • Like Like x 1
  2. graywolf336

    Supporter

    Do they have any monitors specific to Minecraft? I see they have some Java related monitors which looks good so I might have to check that out. Thanks
     
  3. What elements do you want to monitor? I am able to pull tons of data but there was also tons of setup to get to this point. This is why I said advanced admin in terms of system administration, not just minecraft.

    Example things I do monitor:
    - VPS Connections (player count per relay node)
    --- Total Connections (player count per server/port/ip, etc...)

    - Java Processes [spigot]
    --- State, Memory Usage, CPU Usage

    - Server Performance Metrics
    --- Overall CPU, Memory, Swap Space, Process Monitoring, etc etc...

    - Minecraft Server Data [via JSON]
    --- TPS, Player Count, Memory Usage
     
  4. For player count I use a php script with queries the server and returns the player count as an integer. This allows you to also track Bungee counts (though one can assume that is just a total of the others - unless you use multiple Bungee servers).

    Can also use Zabbix for monitoring MySQL and you can set it to perform set actions on certain events. For example, after 1 minute of the server being offline you could have it email you, after 10 minutes you could have it execute a command (which restarts the process etc).
     
    • Informative Informative x 1
  5. I have not dug into the capabilities of pulling data directly from Bungee. Any change you can provide a quick link of where to start to save me some time? Example php scripts?

    One thing about multiple bungees that I tried to get addressed in a feature request is shared state information. If a player disconnects from one Bungee and reconnects to another, their location information and such is no longer the same. Also connected user count (displayed on query to players) is different on each bungee. If I had more throughput on time I would love to tackle a true Bungee distribution system that inter-communicates critical values.
     
  6. joehot200

    Supporter

    @graywolf336, for in-game monitoring i would recommend SuperLogger, it logs everything neatly and by date, and so every day i can look at the latest logs (less than 5MB large, storing all player commands, chat, deaths, etc) and see if staff are abusing, see who is muted/banned, and see who said what (e.g. ban swearers/spammers).
     
  7. Would love to show you a neat solution which can be transferred easily... But here is what I have:

    In my zabbix agent config (on the Bungee Server) I have:
    Code (Text):
    # Key syntax is minecraft.players
    UserParameter=minecraft.players,/usr/lib/zabbix/minecraft players
     
    My /usr/lib/zabbix/minecraft contains (has some other stuff which is questionable in use/broken):
    Code (Text):
    #!/bin/bash
     
    is_running() {
        if ps ax | grep -v grep | grep -v -i SCREEN | grep spigot.jar > /dev/null ; then # Could/Should tidy these greps... note: I run the spigot in a screen session.
            return 0
        else
            return 1
        fi
    }
     
    case "$1" in
      players)
            if is_running; then # This is a function which returns if the process exists
                /usr/bin/php /usr/lib/minecraft/mineplayers.php 2> /dev/null || { echo 0; exit 1; }
            else
                echo 0
            fi
    ;;
      *)
            exit 1;
    ;;
    esac
     
    exit 0;
    My /usr/bin/php /usr/lib/minecraft/mineplayers.php:
    PHP:
    <?php
    require __DIR__ . '/MinecraftQuery.class.php';
     
    $Query = new MinecraftQuery( );
     
    try {
        $Query->Connect( '127.0.0.1', 25566 );
     
        $info = ($Query->GetInfo());
        $players = ($Query->GetPlayers());
        sort($players);
        $player_array = implode(", ", $players);
    }
    catch( MinecraftQueryException $e) {
        $error = $e->getMessage( );
    }
     
    if (isset($error)) {
        exit(1);
    } else {
        print $info["Players"];
        echo "\n";
        exit(0);
    }
    ?>
    With /usr/lib/minecraft/MinecraftQuery.class.php as a dep (and it uses the QUERY protocol set in server.properties. To be honest, looking back at this, the player grabbing could be probably simplified by not using the QUERY and instead the Minecraft ping - along the lines of:
    PHP:
        function mc_status($host,$port) {
     
            $fp = fsockopen($host,$port,$errno,$errstr,$timeout=10);
     
                fputs($fp, "\xFE\x01");
     
                $response = '';
     
                $response .= fgets($fp);
     
                fclose($fp);
     
          $response = explode("\x00\x00", $response);
     
        return $response;
     
        }
     
     
     
        $data = mc_status("127.0.0.1", "25565");
     
     
     
        echo $data[4];
    Then I have the item in Zabbix as a 'Decimal' 'Numeric (unsigned)' (I think the default - though could/should change it?)

    Now, hopefully there are some useful snippets though which you can piece together. Messy and likely a bit overly extended, but I am too lazy to make it better and it works with no security issues?

    Returns 0 for count if the server is offline.
     
    • Informative Informative x 1
  8. This will give you the total active connected players (via network) to 25565 on the entire server. Perfect for a dedicated with one proxy running on it. Also works for other mc servers if you throw in a grep statement for an IP or port. I don't have a need for such a setting though on the network level. Though using the netstat command, speed is good and not intensive until you get to the 5k connections (total of all on server).

    I use tcp_perf for 25565 to detect state of bungee port along with proc check.

    UserParameter=minecraft.host.connections,netstat -n |grep ESTABLISHED |grep 25565 |wc -l
     
    • Like Like x 1
    • Informative Informative x 1