[TUTORIAL] Monitoring your Linux server in Real Time [Medium]

Discussion in 'Systems Administration' started by NewEraStudios, Jul 19, 2020.

  1. This tutorial is only for people who uses some Linux distribution and they have total access to their vps/dedicated! Sorry for my english ^^

    I've been running websites and minecraft servers in Linux and serveral times the idea of monitoring all my machine resources got to my head. I did a quick search in google and found serveral panels to monitor all your server resources, but all of them were or paid or the data was not in real time, or just a few data could be monitored.

    Suddenly and luckly, i found a really good way to monitor your resources and i will teach you how to install the panel and use it. The installation process is really simple, easy and fast.

    With this panel you will be able to monitor in real time your CPU usage, your RAM usage, your Incoming and Outgoing network usage, in case you use Apache or nginx to host a website, also all the incoming requests and their type (post, get, ...), in case you use some service like Fail2ban you will be able to see the banned IPs too, netfilter status, etc..., you can spend HOURS looking at all the real-time graphs that it makes to you.

    1. First, lets take a look at how the panel looks when installed, so you can get a conclusion if you like it or not before insalling it:

    [​IMG]

    [​IMG]

    [​IMG]

    2. I will show you how to easily install it in Ubuntu distrobution, if you want to install it in another distro that is not Ubuntu, you can look at their website to find how to do it, as i don't want to make a thread so large.

    3. First of all run the following linux command to install all the dependencies that the panel needs:
    Code (Text):
    sudo apt-get install zlib1g-dev uuid-dev libuv1-dev liblz4-dev libjudy-dev libssl-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl python cmake -y
    4. Then you can proceed to the installation of Netdata by doing this command:
    Code (Text):
    bash <(curl -Ss https://my-netdata.io/kickstart.sh)
    Auto-updates of netdata are provided with this installation command

    5. Lets start with security, if you have an active firewall filtering ports, you will need to allow the port 19999, as netdata runs there.

    6. You will also need to secure access to your dashboard to prevent attacks, you will find out information on how to do it here https://learn.netdata.cloud/docs/agent/web/server


    7. Additionally, if you want to use netdata to monitor your server TPS and user count of your minecraft server, you can install this netdata plugin https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/spigotmc

    Then you will be able to enter the panel navigating to http://your.machine.ip:19999/ in your browser :)

     
    #1 NewEraStudios, Jul 19, 2020
    Last edited: Jul 20, 2020
  2. Btw you can install it easily by just doing
    Code (Text):
    bash <(curl -Ss https://my-netdata.io/kickstart.sh)
    If you also have UFW installed
    Code (Text):
    ufw allow 19999
     
    • Agree Agree x 1
  3. This way of installating it won't get you automatic updates i think. Thanks anyway for the short version.
     
  4. Would very highly recommend putting netdata behind authentication
     
    • Agree Agree x 1
  5. Actually, that should get you auto-updates by default. If it doesn't, then that's a bug in the installer and should be reported at https://github.com/netdata/netdata/issues
     
  6. Seconded, you _absolutely_ should not allow unauthenticated access to the dashboard unless you're only exposing it on trusted networks. Performance data is actually remarkably useful to potential attackers (it at minimum gives them some insight into what you're running on your system (and thus potential attack targets), as well as possibly feedback on whether their attack is working or not).

    You can find more info about securing access to the dashboard at https://learn.netdata.cloud/docs/agent/web/server.
     
    • Agree Agree x 1
  7. Netdata only supports IP address based ACLs itself, which isn't ideal if you don't have a VPN into your network (this is the superior solution, however, most MC servers don't). You should recommend binding netdata to a private IP, reverse proxying through nginx and using authentication on the nginx side.

    Also, the point of authentication isn't to prevent netdata from being attacked, but to stop attackers from using netdata to know whether their attack is working, and to stop people from enumerating your service easily. An open netdata instance will let me know as an attacker straight away which services you have running on your box, even if they aren't publicly accessible at all.
     
  8. Actually, you should be considering the remote possibility of an attack on Netdata itself too, especially since it has multiple components that need to run with elevated privileges. Security issues in the web server and dashboard are not common, but they have happened before (in fact, we (the Netdata developers) just recently fixed such a bug). The information disclosure issue though is indeed the biggest one to consider, and the general recommendation is to run behind an authenticated and secured reverse proxy if you need to expose Netdata on an untrusted network.
     
  9. Should get a CVE filed off for that - I don't see one
     
  10. I honestly use TMUX to split the screen horizontally with BMON and HTOP, does everything i need :)
    upload_2020-7-28_0-2-22.png
    Note: (this is a single SSH Screen, TMUX allow for multiple splitting, which in my opinion is amazing.)
     
    • Agree Agree x 2
  11. A nice (multi-server capable) alternative would be Grafana as a dashboard with Prometheus + exporters as a data source. You can make such great dashboards!

    Yes I know Netdata is "multi-server" capable. But having multiple Netdata windows, you can switch through is not a real multi-server capability in my eyes.
     
    • Agree Agree x 1
  12. Airee

    Supporter

    Telegraf is also great: https://www.influxdata.com/time-series-platform/telegraf/
     
  13. Yeah. Influxdb is great too. I'm used to Prometheus tho. Never change a running system (as long as there are no upsides of doing it at least)
     
  14. Airee

    Supporter

    Well, there are downsides to Prometheus. In particular, you have to expose a port for every single service you want to observe.
     
  15. Push gateway or just basic service discovery
     
  16. Which makes it easy to debug and even easier to maintain. Plus what @DotRar said.
     
  17. I would modify that ufw command to where it only allows connections from your personal IP instead of being open to the world.
     
  18. Do you know if the program consumes a lot?
    Since he is pulling statistics all the time and putting them on the page, I don't know much about it.