Advices for securing your server/network

Discussion in 'Performance Tweaking' started by andrewromero233, Sep 3, 2019.

  1. Dear spigot community i am starting a small server but no one knows if it can somehow become really big i a short time so i would like to secure it as much as i can.
    I don't need help with optimization, protecting plugins etc.

    I am more interested in hosting everything by myself on a dedi server etc.
    Some of the questions i have but it does not need to be only that are:
    - Should i change ports to mysql, ftp, sftp and server port ?
    - What ports should i open (22, 25565, 3306, others or what not to open)
    - what is the best way to protect your network from overloading (i had a server with 500+ players) bungee setup correctly servers etc but it started to slow down on 200 players, so when it reaches 500 it broke (it was not only one server, it had more lobbies which were spreading players + in one time it had i think 2 bungees(if that is not possible maybe it was something similar)
    - Any other recommendation as i am planing on gathering a lot of youtubers and i can't risk it to start a server which will crash on 200 players

    Thanks to everyone who read and who will try to help without mocking my knowledge

    If maybe this is in wrong section of the forum i would like to contact some of the mods to move it
  2. You state you
    but you are asking for tips on how to prevent your
    ... Maybe edit that :)

    In my opinion, you only really need to change your SSH port as that is the main way into your server.
    If you configure MySQL correctly then people won't be able to access it from the outside either way ( don't bind it to ).
    Use random usernames and long passwords for every MySQL, FTP / SFTP login that you have.

    I don't have that much experience with Bungeecord but I recommend installing as little plugins as you really need.
  3. Finally, some people who actually care about security. I love giving advice on this, so ask away.

    To put it quite simply, don't expose anything to the open world that your users don't need access to. When you run only Minecraft on that machine, and assuming this is with a BungeeCord installation that runs on 25565, the only port you'd want to publicly open without filtering is 25565. This is what your users connect to, so that should be available to all.

    You do not want to expose anything else to the open public, or at least have it filtered or secured in a way. A few examples of common things (including the ones you asked for);
    • SSH; You need SSH, no one else. Treat it that way. It is recommended to setup SSH keys instead of passwords, but a solid password (like, 64 characters, random generated) will do. It is also recommended to setup something like Fail2Ban which will temporarily ban IP addresses that have too many incorrect login attempts. You could also consider changing port, though I personally don't see that as a necessairy thing if it's already well protected. You could also consider to setup an IP whitelist on your SSH. You could do this on either the firewall of the machine (like IP tables), or set it up on your hosts firewall if you have one (like OVH). A note of warning; if you do plan to use IP tables to setup an IP whitelist to SSH, DO NOT SETUP IP TABLES PERSISTENCE. What this will do is permanently save your rules to the disk, and apply them again when the server reboots. This is nice on paper, but if you setup an IP whitelist on SSH, and your IP changes, you'll lock yourself out completely. Without persistence, you can reboot the server through your hosts panel and have the local firewall be reset, giving you the option of at least connecting to SSH again. What is also recommended is to disable direct root SSH access, create a new sudo user, and use that user to login to with SSH.
    • (S)FTP; Again, nobody except you needs access to this. When using SFTP, there's not much to do since your SSH security would secure SFTP. When using FTP, you could also setup Fail2Ban to automatically ban people that bruteforce it. If you can, you could again setup an IP whitelist if you want, but something like Fail2Ban and a proper password should do.
    • MySQL; don't expose this to the public at all. If you have one machine, run MySQL locally and don't expose the port at all. If you have multiple machines that use the same MySQL database, I'd recommend setting up some kind of virtual network or IP whitelist. Again, use proper passwords for everything, and don't use the same MySQL user for everything, and don't use root. Ideally, every plugin or anything that uses MySQL should get its own database, with its own user, with its own proper permissions. If one of these users gets hacked, somehow, at least the damage will be limited to that database and what the user can do. Does a user only read data? Don't give it write permissions then. Does a plugin only write to the database, such as access logging, then don't give it read or delete permissions, and so on.

    The same principles kind of apply to everything; if your users don't need it, don't publicly expose it to the entire internet. It's not an absolute disaster if a few things are exposed (but well secured), but if you can't connect to it, it can't be hacked. Do note that some network setups make this more difficult than others. For example, buying a set of dedicated machines, all with their own public IPs, could make it a bit hectic to setup a bunch of network policies. If you're lucky, you can create a virtual network (like on OVH), and limit access to "exposed" ports to only the servers in the same virtual network.

    A bit off topic, and probably not something you'll be able to get into; Kubernetes. It's what I use in my project, and I'm using the managed service from OVH. It allows me to build a cluster of machines, deploy software in the form of Docker images and setup CPU and memory limitations (which help reduce damage in layer 7 DDoS attacks). Probably most importantly, it's extremely easy for me to internally open a bunch of ports without it exposing them publicly. I can also setup network policities to state which containers are allowed to connect to each other. This is then all handled by Kubernetes, and it literally does not matter where the software is deployed. If I have 10 nodes, with a bunch of software that needs to be deployed, Kubernetes will automatically spread it over the 10 machines, and still allow things to connect to each other. Also note that since I'm using the OVH managed service for this, all connections internally (node to node) are done over the OVH network, and don't go on the public internet. This means that I can create a very simple firewall at OVH stating that every connection not going over 80 or 443 is to be rejected (I don't have SSH access to these nodes). It's a bit off topic, but it's something you could consider when you get larger, as Kubernetes has massive advantages in plenty of topics, such as security. More people should consider using it, especially if you plan to grow and you want a pretty self sustainable network infrastructure that isn't too much work to maintain.

    This all has to do with load balancing. One aspect of security which people often forget about, is how likely your system is to crash and affect a lot of users. Think about it, securing your organisation means securing your processes, your customers, your users. Having the risk that hundreds of players won't be able to connect anyone if *one* particlar service goes down, is a security risk in that sense (see the A of the CIA-Triad, which stands for Availability).

    You may find inspiration in what Hypixel does for their Bungee setup. Though, by all means quite overkill for someone who's just getting started, it is definitely something to consider and take inspiration from. I made a thread a while ago about these kind of things, and @JamesJ (now Admin at Hypixel, or did he quit?) gave me some very interesting insight into how they handle proxies. You can read it here. It's a long thread, but very worth reading! As a TL;DR: They have LOADS of proxies (bungees) available, and they use a round-robin rotating setup where they constantly swap around publicly exposes proxies in groups of 20 if I recall correctly. They swap them every 15 minutes or so, to spread the insane amount of players.

    Speaking of load balancing, there are also different approaches to this. I'm sure there's plenty of guides online on how to do this best, and I am not an expert on this. However, I do know that a few options exist. One is a round-robin setup, which is a DNS based "randomizer" which spreads the load across multiple systems. The way it works is you have a dozen of IP addresses linked on the same DNS record, and your DNS host (if they support it), will send a random one of these IP addresses. This means that if you ping some domain, you'll get a random IP address to connect to. CloudFlare supports it, and supports a very low TTL as well, which means you can quickly take out an IP address when needed, because of the biggest downside of a round-robin setup; assurance. This load balancing is based on DNS, which has advantages because the incredible incoming load is already split up even before it reaches your systems. However, the big downside of this is that the DNS service doesn't know whether that IP address is available or not. You would have to setup some kind of service health checking mechanism to check the availability of all services, and quickly take out an IP if it's no longer available. This would result in a downtime for all users who would have been assigned that random IP, for the time of the TTL (which is 2 minutes max at CloudFlare I think?). So by all means, not a massive impact, but it's something to definitely keep in mind.

    Another option is to have a load balancer in place which will balance network. For example, OVH offers load balancing IPs. These IPs are specifically there to accept a shit ton of load and redirect them over other IP addresses, probably internal ones. These cost money, but I do believe these IP addresses also validate whether or not the service is available, so they're a bit more robust. If you ever get into a situation where you become so stupidly large, like Hypixel, you could use a combination of both round robin and load balancing IPs. Even then, there are load balancing services such as Traefik (which nowadays also supports TCP load balancing, it used to only support HTTP).

    All in all, it comes down to the very "simple" principles of spreading a shit ton of load over a a crap load of machines. At some point, your traffic becomes to big that even a simple load balancer won't be able to keep up, which is where round robin becomes interesting for example. This, in addition with load balancing IPs (CloudFlare also offers a load balancing service) and load balancing services like Traefik or HAProxy. Though, you have to think about where you want to draw the line, because all of this has its price and by all means isn't relevant to most people.

    At 200 players, I think you'll be fine with the cheapest and simplest approach of using a round-robin setup with maybe a handful of Bungee's. However, keep in mind that plenty of people would consider even this to be overkill for 200 players.

    A long reply, but I find this topic very interesting myself, so I'm very willing to help out other people, and even receive some feedback from others.
    • Like Like x 2
    • Useful Useful x 2
    • Agree Agree x 1
  4. you are right in some way but it is not really optimization if it is maybe related to configuration or something not installed well or correctly either way i am sorry if i am little confusing
  5. I've read everything and only thing i can say is that is i am amazed, some of the terms i did not understand but i will try and look for more documentations as soon as i tend to reach more players, if i have issues or trouble with installing or starting that project do you mind if i contact you for help privately ?
    As i wanted to host everything on multicraft and i am new to system administration and linux (Not a complete newb but new to it ) so i admit i will need some help on installing everything and connecting it the right way

    Again thanks a lot as you helped me to understand how large networks handle large amount of traffic