Resource Automate Bungeecord Proxy with Docker Event Listeners

Discussion in 'BungeeCord Plugin Development' started by Muehre, Aug 8, 2018.

?

How do you host you Network?

  1. Virtualized with Docker (kubernetes, swarm, compose etc.)

    6 vote(s)
    50.0%
  2. Different Virtual Machines (Vagrant etc.)

    1 vote(s)
    8.3%
  3. Multi proccessing without Virtualization (Screen / detached Process)

    3 vote(s)
    25.0%
  4. 3rd Party Host

    1 vote(s)
    8.3%
  5. Something else

    1 vote(s)
    8.3%
  1. Hey Folks!

    I developed a plugin for the Bungeecord Proxy and I am thinking about to open source it and give the Community access ;)

    Yes it's about Docker and Clouding Technologies. (If you do now know what Docker or Visualization is, you should stop reading now).

    I got the basic idea after using docker-gen which is a reverse proxy that automatically generates a nginx config for containers based on their environment. i.e starting a container with VIRTUAL_HOST=mydomain.tld will add an entry + upstream to nginx using the IP from the connected network. (It also supports letsencrypt and much more, worth a look!)

    My basic idea was: Why not doing the same for the Bungeecord Proxy?

    My plugin allows the following:

    Docker event Listener

    By running i.e.:
    docker run -d -t -i \
    -e SERVER_NAME='lobby-1' \
    -e SERVER_PORT='12345' \
    -e SERVER_TYPE='spigot' \
    -e SERVER_MOTD='A Minecraft Server' \
    -e SERVER_GROUP='lobby \
    --name minecraft-lobby-1 itzg/minecraft-server


    The plugin:
    • automatically listens to docker events and add servers to BungeeCord as soon as a container starts.
    • is removing servers from BungeeCord as soon as the container gets stopped.
    • is able to listen on healthcheck actions like "health_status: healthy" or "health_status: unhealthy" to add and remove servers
    • inspects all running containers on start to add them initially

    Connection Balancing (Don't want to call it Load Balancing because in theory it isn't)

    Configure server groups:
    • Define default server groups (like lobbies) which players will get connected to
    • Define which server groups a player can directly reconnect to and which not.
    • Define force_hosts for server groups.
      (like eu-lobbies = eu.mynetwork.net, us-lobbies = us.mynetwork.net)
    • restrict access for server groups
    • Define group Traffic rules (balance, random or fill-up strategy)
    A server will get assigned to a group based on his environment variable (i.e SERVER_GROUP=lobby)

    Last but not least
    Plugin Messaging

    Using Bungeecords messaging system to notify Plugins with server data and server list.
    This would enable automated server selector menus which are always showing up available servers.

    Message listeners:
    Imaging spawning new Minecraft instances by clicking a button in minecraft which would cause docker to run a defined preset.

    Still in Development: Auto-Scaling

    Configure auto-scaling to automatically create new servers if certain condition is reached (like 80% players slots are taken)
    Works out of the box with scaling system like swarm.


    I am looking forward to get some Feedback and also ideas!
    Everything is highly configurable even the names of environment variables.
    If I get the feeling that I am not the only one who dockerized Minecraft I am happy to share and open source this plugin.



    Edit:
    Pre-Release: https://www.spigotmc.org/resources/dockerizedcraft-container-management.59693/
     
    #1 Muehre, Aug 8, 2018
    Last edited: Aug 10, 2018
    • Like x 1
    • Informative x 1
    • Friendly x 1
    • Useful x 1
    • Creative x 1
  2. Okay, it's official, I think you're very probably a god. That Auto-Scaling is a dream for any half decently sized network.

    This + AWS = Dream life.
     
    #2 Potters, Aug 8, 2018
    Last edited: Aug 9, 2018
  3. Looks cool. Funny enough we at our own server are looking into the exact same thing right now where we're building a fully automated, demand based auto scaling cloud. The idea is that we'll never have more machines than the network needs to host the game and the amount of players on it. Currently the first stages we'll apply this to is to our content team's development environment.

    The idea behind is that with some magic, we buy a server through some the hosting company's API, turn that server into a Kubernetes node/slave (whatever it's called), deploy a private development environment on there, and give limited access to its files (mainly the configuration files). This means that when a content team members needs the environment, they can request it, something automatically buys a server, turns it into a K8S slave, deploys a pod on it (a few containers), and boom. When the guy is done after a few hours, the server is removed again and we paid for a server for no longer than 3 hours that day. Ideal.

    I'm happy to see more people get into this kind of stuff (Docker, K8S, Vagrant, etc). More people need to realize that Minecraft is nothing more than a Java process. You can literally take it anywhere, as shown here. I would happily contribute my ideas or even code to the project if it's open sourced. We already have our setup planned out and maybe we too can share some of our details with the world.
     
    • Useful Useful x 2
    • Friendly Friendly x 1
  4. @MrDienns I agree :)

    The proof of concept is actually done. I've set up K8s with 80 running instances working like a charm.
    What I defenetly still need to try is to solve the networking between multiple pods. I will look into it within the next days.
    Did you setup Ingress to load balance multiple bungeecord instances?

    @Potters I am also looking forward to play around with AWS. I could image it's awesome to scale up by using some micro instances during user peeks and pay the hour based tarif. Like Saturday afternoons. Paying 0,0116 USD per Hour/Instance sounds like a good deal comparing to keep 2 Root servers idling the whole week.


    I will work on the final version within the next days and let you know when there is something to try out ;)
     
    #4 Muehre, Aug 8, 2018
    Last edited: Aug 9, 2018
  5. Ok I made the decision that the Auto-Scaling Plugin gets an own project as it has not relation to the container inspector at all.

    I will now create a basic Setup Tutorial containing a sample docker-compose.yml and publish the Plugin.

    Greetings
     
  6. Why not just use something like kubernetes to manage your docker containers?

    (I personally just started looking into kubernetes a day or two ago, and it seems like something that could be very useful for server management)
     
  7. It's the best and everyone should be using it. There's this video on YouTube which is a 5 minute introduction into Kubernetes where they quick, but fully explain why one should use it.
     
    • Like Like x 1
  8. If you're talking about the following video, then yea its definitely worth a watch as it explains it extremely well and gives you a place to start learning kubernetes/docker if (like me) you have 0 experience with them.

     
  9. First of all: Kubernetes is a platform to manage containerized applications which offers some on top functionallity (Pods, networking, load blalancing with i.e. Ingress etc.). It does not matter if you use Kubernetes, AWS, Docker Swarm or native Docker. Everyone should decide this for himself.
    If you are running a single node root server Kubernetes is not offering much benefits comparing to swarm or others as you wont be able to keep certain services redundant.
    The decision is not that easy that a 5 min Video about Kubernetes will make you a Cloud Engineer. I don't want to act as the bad cop but let's try to stay focused and please don't join this discussion if you are actually not able to contribute to it.

    This plugin is about Docker events (It support Kubernetes, AWS, Swarm etc. as they are just wrapping Docker and are working with the Docker socket as well).
     
  10. But back to K8S: Should be quite easy to make it work by using the tcp docker socket host and leave SERVER_NAME environment variable empty (it will use container id /container name then) :)
     
    #11 Muehre, Aug 10, 2018
    Last edited: Aug 10, 2018
  11. MiniDigger

    Supporter

    I only glimpsed over the resource page and I got stuck on the config for registry credentials. Why would it need those? Does it have the capability to deploy images?

    Being heavily invested into the docker ecosystem myself this looks like a fun project, but I took a different approach. Instead of listening to docker events I plan on taking control and doing the orchestration over multiple worker nodes myself. I feel like that allows for way more flexibility.
    The current plan is to have a raft cluster of manager nodes (so that a manager can fail without taking the network down with it) and then having multiple worker nodes, one for every VM or bare metal server, which then execute tasks based on a task queue that is distirbuted over the full cluster. A task would be "run a bedwars server with this config". When one worker goes down, it can no longer confirm that it's tasks are running and they get put back into the task queue so that the remaining workers can run those servers. That way you can also implement draining of workers if you need to up hardware maintenance on a node or if you want to update a node or smth.

    Anyways, way too off topic, I will be lurking in the shadows and see how your approach works out for you :)
     
  12. @MiniDigger Yes, I am planning to implement an API to allow the creation of containers based on images. I already added the registry config which is obsolete atm.. My basic idea is that every server type (like bed wars) is an image and configureable by environemnt variables (sql host etc.)
    It would be easy to spawn new instances that way. The Problem I see here is to support all of the Container Managers out there. Lot of work to do :p
     
    #13 Muehre, Aug 10, 2018
    Last edited: Aug 10, 2018

Share This Page