Stuck trying to figure out how to automate with Git, CI/CD and Docker

Discussion in 'Systems Administration' started by Miniroid, Jul 5, 2019.

  1. Hello, so for the past few months I had been trying to automate my server maintaining process.
    I am trying to implement the following flow to my server


    As you can see, I am looking in having one git repository with all my server and plugins configurations.
    I have done some research and am still fairly new to docker. I also found this docker image by itzg. However, there are a few things that I am unsure of.
    1. Is it advisable to use a custom docker image (the one by itzq), or should I be building my own docker image?
    2. How do I test to see if server doesn't crash with latest configuration? While ensuring that the test does succeed if server starts successfully.
    3. Currently what I have tried is using a docker compose yml file based off itzq docker image. I have a repository that contains the plugin folder and the docker compose file, which works by running docker-compose up -d. Upon changes in plugin folder, I will just restart the container. What would be the build stages on Travis for this setup? And is this advisable?
    As I am extremely new to this, whatever I am asking may not make sense too. But I would appreciate it if someone could guide me on this. Thanks!!

    P.S I am running both the production and staging servers on Google Cloud Compute Engine if it helps.
  2. 1. Build your own docker image from scratch.
    From jre-8-alpine
    Add your spigot jar
    Add your entries

    Additionally I recommend using a volume for your plugin directory. A third thought you need to consider is world saving. A world is reset when container is stopping so the world needs to be saved elsewhere e.g volume.

    2. Automatic? Probably by observing the jvm and defining how it should behave. Currently not familiar with that.

    3. Building the image is probably build/package
    After that deploy staging.
  3. MiniDigger


    I would build my own image, allows for more control and it's more light weight.
    So like from openjdk-11, add paper jar and plugin jars, add volume for data stuff, add startup command and that's it.

    For your integration test I would just startup the server and connect via a bot of some kind and make sure that goes thru.

    If you want proper CD, you need to use docker swarm, then Travis can actually deploy your docker compose file.
    Your stages would be something like build image, integration test, deploy image to repo, deploy to staging or smth like that.
  4. Ah thanks! These are extremely detailed and helpful in helping me to understand.
    1. For your plugins jar files, do you commit them into repositories too or does docker download them on the spot? If so, how do I deal with situations such as paid plugins?
    2. Is there a need for usage of Docker Swarm or Kubernetes if I am only running a single server? What are the benefits of using it?
  5. MiniDigger


    I personally use docker swarm for my private vps too, it allows me to easily deploy stuff from the outside, in my case from gitlab-ci. K8s are a single server is overkill.
    Ideally you would add them into your docker file and download the jars from your maven repo. Having binaries in git is meh.
  6. How do you make it so that the git repo updates the plugin data volume on Docker?
  7. MiniDigger


    It just invokes a ci pipeline that will clone the data into a folder and add it to a docker image.
  8. Hmm, is it possible for you to add me on Discord or something and guide me on everything? If you are free that is. I would really appreciate the help
  9. MiniDigger


    I can't access discord at work, so thats a no sadly. you can come join irc and ping me tho and ill reply whenever I have time
  10. That works too! I will ask you questions from there. Thanks!