Backups on or off server?

Discussion in 'Spigot Discussion' started by CookieGamerPT_, May 20, 2016.

  1. You may experience problems when backing up my server with it on?
    And what I choose zip or tar.gz? And for what reasons?
     
  2. Tar.gz because it retains things like permissions. You can back up while your server is on, backing up while your server is off can take time.
     
    • Like Like x 1
  3. Thanks!
     
  4. Why create a folder and copy the files with original owner to it?
    like foldername: <date>
    cp -rp <date>
     
    • Like Like x 1
  5. If you compress it then you can automatically upload it to another server for file storage easily.
     
    • Like Like x 1
  6. Do both, keep 1-2 recent backups on the active box so you can quickly restore if needed be, then have the rest offsite.

    As for smaller backups use tar.gz, for larger scale use pigz (multi-threaded).
     
    • Like Like x 1
  7. JamesJ

    Supporter

    I keep regular backups on my local machine, then I backup 2 of those to an external machine every 12 hours.
    If my drives fail/whatnot, I've got a somewhat recent backup that way, however I'm also not clogging up my backup machine either.
     
    • Agree Agree x 2
    • Like Like x 1
  8. Thanks for the explanation.

    Yay, use the find to delete backups older than 30 days. So do not accumulate many backups and will be made up 1 per day only.
     
  9. No need to make it upload to another server, it will lag your server if you have a lot of players on, since it will be writing to another machine (causing network lag and a lot of CPU usage). I run my server off a user, and root user disabled, and another user that has a folder in /home/backups/ that backs up at 1AM every night in a .zip.
     
    • Like Like x 1
  10. JamesJ

    Supporter

    Have fun when your drives corrupt / you accidentally delete it all / your server gets hacked and rm -rf'd :)
     
    • Agree Agree x 2
    • Like Like x 1
  11. I am running off a proxy tunnel and tight firewalls, so my backend is going to be extremely hard to find. & I download backups every 2 days, a drive corrupting will be very rare..
     
    • Like Like x 1
  12. Off-site backups are a lot better.
     
    • Like Like x 1
  13. lool ok. You'd rather take the chance of not having an off-site backup solution rather than having a failsafe just in case something happens, just as (for whatever reason) the node dies????
     
    • Agree Agree x 2
    • Like Like x 1
  14. On our network we shut the server down each day at 8 am for the full backup. MySQL backups are done once an hour while the server is online and treated separately. We only keep the backups of the last 5 days on our nodes and on the server itself we have a daily backup for the last 30 days, a weekly backup for the last 60 days, and a monthly one after that :)

    The full backup works like this:
    - server shutdown
    - server files get copied to an extra folder together with the hourly mysql update for that day
    - server start
    - compression of the content of that extra folder with 7z on maximum compression + aes 256 bit encryption
    - after compression this file gets copied to our backup archive on the same server and another directory where those 7z files get synced over BitTorrent Sync with several backup nodes and also uploaded to an ftp server owned by me

    Pro:
    - maximum security due to several backup nodes
    - due to encryption and read only access we can also share those backups with friends without having to worry about them being able to clone the server :D
    - fast reupload since the backups are distributed over BitTorrent and every single node we use has at least 10 MBit upload

    Contra:
    - high traffic (about 40 GB per day due to our backup size)
    - higher load while the backup is done (actually no problem since we use a raid system, 64 GB of ram and an Core i7 3930K (6 cores & 12 threads)
    - takes some time to set up since you need some scripting skills and linux knowledge to do it
     
    • Like Like x 1
  15. How do the 06:00 every morning backups, there will be a little amount of players. You then use ZIP and tar.gz not? And who preferred the zip?
    Off site? I did not understand
    Really, you have a well-designed backup system!
     
  16. He means storing the files on a different server (not the same one that the Minecraft server is being run on).
     
  17. Thanks a lot :)
    It took us some time to develop it but now it runs perfectly well.
     
    • Like Like x 1
  18. Once this is setup, this is a fantastic system!
     
  19. thanks. It works perfectly well and if someone of you would like to get a system like this i might be able to help you setting it up :)
     
  20. Really? I'm actually in the process of trying to set something like this up, on a new dedicated server I've purchased. :)