Load all playerdata on server start

Discussion in 'Spigot Plugin Development' started by Hex_27, May 13, 2017.

  1. Is it a good idea to just load all playerdata while the server is starting to objects? Or is it better to load each player when they join? Because now I'm facing a problem where players join, and the data doesn't load fast enough
  2. What/how much data are you loading? And also, how many people do you plan to store? Because your plugin will eat up memory like crazy if you have a large amount of people's data just stored in memory.
  3. Loading everything on start will probably eat up too much ram, Id advise switching over to mysql or mongodb, can drastically make this faster.
  4. Welp you could freeze the player while he's joining. As far as i know, im not too sure though, you can delay the joining process a bit.
    You could load all data at start, but you would have to do performance and balancing checks.
  5. How are you storing the player data? Everything for everyone is stored in one giant file? If so, make a new file for every player. It will drastically improve save/load times (even to the point of actually being faster than using an SQL/DB approach in most cases).

    I know a lot of people dont think this is a good idea, especially server owners that say "have you ever tried to FTP transfer 100k+ files?" But trust me. Theres a reason plugins save data like this. Its fast.
  6. Hes probably talking about the server locking up for a split second while a file is being opened, happens sometimes if theres a ton of files in a folder and it has to look for the players file (can also be fixed by moving this away from the main thread), simply freezing the player wont help much :p
  7. Perhaps 10k players

    It's already mysql

    I could try freezing.
  8. Move the database to the same machine (or close), the response time is likely killing you. (If you request something from a server 80ms away, it takes almost 1/5s to load the data)
  9. In that case with 10k players, you would definitely not want to load all data at once and store it, that would require huge amounts of RAM that will just be eaten away from objects. Freezing players like what @GoRoK suggested, and/or find ways to optimize the data grabbing system.
  10. electronicboy

    IRC Staff

    for MySQL, it's more about optimizing your database and your queries. e.g. fetch data using JOINs and layout your tables properly to reduce the number of queries you need to make. a bad MySQL setup can have *massive* performance impacts over a properly setup database schema
  11. I'm honestly just speculating, because there's a issue where somehow player data is null when they join, so all I can assume is that the data isn't loaded. But in reality the sql machine is right next to the server machine
  12. I'm hooking into another plugin for database stuff, so I can't quite manage the micro things
  13. It depends of what kind of data it is, if it's just players data then you should only load data when needed, if you're talking about a plugin like Factions where you need all the Factions loaded then you might want to load it all onEnable, anyways i would load the data async or using event like
    AsyncPlayerPreLoginEvent to load it directly (the event is async so it doesn't matter), you can also create a dummy object and then load the data into that object so the players data won't be exactly "null" but "empty" while you load it. Also with only 10k players you shouldn't have any problem with MySQL at all, i had way more players in a SQLite database and it worked just fine.
  14. What about in a json database? And yea it's something like factions, but it's the player's information instead of the faction object
  15. Then it would depend if you actually need that data loaded, if the plugin can work just fine without the players data being loaded in memory, then don't load it, as i said if you do really need some kind of data loaded, you may for example need to access to basic player's data like their "UUID", "Name" or stuff like that, then you can load a dummy object with only the basic data you want, and once the player does actually connect you load the rest of data so if the player isn't loaded you don't load all the data using extra memory.

    Once you have a good database system it doesn't really matter what kind of database you do use (FlatFile, SQLite, MySQL) you should be able to implement any kind quite easy, so if it's a Json database i would personally have a file per player, i wouldn't use a single file, i think using a single file is better on the long run, also it works better if you want to access a specific player data.

    It is up to you to decide what you want to use, but you have to think about what suits you the best in order to take a good decision.