[Discussion] What Configuration type do you use for your plugins?

Discussion in 'Spigot Plugin Development' started by John_Willikers, Jun 5, 2018.

  1. When you are talking about people wrapping things in asynchronous are to referring to MySQL or just general tasks.

    And I can't see how they think asynchronous is faster. What if you need the result right then and there.

    I converted most of my heavy MySQL calls to async and ran into the problem of not being able pull player stats (custom defined) into a EntityDamageEntityEvent quick enough without async and async wasn't going to help this.

    So I made a local file system that keeps a copy of the players stats in JSON and uploads it to the MySQL on logout and downloads the latest MySQL data on login.
  2. MiniDigger


    You have to do it async. You can't do it on the main thread, you would demolish TPS.
    What I do is fetch most data on join and keep it in memory, scheduling a async job to save it to db. For stuff like stats when I don't have them loaded yet, I just schedule an async task, load it, increment it and then display the message. This applies to basically everything. You never need the data at the exact second, you just want to do smth when the data is here.
  3. Well in my case I'm adding a Stat Skill system. So for damage I needed to be able to get that immediately so I can get the damage output.

    Even in the scenario above would waiting on data work? I never tried cause I figured something like that would do better with the data being available asap. and I learned about doing MySQL calls on the main thread the hardway
  4. Sure. But then no other plugin could use that data without explicit compatibility. Imagine adding a new mana stat that any plugin could tap into. Sure you can make your own, and even inject it into the player.dat file, but any other plugin wanting to use that mana stat would have to be coded specifically to your plugin.

    If Bukkit API gave us access to store data to that file (could be same API as item/block metadata we already have) then any plugin wanting to store/use a stat called "mana" could all share it without being dependent on each other.

    This is how forge mods work with many things, including their stored energy machines. If your thing consumes the mana stat, it doesn't need to care how the mana stat got there. Same vice versa, if you produce the mana stat you don't care who uses it.

    Edit: mana is just an example, it could be anything with the right API. Imagine if mcmmo stored it's skill level stats in there.. any plugin would have access to read/edit that data without having to directly hook into mcmmo.
  5. Optic_Fusion1

    Resource Staff

    That is true, i guess someone would just have to do a PR to see if it would get accepted or not, if no, a community plugin would have to do, i mean, if i were to get all the of the client/NMS code related to NBT i could build upon that code to make it easier to use, however that alone would be a few classes, or an update for each NMS if i wouldn't pull the deobfuscated code out
  6. Iirc, there's been a pr in spigot for years. Like most good things, though, it just sits and collects dust until it's no longer mergable (if submitted to paper it might have a chance). A community plugin would be nice, but they don't last (see vault and luckperms incompatibility).
  7. Optic_Fusion1

    Resource Staff

    Eh, kinda not surprised, and i mean, if i were to do a community plugin, as long as it gets used id keep it compatible, hell, i could add a patching system to my server builder and make it a custom patch if i fucking had too
    #27 Optic_Fusion1, Jun 5, 2018
    Last edited: Jun 5, 2018
  8. Optic_Fusion1

    Resource Staff

    So, i finally got around to doing something to make accessing/using the .dat files easier, i mean...i took all of the NBT classes required so i don't have to deal with obfuscation each update (and because spigot related issues) so i might have to do a patch *shrug* but besides that, i have a start, the only thing i really have to do is simplify the process >.>

    Edit: when it comes to the NBT writing/reading it's actually easy as fuck when you have all the files
    Code (Text):

    public static void main(String[] args) {
            try {
                //Loads a .dat file
                NBTTagCompound compound = CompressedStreamTools.readCompressed(new FileInputStream(new File(".datPath")));
                //Sets a string called TESTCOMPOUND with the value of TEST
                compound.setString("TESTCOMPOUND", "TEST");
                //Saves the now edited .dat file
                CompressedStreamTools.writeCompressed(compound, new FileOutputStream(new File(".datPath")));
                //If you loaded a players .dat file this would get their lastKnownName
                //This prints the entire compound to a single string
            } catch (FileNotFoundException ex) {
                Logger.getLogger(NBTUtils.class.getName()).log(Level.SEVERE, null, ex);
            } catch (IOException ex) {
                Logger.getLogger(NBTUtils.class.getName()).log(Level.SEVERE, null, ex);

    #28 Optic_Fusion1, Jun 6, 2018
    Last edited: Jun 6, 2018
  9. My configuration files and storage files are all in YAML. Nowadays I would probably have made the storage files JSON, but at the time I made it I just went with YAML, because I wasn't that experienced yet. I'm not really planning on switching anything over to JSON, users seem to have no problem with YAML and if it becomes a problem later down the line I can always add JSON support.

    The statistics do allow for a MySQL database as storage as well as a YAML file. The user can choose themselves and if the database connection fails the plugin will automatically convert back to using the YAML file. I thought I'd be nice to keep YAML as well since not every server wants to hook up some database and for small servers where you just want to play with your friends I totally understand that.

    Internally there are some default YAML templates for the config and messages files to ensure that, if the user some how ruins it, the plugin will restore the file to a working state. There are some XML files for the GUIs that the plugin uses, but those are never exposed to the server owner (unless they open a zip archive). The framework just loads the files and then I can directly show them to the player without much more manual programming required.
  10. Forgot to take into consideration online players ;) If I'm online and something edits my .dat file manually then it will get overridden on next save/log out. It gets pretty complex when it's not part of the OfflinePlayer object natively in the API. I forked the server with a modded OfflinePlayer that opened up lots of stuff to me (location, enderchest, inventory, etc) and it worked great. This allows the server to handle when data gets saved/loaded so nothing overwrites each other. I'm thinking something along those lines to add arbitrary metadata would be the best option.

    Some of the odd (but not rare) cases would be:

    • Offline data
    • Online players logs off while new data is on disk but not in memory
    • Offline player logs on while data is loaded and modified in memory
    • Etc
    Probably lots more cases to think about that I'm forgetting, but you see where I'm going ;)
  11. Optic_Fusion1

    Resource Staff

    Eh, i knew i'd have issues with online players, if needed i could again as said before add a custom patching system to my server builder which does things like this, hell i could even add patches for prs which are useful but were never touched *shrug*

    edit: I'd PR it for the spigot api, however we all know how well that went with the original one
    #31 Optic_Fusion1, Jun 6, 2018
    Last edited: Jun 6, 2018
  12. PR it to Paper. Every PR I've sent them has been accepted. I feel it will eventually take over simply because of all the new options available to both devs and server admins they keep adding ^_^
    • Agree Agree x 1
  13. Optic_Fusion1

    Resource Staff

    I'll look into it, i'll have to clean it up a bit, it will support offline players by default, however someone will probably have to expand on it to support online players

    Edit: unless someone can find the files required, then i can do it my self
    #33 Optic_Fusion1, Jun 7, 2018
    Last edited: Jun 7, 2018
  14. I use JSON. It's nice because you can directly (at least more easily) serialize and serialize objects with it, as well as store configuration and/or player related data directly in document based databases such as MongoDB.
  15. electronicboy

    IRC Staff

    The horrors of needing fast queries, but then again; a properly setup database would likely deal with such a query at a speed that it really doesn't matter, async where possible, cache/optimize as much as possible for where it's not

    There have been talks about Aikar pulling his persistent metadata API back into paper, modifying NBT isn't the concern, it's "how do we expose this information and provide a way to control it"; I've had some thoughts around this, but Aikar stuff is generally best left to Aikar (otherwise it ends up on my todo list and often tends to fall out of reach due to life stuff)

    Me, personally; I store data in MySQL, configuration stuff I'll generally do in json as it's easier to work with, especially with gson; My only work with stuff akin to hibernate made me want to cry trying to get it to resolve stupid issues, writing queries is just easier and I know what's going on under the hood
  16. YAML for config and SQL (SQLite) for data.
  17. I async query a copy of the players stats on log in and keep a local copy in .json form. I then re upload it on the player logout. So im not technically doing a MySql Query during the OnPlayerDamageEvent cause my server setup was causing small spikes of lag which was pissing me off. (I have an old AMD Phenom II x6 1045T and 16 GB DDR3 RAM 1300ISH MHZ so my hardware could have caused it)
  18. If you're storing it on logout and requesting it on log in, why use a file at all? Just store the object in memory. Sounds like a Guava Cache is exactly what you need for this, using a file is pointless and just introduces filesystem load time overhead
  19. I use Yaml for a simple reason the “user” side that wil use my plugin knows Yaml and it is already on Spigot api you don’t need really much work to create a custom file while to create another file like a JSON file you need to create all the configuration or add on your plugin a libery to do it
  20. MiniDigger


    Gson is already included with spigot. Writing an object to a json file with gson is done with two lines of code (plus a try catch block), so that's not really a concern, it's easier than dealing with yaml (that's why I use it)

    The user side might be true, but when a user doesn't even know (or can't to bothered to learn) how to deal with json I rather not deal with that user at all.