Solved Delayed retrying task

Discussion in 'Spigot Plugin Development' started by SteelPhoenix, Apr 16, 2019.

Thread Status:
Not open for further replies.
  1. I have a method (called async) that returns a boolean value (the method executes a database query). When it returns false (unsuccessful) I want to wait a certain amount of time and then try again until it returns true (success) or a certain amount of attempts have failed. Once this loop is done I want to execute some more code based on the final result, but I don't know how to do that.

    Code (Java):
    // Example:
    public IData getData(UUID uuid) {
      boolean canLoad = ...; // Where I need to query until success or a certain amount of retries
      // Returning a different object depending on the result above
      return canLoad ? new LoadedData(uuid) : new UnloadedData(uuid);
    }
     
  2. That will become tricky this way. You should really use either callbacks or fully do things asynchronous. For example, if you have a command to update a database table, which must be done async, you could simply choose to start an async runnable from within that command. Inside that async runnable, you could call your database activity. Working with multi threading is not the easiest thing, you cannot just do what you (from the looks of it) want to achieve.

    Can you give a bit more context? Perhaps I can give some better advice on how I would tackle this.
     
  3. I have some per-player data stored on a database. When a player's data is requested the data gets cached from the database to an object containing all the data, and once the player leaves I push all updates back to the database. To prevent data from being desynced I also store a boolean that indicates if the data is in use somewhere (think about multiple servers pointing to the same database).

    When requesting data (#getData(UUID) method (which is always called async)) I check if the player's data is already cached and return that (simple Map). If not, I cache the data and put it in the map. I want to check if the data is in use before caching and if it is wait a bit and try again.
    If not in use I want to put cached data in the map.
    If unsuccessful I want to return an empty object (and not put it in the map) that doesn't push updates (the data objects have a load/save method, the empty object doesn't do anything when calling these methods). This also prevents players from using some functionality from the plugin.
     
  4. Interesting, but probably a very common problem. Have you tried using database frameworks like Hibernate? They tend to take care of these kind of things. I know Hibernate also has its own caching mechanism. You'd have to look up if you can forcefully reload the data from time to time if that's what you want. It uses JPA, which will serialize/deserialize your data into Java objects, making things a bit easier to work with.

    I'm not too experienced with solving these kind of issues due to the mechanism that I've always used for my project. We use a fork of MassiveCore, which is an open source core plugin. The database setup works quite simple and straight forward. There's 3 parts to it. There's local "cache", a poller and pusher. The local "cache" is actually just a full blown copy of the database stored in-memory (I want to put some restrictions on this soon, to only load data of players that are online). When we load or update player data, we only talk to this "cache", and never directly to the database. Meanwhile, there's 2 separate threads running; the poller and pusher. They are responsible for keeping this "cache" synced. It's basically just two threads in an infinite while loop which constantly check if 1) the local "cache" was modified, and 2) the remote database was modified.

    In case the remote was modified, it will pull that data and put it in the local "cache". If the local "cache" was updated, it will push those changes to the database remotely. The massive advantage to this (pun intended) is that none of our plugin code ever directly invokes the database. This means that we can always, on the main thread, load and modify player data without any performance impacts. This is important because otherwise you'll have issues with things like event handlers and needing to cancel an event based on the state of some database field. In fact, I believe this setup is better performance wise than constantly making callback threads or asynchronous runnables.

    Perhaps you can use that as inspiration to solve your problem. The project is open source so you can simply look at the code if you want to, though it's quite massive (again, pun intended). I find it a bit difficult to solve an issue like this if I don't have full access to everything, but maybe the above can give you a few ideas. If you need more details, feel free to PM me or add me on Discord, or simply ask here.
     
  5. Maximvdw

    Benefactor

    That will cause data races. Use transactions such as Hibernate/Ebeans,... those can retrieve snapshots of the data so you are certain that you do not loose data
     
  6. Maximvdw

    Benefactor

    The idea with transactions is that you "cache" changes locally, then push them to the database in one go.
    I understand that you want to keep a local cache per server, contacting a database every time is not really good... but you should
    be careful with this.

    It would help to get some insight in why you want to edit the same data on two servers. (Or more importantly, why it needs to happen in the same row)

    Imagine the following scenario, you want to keep track of how many seconds a user plays on a server/network.

    1) Identifying servers:
    The first solution is to identify the servers and keep track of how many seconds the user plays on each server. This not only prevents data races between servers, but even adds extra features.

    2) Keep track of changes, not of data
    Instead of simply "changing your data", you keep track of the changes between the syncs. This is how big sites like youtube/twitter work for storing likes/views/subscribers (thats why they sometimes jump up and down). Basically every server synchronizes with the database every X minutes. After the sync, it keeps track of changes the user made to the cache (+ 100 seconds online, +3 ores mined ,.) and simply tells the database to do +100 and +3 to whatever number is on there.

    3) If you really need up-to-date data. The best is to not cache it. But in most cases you can always find a better solution. A player can only be on one server at a time, this should make it way easier to find a better solution
     
    #6 Maximvdw, Apr 16, 2019
    Last edited: Apr 16, 2019
    • Agree Agree x 1
  7. I probably approached my problem *completely* wrong, as I have never attempted doing such a thing before, but when a player switches between two servers I want to wait with caching data until the pending push on the old server is finished. The data stored is a String, an int and an object serialized to JSON.

    Edit: Only the string needs to be accessed often.

    If I am doing this completely the wrong way let me know ok and perhaps post an alternative solution :d
     
  8. Maximvdw

    Benefactor

    any reason why you do not store the json object as columned data?. What do the string and int represent? I assume the string is the player uuid?
     
  9. I do have a bit of experience with exactly this, but it's not all sunshine and rainbows. You've probably experienced that the player joins the new server faster than it leaves the old one. That's I believe how things work in BungeeCord; it tries to connect you to the new server before making you leave the old one. I can understand where your issue comes from. That's why it may be a good idea to look at my message posted earlier, since it continuously keeps the database up to date while also having a local "cache". Should be the best of both worlds in terms of stability and performance (in my opinion).
     
  10. Maximvdw

    Benefactor

    Assuming the json data is a black box of whatever data it is: The best solution would be to invalidate the cache of your other servers when you actually change something.

    Bungeecord already uses a well established protocol for sending messages to other servers: https://www.spigotmc.org/wiki/bukkit-bungee-plugin-messaging-channel/

    You do not want to use that messaging protocol for data. But you can use it to say to another server "hey, IF you have cache ... do not use it.. cuz I just changed it" (1). Its a simple, short message. With that message, the servers know that their cache is outdated (event based instead of Poll based like @MrDienns ). It would be a very efficient method that is very flexible. You can invalidate caches whenever you want (saving it to the database, changing a local cache,...), you can even use it to ASK other servers if they have altered cache instead (2).

    To give more details we really need more info on what you are storing.. is reading more important than writing ,...etc

    (1): Scenario: Server A saves data to the database, sends a message to all servers with "User A's data has been altered", All other servers now know that they need to update the cache

    (2) Scenario: User A joins Server B. Server B wants to load his data. He asks: "Does anyone have an unsaved state of User A". All servers will reply, but they know someone else needs the data. so before they reply yes, they save it to the database if they have uncommitted changes

    Scenario (2) is the best if you have the same issue as MrDienns. It is ask based and will always result in up-to-date data. Unlike MrDienns, it only contacts the database when needed. However scenario (1) also has advantages if you have multiple servers editing the same data at the same time
     
    #10 Maximvdw, Apr 16, 2019
    Last edited: Apr 16, 2019
  11. I keep track of quite a bit of player data that is stored in an SQLite database. I have a class that retrieves this data from the database as necessary and stores it in instance variables. This class is created when a player logs in and cached in a hashmap. When a certain data value is requested from this object that hasn't been read, it reads from the database once and stored internally. All subsequent reads are from memory.

    For values that are infrequently modified I just write them to the db immediately if they change. For frequently modified values I just change the instance variable and write the changes when the player logs out. I also have a task than runs every 5 minutes to write all these memory values to the database.

    There may be more efficient options, but this method is quite simple and doesn't involve async threads or synchronization. I've never noticed any kind of server performance issues from doing it this way.
     
  12. The JSON is a List holding objects. I serialized it to JSON so I could just put it in one column and (de)serialize it easily.

    I like your idea of an event based system though I'm not sure how to implement it.

    I'm making a titles plugin with personal titles visible in chat. I have the following interface:
    Code (Java):
    /*
    * The Saveable interface includes a save() and load() method.
    * load() currently gets called when initializing a new data object.
    * save() currently gets called when the player leaves and every 30mins on a timer.
    */

    public interface ITitleData extends Saveable {
        // The player's UUID.
        public UUID getUniqueId();
        /*
         * Getting a list of the player's current custom titles.
         *  => Only used on certain commands and called async
         */

        public List<CustomTitle> getTitles();
        public void addTitle(CustomTitle title);
        /*
         * Getting the title currently equipped
         *  => Used everytime the player chats (and in a placeholder) which is why I want it cached
         */

        public String getCurrentTitle();
        public void setCurrentTitle(String title);
        /*
         * Balance and related methods.
         *  => Only used on certain commands (and in a placeholder) and called async.
         */

        public int getCoins();
        public void setCoins(int coins);
        public void addCoins(int coins);
        public boolean removeCoins(int coins);
    }
    I don't want to change much or anything about this to keep it working with other implementations for different means of data storage.

    I'm thinking about implementing it the following way:
    - As the coins and the custom titles aren't requested often I can just keep those synced to the database at all times and not caching them.

    - When initializing a new data object (and thus loading data) check through plugin messaging if there is an updated current title that's not pushed to the database yet and if so request it. When calling #setCustomTitle(String) broadcast this through plugin messaging and have listening plugins change their cached title.

    Is this a good way of approaching this?
    Is it true that with this approach data is kept in sync between servers?
    I do not need to change anything with the #save() (just an update query) functionality right?
     
Thread Status:
Not open for further replies.

Share This Page