1.15.2 Multi-Tick Synchronous Tasks

Discussion in 'Spigot Plugin Development' started by CrypticCabub, Jan 25, 2020.

  1. Not sure if this is the right place for this but here goes.

    I am currently working on some minigame code that would like to be able to paste .schem files into the minecraft world automatically without introducing lag. I have already written my own libraries for handling the .schem format which have been working well for several other minigames I have written, but having run into the problem of large schematics stalling out the main Minecraft thread before I am now strongly considering writing a library scheduler to manage these synchronous tasks in such a way that will not exceed the allotted 50ish ms per game tick.

    My idea to do this would be some form of SyncRunnable interface similar to Spigot's existing Runnable interface that has a bool run() method that is intended to run small (1-2ms) steps of the task before returning a taskComplete (true) or taskIncomplete(false) result that a scheduler can use to decide if there is enough time left in the current tick to allow the task to run again or if it should wait until the following tick to run.

    My question would be if there is already a better way to handle dynamic schematic pasting within Spigot (think generating or cleaning up arenas on the fly). I know FastAsyncWorldEdit is a thing but I have been trying to avoid creating a dependency upon it when it has often been slow to keep up with the latest Minecraft version. If such a thing does not exist, I am also curious as to if Spigot has some way of finding out how much time is left in the given tick to execute tasks and if such a SyncRunnable interface would be something that should be added to the Spigot scheduler itself rather than my building a separate runner just included in my own personal library.
     
  2. Personally I would implement arena generation and cleanup via a dedicated world that is never saved and unloaded then re-loaded as needed.
     
  3. That is definitely an option, tho I never properly figured it out when I looked at it last time. For one of my minigames, I have small destructable arenas that are generated on the fly by the matchmaker as needed. Players don't join specific arenas, they join a matchmaker that chooses a random arena from a given map pool and then I kick off a task to generate the arena in the background while players are going through a character selection step. By doing it this way I have no limit to the number of games that can run at a time because arenas are generated dynamically as needed. As for where to put them, I have an arena-world manager that breaks the world down into partitions that can be assigned to an arena and reclaimed later when the match is done. The intent being to reduce the overhead of reloading arena worlds constantly and keeping the server as lightweight as possible.

    So I guess the first question would be have I actually found a less optimized way to handle the problem compared to, say, copying world folders from the plugin folder and then loading/destroying them later?

    Then my SyncRunnable (or maybe DynamicRunnable?) idea. The reasoning being that such a runnable interface is only useful if all plugins with such tasks use the same scheduler. I could implement such tasks the way I did with the minigame I mentioned above by just using sync repeating tasks and keeping the usage down, but that option does not have any protection against too many of these same tasks being started at the same time and overrunning the tick allotment (if my task intentionally limits itself to 10ms, I only need to kick of 5 of them to overload the server -- some sort of central manager is needed to protect against that)
     
  4. Paper has ServerTickEndEvent, which exposes information about the current tick duration and the time remaining before the next tick should start.
    I believe it would be the same as scheduling a task repeating every tick, but with that information available.
    For example:
    Code (Java):
    while (event.getTimeRemaining() > 10L * 1000L * 1000L && run()) ;
    Or:
    Code (Java):
    int times = (int) (50D - 10D - event.getTickDuration()) / /* expected task execution time: */ 5;
    for (int i = 0; i < times && run(); i++) ;
     
    #4 System, Jan 26, 2020
    Last edited: Jan 26, 2020
    • Useful Useful x 1
  5. Yeah, something like that. Though I would do a while loop that keeps checking for each iteration (just in-case a naughty task oversteps its expected execution time).

    Is this something that would be worth looking at adding into spigot directly? I'm not too familiar with the inner workings of spigot itself, but I would suspect that Spigot's internal scheduler for the other repeating tasks would be the best place to know how much unused time is left on a given tick (only spigot can really know how many tasks it has not yet run for a given tick)
     
  6. To me it looks like the code displayed in that guide is not thread safe.

    What about unloading chunks over multiple ticks instead of whole worlds? Would that be enough to keep the map reset lightweight?
     
  7. Thanks, That is pretty much exactly what I had in mind though the abstractions are slightly different. What I was wondering though is that if this is a common problem should it be implemented into the Spigot scheduler directly. I was about to code something like this up for my own projects and then realized that its effectiveness goes up significantly if it became part of the core spigot library

    It doesn't need to be. The common problem I am referring to is when we have a long-form task that must be executed on the primary minecraft thread thanks to many in-game physics actions not being thread-safe themselves. if I could just kick-off a background thread to do my arena setup like I can do to import/export databases into text files this would be soooooo much easier

    I am not familiar with the details of the world and chunk loaders. It may be due to my lack of familiarity but abusing save state like that makes me feel a little uneasy when it comes to maintaining map integrity and simplicity of setup. How do I get the initial arena caught in this time loop and how do I ensure said time loop doesn't fall apart on me? I currently have a problem with cleanup after pasting an arena into my arena world (sometimes water has flowed out of the arena and cleaning that up requires doubling the size of the cube I need to clean up), so resetting the chunks to air does sound like a much more efficient solution for cleanup. Are there any references I can use for chunk-by-chunk manipulation that doesn't require deep NMS (I can use NMS if I must, but avoiding possible plugin breaks is preferred)?
     
    • Agree Agree x 1
  8. I mean, the framework itself displayed in that guide does not seem to execute workloads on the server thread.
     
  9. Then ask @md_5
     
  10. What would be the best way to do that?
     
  11. ??? That's the whole point of the thread, otherwise why not just use runTaskAsynchronously?
     
  12. WorkloadThread calls Workload#execute(). WorkloadThread implements Runnable, therefore I guess it is run asynchronously. Workload implementations call Bukkit APIs that are not thread-safe.
     
    • Optimistic Optimistic x 1
  13. ??? The whole point of the tutorial is to split up Bukkit API dependent tasks over multiple ticks, with cool features such as checking computation times and not going above a certain limit per tick. The whole point of that resource was to be able to create big ***Bukkit API reliant tasks (which are NOT thread safe)*** that don't create a large amount of lag, otherwise again, what's the whole point when you can just run it on a seperate thread (which will NOT work with Bukkit), the tutorial is to chunk a bukkit reliant task into parts.

    The whole argument that because it implements runnable it isn't thread safe is completely incorrect, just because something is a runnable doesn't mean it's not run on the main thread. It can be run on anything, it's completely irrelevant.
    For example:


    Code (Java):
    class RunnableA implements Runnable {

        @Override
        public void run() {
             // do bukkit stuff here
        }

    }

    class MyPlugin extends JavaPlugin {

        @Override
        public void onEnable() {
            new RunnableA().run();
            System.out.println("Runnable has finished executing!");
        }

    }
    It's still running that code on the main thread, if you have ever looked at a stacktrace you would know that when one method calls another it's callback is stored in the stack, hence it's called the stacktrace, and when that piece of code is finished executing it will keep moving on with the other code that's in the method. Hence here onEnable() gets called on the main thread, creates an instance of RunnableA, runs it, and then keeps moving on, there's no new threads spawned, no new tasks made, just simple code being run.
     
  14. TeamBergerhealer

    Supporter

    In light cleaner I solved this by writing everything in such a way that there is a main tick() function, or repeated function, that takes a <5ms time to run. Then its a matter of looping tick() while the amount of time spent (currentTimeMillis()) does not exceed the remaining tick time on the server.

    For measuring the %tps already spent some people already gave some answers. I do believe it is a bit unreliable to measure it, and youre better off configuring some allowed amount of ms in your plugin configuration. Far too often do servers already run at 110% tps, and your task would never run under those conditions, or take ages. If you're fine with that, you can measure and schedule accordingly.

    So first start thinking of your schematic pasting problem as something that must be done in tick() steps. For example, copy it on a chunk by chunk basis. I dont know if worldedits api allows for this, or if you will need to split your schematic up into portions for it to work properly.
     
  15. I have a custom Schem class that can read in a /schem file and turn it into an iterable of blocks. I have code already that breaks the task up into a series of block changes and my question was more about the scheduler itself. Currently I have one built in to my plugin that I believe is set to keep execution time under a configured value (default of 10ms I believe). I have just been studying thread scheduling algorithms in uni recently and it got me thinking as to if a more advanced scheduler would be possible to make such scheduling more scalable. That said, my code is currently using the only slightly better than awful nms setblock method (through an abstraction). I keep seeing reference to being able to modify entire chunks as a potential way to dramatically reduce server overhead (even if the calculations are a little more complicated at first). How exactly does that work, and is it also NMS-only?
     
  16. I might not be correct on this one, but don't see how you could use an asynchronous task here and have it anything but thread safe.
    for example:
    Code (Java):
    for (Material material : schematicBlocks) {
        Bukkit.getServer().getWorld("world").getBlockAt(location).setType(material);
        // This does not modify the block on another thread.
        // The other thread makes a call to the main thread when it is available.
        // The main thread then retrieves that block and changes it's material type.
    }
    I believe that is how this would work, and you will not have any collisions, because the only thread capable of changing blocks around is the main thread, is it not?
    Worse case scenario you surround that with a try catch, and if it fails because the main thread was accessing the block, in the catch you can add the failed ones to a list a blocks you need to try again on. If the main thread can not read the block because the other thread is changing the block, too friggen bad it was getting changed anyways.

    Honestly just use an async task, easy, efficient, and I don't see it causing issues. Hey it's even built right into the Bukkit API...

    EDIT
    Forgot to mention if you really need thread safety use a co-routine, but all the issues you have with proper performance allocation you mentioned (like multiple co-routines running at once) will still need to be addressed some how.
     
    #17 ForbiddenSoul, Jan 28, 2020
    Last edited: Jan 28, 2020
  17. TeamBergerhealer

    Supporter

    @CrypticCabub there is no way to 'schedule' your way out of a single thread limitation. Accessing blocks from multiple threads will either give you an async access exception or itll crash the server. You might be able to do something like this with chunk snapshots though, I know obtaining a snapshot can be done on another thread. Im not sure if applying also works asynchronously.
     
  18. I've never seen that happen though?
    Like you are not saying "Change the memory directly from a new thread", you are saying "Call a function that leads to memory being changed".
    Would that not just scheduled the change for a safe moment in the current tick?
     
  19. if i understand you correctly, youre saying the above logic could be called in an asynchronous thread and be fine supported by your comments. if that is what you mean, then:
    // This does not modify the block on another thread. - yes it does. or it tries. but you cannot in many cases.
    // The other thread makes a call to the main thread when it is available. - no it doesnt. thats what it means when bukkit isnt thread safe. it does not automatically transition block set calls to the main thread
    // The main thread then retrieves that block and changes it's material type. - ^
     
    • Agree Agree x 1