Resource TaskChain - Proper Async Operations and more!

Discussion in 'Spigot Plugin Development' started by Aikar, Oct 30, 2016.

  1. Ok, I'm totally open to tips on a better headline for the thread...

    For the past few months/years I've occasionally advertised my TaskChain system in IRC, which I hosted on a Gist (Pastebin on GitHub for those who don't know). It was a single class, and people could just drop into their plugins code base and use it.

    Well, I've now properly moved it to a GitHub project, and released it as an artifact you can include in projects.

    Getting started with Bukkit/Spigot/Paper Plugins:
    Very important read on WHY to use TaskChain:

    Current version at time of this post (please check project for latest ver!) 3.4.3

    Quick Code Teaser to demonstrate what it looks like

    Code (Java):

    public static void showInfo(Player player, Long userId, Integer page) {
        TaskChain<EmpireUser> chain = ((EmpireUser) player.getUser()).getTaskChain("VAULT");
        chain.asyncFirst(() -> {
            EmpireUser user = EmpireUser.getUser(userId);
            chain.setTaskData("name", user.getName());
            chain.setTaskData("numVaults", getPurchasedVaults(userId));
            chain.setTaskData("limit", getVaultLimit(user));
            try {
                return DB.getResults("SELECT vault_id, alias FROM vault " +
                        "WHERE user_id = ? AND vault_id > ? ORDER BY vault_id LIMIT 27",
                    userId, (page - 1) * 27);
            } catch (SQLException e) {
            return null;
             .syncLast((rows) ->,
                 new VaultsDisplay(userId, chain.getTaskData("name"), chain.getTaskData("numVaults"), chain.getTaskData("limit"), page, rows)))

    public static void onOnline(Player player, @Default("1") Integer page) {
        EmpireUser user = player.getUser();

            .syncLast((friends) -> sendList("Online Friends List", player, friends, page, false)).execute();

    public static void readMail(final Player player, final Long mailId) {
        // ... code
            .asyncFirst(getMessageTask(user, mailId))
            .abortIfNull(COLOR_MESSAGE, player, MailLang.MAIL_NOT_FOUND)

    public static void deleteMail(final Player player, final EmpireUser user, final Long mailId) {
            .asyncFirst(getMessageTask(user, mailId))
            .abortIfNull(COLOR_MESSAGE, player, MailLang.MAIL_NOT_FOUND)
            .syncLast((msg) -> {
                Confirmation.confirm(player, "&aConfirm Mail Deletion", input -> {
                    Util.sendMsg(player, "&cDeleted " + user.getName() + "'s mail: " + mailId);
                }, "&b" + user.getName(),
                    "&b" + msg.getSubject() + "&e from &b" + msg.getSender(),
                    "&bmail: " + mailId);

    What the hell is this and why would I use it?
    Good question!
    This page answers that question:

    Which btw, was linked above!

    TaskChain helps make performing thread context switching very easy. By having it available, it will encourage you to keep long processing actions off the main thread, and keep TPS high.

    So this lets me use Bukkit API on another thread?
    NO! It makes it super trivial for you to switch back to the Main Thread after finishing your asynchronous logic, and carry the result with you.

    It provides the framework for writing heavy operations async, that then act on the return sync (on main thread), and then maybe return back to async to mark the operation successful or failed.

    I use Java 7
    Get off your dinosaur and get on this rocket ship!

    On a serious note, Lambdas provided a much cleaner API and easier implementations.

    Dinosaurs have been dead for a long time, so get off it before you start to smell.

    Download Java 8

    But seriously, Plugin developers continuing to support Java 7 is why server owners don't upgrade or switch to host that have upgraded. The community needs to put pressure on hosts to move to 8.

    Java 7 is END OF LIFE. It is unsafe to use!!! Java 8 is a massive update for developers. It is too good to give up.

    ALSO - Mojang has announced the upcoming end of support for Java7. 1.11 will likely be the last version of Minecraft to be using Java 6.

    Minecraft will be forcing everyone to update soon.

    Can I get Support / Help using it?
    Sure, join #aikar on Spigot IRC ( )

    Open Source?
    Yes. Anyone familiar with me should know I pretty much use MIT for everything. I love to share :)

    Will there be punch and pie?
    Only if you do not smell like a dead dinosaur.
    #1 Aikar, Oct 30, 2016
    Last edited: Dec 20, 2016
    • Useful x 23
    • Like x 18
    • Winner x 7
    • Informative x 1
    • Optimistic x 1
    • Creative x 1
  2. This is pretty cool, I'm a huge fan of Java 8, and finding ways to reduce the boiler plate between operations.
    • Agree Agree x 1
  3. If you love reduction of boilerblate, you'd be using kotlin ;)
    • Like Like x 5
    • Agree Agree x 1
  4. Yay! I've been a user of this tool for a while now already, it's an amazing tool and definitely something I use in a lot of my projects. Glad Aikar is releasing it formally now!
    • Like Like x 1
  5. Your signature is some weird mix of Kotlin and Java!
  6. I took a quick look at this, and I fear that the functionality this provides is already implemented in ReactiveX :p
  7. Sure with a much bulkier API and not as clean integration to the game layer, or a concept of your games units for delays.
    Nor does it have a concept of "run this on the games main thread" that I can see.

    I sure wouldn't want to use that API in plugin development. TaskChain is designed around game development, where as ReactiveX is designed around super generic and alternating thread pools.

    "Getting back to main" is the entire point of the system, which I don't see as (Easily) doable in ReactiveX.
    If it is possible to do everything TC's doing, it surely won't be as clean of an API and you'll be back to a problem TC solves: Boilerplate.
    • Informative Informative x 1
  8. Great API, I'll probably use it sometime.
    i giggled
    • Agree Agree x 1
    • Funny Funny x 1
  9. :( Trump has invaded.
    • Informative Informative x 1
  10. [​IMG]
    • Funny Funny x 6
  11. note: I am pretty sure there's a concurrency bug in the Shared Chain logic. I will be replacing the logic with something simpler and more guaranteed to work tonight.
  12. If anyone has stated using TC 3.x, please update to 3.3.4 ASAP.
    I have strong suspicion that there was concurrency issues with Shared Chains from 3.0.0 to 3.3.3, as things got moved around from 2.0 and cleaned up and I was seeing weird issues on my server where it looked like some chains simply did not execute at all or the chain pipeline was frozen.

    I rewrote the whole queue system for it to be much simpler and easier to understand the logic, and removed all of the rule breaking that Shared Chains was doing about adding tasks after a chain had been executed.

    Commit here if interested:
  13. It can be integrated quite well
    But I understand that the scope of this project is to reduce boilerplate and complexity, which you will not get with rxjava (there's a lot of functionality!).
  14. That looks rather complex, but also really a different goal to re-engineer the event system. While I agree the event system could of been done better (Future style async events namely), TC is about managing the use of the current system, not re-engineering the event system like RX.
  15. You're right, all I really wanted you to see was Observable#subscribeOn(syncScheduler) which does the main thread execution :p
  16. hmm, I see. But yeah RX is pretty advanced, and way out of scope for most of this community :p

    But back to TC! I'm looking for feature suggestions (ones that don't break existing API.... Not ready for a 4.0 yet!)

    One idea I just had is a method that takes queue from the previous return, and can process it in sync or parallel (with configurable concurrency), and then only go to the next task in the chain once the queue is flushed.

    This is possible already with callback API's, but to provide it as a clean built in method.

    Think about ideas like that!
  17. Very.. very cool idea. Especially integrating lambda in it, my only question is how many times do you need to do something async and then go back to sync from the looks all your examples are perfectly thread-safe.

    Also doing
    Code (Java):
    .asyncFirst(getMessageTask(user, mailId))
    .abortIfNull(COLOR_MESSAGE, player, MailLang.MAIL_NOT_FOUND)
    Makes you use 2 threads from the threadpool instead of just one which is just rather unnecessary.

    I might just be nitpicking but this is still a very cool project.
  18. Cached Thread Pools re-use threads and only spawns one if none are free.

    But in TaskChain terms, plain TaskChains aren't about thread safety. It's about getting the heavy operations off of main and then going back to main to do the API calls, then get back off main for deletion.

    Shared Chains are a concurrency tool, but more so about logic than handling concurrency for you. If you need to dispatch 3 async tasks, and guarantee the order of execution, then shared chains help with that.

    For my mail stuff, its critical that a user can not open a mail message before its deletion finishes... Shared chains enforce that.
  19. Code (Text):
    .asyncParallel(something)  // these three tasks occur afterwards in parallel off the main thread
    .syncParallel(something)  // These three tasks occur in parallel on the main thread
    .syncSplit((idk) -> {
        for (int x = 0; x < 100; x++) {
            for (int y = 0; y < 100; y++) {
                for (int z = 0; z < 100; z++) {
                    world.getBlockAt(x, y, z).setType(Material.STONE);
                    // You can have multiple split points
    A split task will block the main thread while it is running. The split method will check if there is time to continue running the task without dropping tps, otherwise it will halt execution of the thread until the next tick.

    Parallel tasks should use runtime.availableProcessors() for the concurrency.
    #19 Empire92, Nov 3, 2016
    Last edited: Nov 3, 2016
  20. asyncParallel is initially what lead to the queue idea.
    For that, i would go with this style:

    .asyncParallel(FirstTask<T> tasks...)



    .asyncParallel((foo) -> { doStuff }, (foo) -> { do stuff }, (foo) -> { doStuff })

    One step in a chain pipeline should expect to complete before a next step would run.

    Now on sync -- this wont work. Sync tasks are expected to run on the main thread. You can only do 1 thing on a thread at a time, so can't run it in parallel.

    Now, you could block the main and suspend it while all tasks in the queue run async, but that will corrupt the API Design of TaskChain in that a sync task is expected to be API safe, which it no longer will be in that design.

    I think an .async(task1, task2, task3) style (dropping the parallel word to avoid new terminology to understand)

    As for TPS oriented method, TaskChain is no longer Minecraft bound. TC Core has no concept of TPS.

    an syncQueue that only processes 1 at a time on main, and provide a TaskChain.backOffQueue(5 /* game units*/) or backOffQueue(5, TimeUnit.SECONDS);

    This would provide the same results.

    You would need to do your for(;;) 3x iteration before hand to build a queue in the previous task ,then pass it to a queue task to process.

    This would accomplish your goal.

    the asyncQueue would be configurable concurrency, default to processors yeah.
    #20 Aikar, Nov 3, 2016
    Last edited: Nov 3, 2016