Server crash ($10 Reward)

Discussion in 'Spigot Help' started by SuisDorn, Jul 13, 2018.

Thread Status:
Not open for further replies.
  1. Hey fellows my server has crashed multiple times today with this crash log (It’s 17 MB so that’s why it’s a download):
    Code (Text):
    ---- Minecraft Crash Report ----
    // Quite honestly, I wouldn't worry myself about that.

    Time: 7/13/18 3:12 PM
    Description: Exception in server tick loop

    java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
        at org.bukkit.craftbukkit.v1_8_R3.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:364)
        at net.minecraft.server.v1_8_R3.MinecraftServer.B(MinecraftServer.java:723)
        at net.minecraft.server.v1_8_R3.DedicatedServer.B(DedicatedServer.java:374)
        at net.minecraft.server.v1_8_R3.MinecraftServer.A(MinecraftServer.java:654)
        at net.minecraft.server.v1_8_R3.MinecraftServer.run(MinecraftServer.java:557)
        at java.lang.Thread.run(Thread.java:748)


    A detailed walkthrough of the error, its code path and all known details is as follows:
    ---------------------------------------------------------------------------------------

    -- System Details --
    Details:
        Minecraft Version: 1.8.8
        Operating System: Linux (amd64) version 4.4.0-116-generic
        Java Version: 1.8.0_151, Oracle Corporation
        Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Oracle Corporation
        Memory: 4683414640 bytes (4466 MB) / 8589934592 bytes (8192 MB) up to 8589934592 bytes (8192 MB)
        JVM Flags: 13 total; -Xms8G -Xmx8G -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:MaxGCPauseMillis=100 -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:G1NewSizePercent=35 -XX:G1MaxNewSizePercent=60 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=50 -XX:+AggressiveOpts -XX:+AlwaysPreTouch
        IntCache: cache: 0, tcache: 0, allocated: 0, tallocated: 0
        CraftBukkit Information:
       Running: CraftBukkit version git-Spigot-21fe707-e1ebe52 (MC: 1.8.8) (Implementing API version 1.8.8-R0.1-SNAPSHOT) false
    https://ufile.io/vceip
    I’m not an expert on this field so I’m asking here for some more info, I think some plugin is creating loads of threads which eventually causes the server to crash. This would also explain why the crash log is this big, because it’s parsing all the threads. In the crash-log itself I can’t really determine which plugin is causing this but my guess would be featherboard so I’m tagging @Maximvdw . I checked my memory usage and it’s totally fine 10 gb is “available” for system recourses and system thread limit is set to something like 223500 which should be enough;) and the user thread limit equals the system thread limit (I changed this after the crashes to check if it would help which it did not).

    Edit:
    This crash started to occur after I downloaded an updated version of this plugin:
    https://www.mc-market.org/resources/6611/updates
    I also did a lot more things so I’m not entirely sure if this plugin is really what started the crashes. The crashes mostly occur after like 3 / 4 hours.
    I just removed the spoofmachine plugin and I’ll keep you guys updated if the crashes will still occur.
     
    #1 SuisDorn, Jul 13, 2018
    Last edited: Jul 13, 2018
  2. Update, removing the spoofmachine plugin did not work.

    Running htop command on ubuntu 16.04 with server running for 2 hours:
    [​IMG]

    Running top -H command on ubuntu 16.04 with server running for 2 hours:
    [​IMG]
     
  3. All i can see is that the server runs out of memory. Don't you have problems with too many entities? If not i guess it's a serious memory leak caused by 1 of the plugins. Don't u maybe have your own plugin? Idk how it's possible to run out of 8gb.
     
  4. Show your timings via /timings
     
  5. Also change your start script so that your -Xms tag is at like 1 gb instead of matching the -Xmx tag so that it can release the ram as it doesn't need it so that other programs on the server can use it
     
  6. Maximvdw

    Benefactor

    Support is only given to buyers
    [​IMG]
     
  7. kk that’s understandable just note I bought this server (with domain playerbase etc) from mc-market so that’s why they’re not registered on my spigot account.
     
  8. In the crash log itself it parsed:
    Code (Text):
    Memory: 4683414640 bytes (4466 MB) / 8589934592 bytes (8192 MB) up to 8589934592 bytes (8192 MB)
    So there should be enough memory?
    Could be a memory leak yeah but would that cause the server to crash with this specific error:
    Code (Text):
    unable to create new native thread
    I think @electronicboy can help us out here.
    Okay will start it now.
    I read here:
    https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/
    This should be the best flags to use so I think I’ll stick with that.
     
  9. Tux

    Tux

    I think the problem is with VotingPlugin. I'd remove it and switch to another plugin (I'm going to shamelessly plug SuperbVote, which I made).
     
  10. Might try your plugin when I fixed this problem but why do you think votingplugin is the problem?
     
  11. Looks like world save is pretty hefty. What’s the combined size of your worlds?
     
  12. From my experience world saves are most of the time pretty hefty. The world size is 16k x 16k.
     
  13. Timings don’t really show much. I’d say remove plugins one by one and debug it out. Start with jobs
     
  14. 1. If you think the problem is Featherboard use Liteboard, Featherboard is really bad my biggest regret and has lot of performance issues.

    2. Use Paper 1.12.2 if you want to have a lot more performance, if you see your actual performance isnt enough updating will fix lot of your issues.
    I went from 80% - 90% usage to 20% - 30% with 30 online in Survival (Tested in HTOP, 8 cores)

    3. Dont use flags! In my experience they gave me less performance than more, and your issue is likely about memory, so stop using flags. Java manages everything better than humans do.

    4. Deactivate metrics and update from all your plugins, and check to disable functions you dont use from them.

    5. Make world saving every 1-2 hours, if your world is too big maybe is making memory leaks. (I dont really know though, but making the world save every 1 - 2 hours will obviously help)

    6. AutoRestart your server every night! This is really important to keep everything fresh because almost every plugin (and spigot) has memory leaks that leads to server starvation, restarting will clear the memory and make the server start "fresh" i personally use autorestart reloaded for this and restart everyday in the morning.

    7. Keep all your plugins updated.- And dont use old public plugins! (Atleast you know it wont harm your server).
    PD: if you use essentials i recommend updating it to essentialsx if you didnt yet.

    In my personal experience, the best thing i ever did was updating from 1.8 to 1.12, if you see a performance increase in Paper 1.12.2 and want to add compatibility with 1.7/1.8 and pre-1.9 combat i have a special plugin pack configured and programmed to make your pvp the most similar as posible to pre-1.9 (KB is based on 1.7 tho) and add compatibility with older versions. You can download it in my pluginpack discord http://discord.gg/cjt9bPA we give general support too.

    And i want to add; Its not any plugin; I use the same with no problems; Plugins show that they are using lot of resources because the server is lagging and that makes all the processes to make something similar to "collapse".

    I had problem with this too; this is my personal experience.

    Hope this helps. And i dont want any money, but if you wish to donate; http://paypal.me/mchurruarin ;)
     
    #15 LinsaFTW, Jul 14, 2018
    Last edited: Jul 14, 2018
    • Informative Informative x 1
  15. Tux

    Tux

    I looked at the crash log itself and discovered a bunch of threads tied up calling into VotingPlugin. That's why the server is complaining that it can't launch any more threads. I'm fairly confident the issue lies there.
     
  16. I think thats normal if many people votes, as the server is lagging, the votes maybe take time to process.
     
  17. Tux

    Tux

    No, this is not normal. Did you even examine the log yourself?

    I'll include one of the threads, tell me if this is "normal":

    Code (Text):
    BLOCKED Craft Scheduler Thread - 2197: [com.Ben12345rocks.VotingPlugin.Util.PlaceHolders.PlaceHolders.getPlaceHolder(PlaceHolders.java:54), com.Ben12345rocks.VotingPlugin.PlaceHolderAPI.expansion.VotingPliuginExpansion.onPlaceholderRequest(VotingPliuginExpansion.java:75), me.clip.placeholderapi.PlaceholderAPI.setPlaceholders(PlaceholderAPI.java:212), me.clip.placeholderapi.PlaceholderAPI.setPlaceholders(PlaceholderAPI.java:188), be.maximvdw.placeholdercore.placeholders.PlaceholderAPIPlaceholderPack$1.getResult(PlaceholderAPIPlaceholderPack.java:37), be.maximvdw.placeholdercore.placeholders.PlaceholderAPIPlaceholderPack$1.getResult(PlaceholderAPIPlaceholderPack.java:26), be.maximvdw.featherboard.cJ.a(cJ.java:224), be.maximvdw.featherboard.l.a(l.java:38), be.maximvdw.featherboard.bH.a(bH.java:19), be.maximvdw.featherboard.bH.a(bH.java:56), be.maximvdw.featherboard.api.scoreboard.ScoreboardLine.updatePlaceholders(ScoreboardLine.java:78), be.maximvdw.featherboard.api.scoreboard.ScoreboardGroup$AnimationUpdater.run(ScoreboardGroup.java:7), org.bukkit.craftbukkit.v1_8_R3.scheduler.CraftTask.run(CraftTask.java:71), org.bukkit.craftbukkit.v1_8_R3.scheduler.CraftAsyncTask.run(CraftAsyncTask.java:53), java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149), java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624), java.lang.Thread.run(Thread.java:748)]
    Examining the code itself, I get the impression that the author of this plugin does not know much about safe concurrent programming in Java, or even much about recommended practices for Java projects in general.
     
    • Informative Informative x 1
  18. Oh yeah, thats a exception.
     
  19. Tux

    Tux

    Not bad advice.

    The flags that any random shmuck off the net throws out will ruin your performance. Use Aikar's flags instead. He has extensive experience with the inner workings of the Minecraft server and did extensive research into Java's garbage collector, so his advice is probably the best you'll get when it comes to Java flags. He also works on Paper.

    The OP is already using Aikar's flags, and in any event this isn't a memory leak issue.

    ...this makes almost no sense at all.

    Autorestarts should not be needed in any sane server setup.

    Good advice.
     
    • Informative Informative x 1
Thread Status:
Not open for further replies.

Share This Page