[Server] Building a new server architecture. Need help with NBT

Discussion in 'Programming' started by zombie_striker, Jun 1, 2021.

  1. Hello all,

    So, I'm working on a new server architecture:
    Project Omega (Name still WIP) is an open-source minecraft server type that is designed to run better than spigot. It does so by stripping out all of the base game mechanics, letting you design how everything interacts with each other, so you only run what you need. It has built in protocol API, and will be designed to have all the essential features like points systems/ economies, world guard, world edit, multi-server support, and permission handling all done in-house. The code for the server is written from scratch, with no need to patch or inject code into a minecraft server jar, so it would be the first legal mod-able server architecture.

    I've gotten quite far. I've been able to support 1.13-1.15 clients (more on this later) joining the server, got the chat system working, got ability to play sounds, send titles, send bossbars, and spawn entities. Everything was all fine, except when it came to NBT tags...

    Whenever I try dealing with packets that require NBT data, the packets are sent correctly and accepted, but the NBT part disconnects the client. The only error messages are:
    Code (Text):
     Internal Exception io.netty.....DecoderException: q: Loading NBT data
    Code (Text):
    Internal Exception io.netty.....DecoderException: u: Loading NBT data.
    which aren't helpful. It basically just says that there was an error decoding the nbt data.

    The problem with 1.16 is it sends NBT data in the join game packet. That means, in order to even get 1.16 to work, I would need to get the NBT sending working.

    I know I'm sending valid NBT data, because I'm using Nedit, and the bytes are in the valid structure as defined here:

    I tried to message Mojang and see if they had any advice for me, but they just sent a generic couple of sentences basically they're only there for troubleshooting the base game.

    My question to anyone reading is this: How do I troubleshoot/fix sending NBT data? I've run out of ideas of what to do to try to figure out what the problem is.

    If anyone is interested in the project, the source code is available here:

    If anyone is interested in joining the team, message me. I'm more than happy to have more people working on the project.
  2. Debugging client errors has gotten much easier thanks to the Mojang obfuscation maps. I use MC-Remapper to dump a vanilla client code environment. Then just add debug messages wherever to figure out how the client works.
    • Useful Useful x 1
  3. I don’t see any NBT or code for joining the game so it will be hard to see if you are doing anything wrong from a cursory inspection.

    The easiest way would just be to compare what you are sending to what a vanilla server sends. You should be proficient with either ProtocolLib or Wireshark or reading the client source as the above user mentioned for this task.

    Also would be worth looking at Minestom or Glowstone or the dozens of projects out there that are also doing the same thing lol.

    IANAL but I’m fairly certain there is a higher bar to clear to be safe from litigation of this sort.
  4. Yeah, there are likely a lot more caveats legally, but I should be able to distribute the server jar without it being taken down.

    I've honestly never heard of minestom or glowstone before. I'll check them out to see if they can help coding-wise. I'm still going to work on the project because I'm stubborn and it looks like they are *Really* bare bones, where things like block inventories or other "special blocks" are not even included. As much as that may make it easier to write, and somewhat better for servers that *completely* disregard the base game, I still want some base game functionality to exist and instead be togglable/editable, to make it easier for developers that may want to change a lot of things, but may want some of the base game features to work specific ways. I also plan on incorporating as many plugin features as possible in-house (like a disguise module, particles module, a protocol module, a worldguard/worldedit module, a pets and other entity AI module, ect.), so some of the more essential features for big minigame servers are still there.

    I'm also writing it with more on an emphasis on custom blocks and custom items with resourcepacks, which is the closest I can get to modding without needing the client to manually download anything.

    I've read the source code for minecraft, have dealt with ProtocolLib, and have wireshark. As far as the source code, that would have been my next step. I tried wireshark, but the packets are encrypted, and I'm not sure how to de-encrypt it to see what exact bytes are being sent. ProtocolLib still uses a lot of injection, so they don't directly handle what I am looking for.
  5. Compression threshold should be -1 and use offline mode.
  6. Encrypted or encoded? Normal game traffic should not be encrypted, but encoding may apply to some packets I suppose.
  7. Either way, whether its just compression or encryption, I was not able to read the bytes that were being sent. Setting the compression threshold to -1 fix the issue and I'm now able to clearly read the packets being sent.

    I'm currently viewing the packets, and its hard to see the difference between what I'm sending and what a working server sends.
  8. I mean that doesn’t really much help. If you are certain that it’s the join game packet that’s causing the issue, filter that packet specifically from vanilla and compare between the TCP segment data and with what you expect from the packet schema.

    All packets post-authentication are encrypted and “compressed” in that their schema shall follow the compressed packet header (if not also being ZLIB deflated as well if the size of the payload exceeds the compression threshold). If you disable online mode, no encryption shall be done. Compression can similarly be disabled as I described in my prior comment.
    #8 xTrollxDudex, Jun 1, 2021
    Last edited: Jun 1, 2021
  9. You haven't done enough research.
    1. Spigot, Sponge, Forge, etc. are legal with what they do.
    2. There are a couple other projects with similar goals to yours as well.
      Examples: Glowstone, cuberite
    • Agree Agree x 1
  10. Alright, I've done some research:
    1. Glowstone and cuberite seem to both be stuck on 1.12.
    2. cuberite, Feather, and MCServerSharp are written in other languages beside Java, which isn't bad, but I would perfer a server that still uses Java.
    3. Minestom looks promising, but it again, its a bit too bare bones.
    Now, for the next me reading this in the future, Nedit does not include the end tag when writing the NBT tag compound. You need to manually write the last 0 byte to state that its the end of the nbt tag. After doing that, It accepts the packet and actually loads the chunk.
  11. Us being barebone is mostly an excuse to explain why we do not have parity with the official server. There is just a monstrous amount of work required, Glowstone & Cuberite are great examples of why this is not viable.

    Perhaps that you can prove it wrong, but at least it would be a first. And you could still base your work on Minestom or any other server implementation that already messed with the protocol and ensured some stability/performance

    For packet listening, I highly suggest Pakkit: https://github.com/Heath123/pakkit which saved us a lot of time
    #11 TheMode911, Jun 9, 2021
    Last edited: Jun 9, 2021