1.8 Progress Check

On September 2nd 2014, Minecraft 1.8 was released to the general public. Since this date we have stated that our team will be updating Spigot to this version. Today, almost six weeks later, I am pleased to announce the first glimpse of where we’re at, what this process has entailed, and where we are headed.

Due to accusations and legal threats filed against both our source code and download repositories, we were unable to begin work as soon as we would have liked to; however after many weeks of advice, legal consultation, and your support, we have resolved to complete this Minecraft update as previously scheduled. This is an extremely complex and tedious process which takes a great deal of time; however it can be broken down into a series of about a half dozen steps.

As a point of interest, Minecraft 1.8 is one of the largest single updates ever made to the game, with 11 months spanning from the release of Minecraft 1.7 and numerous internal changes such as the much hyped “Block State” system as well as over 350 additional classes added.

1. The first step in completing an update is to deobfuscate the Minecraft server jar. This is something which is aided through the use of a variety of public and private tools, however ultimately it boils down to painstaking, manual labour. Although some team members had begun dabbling with this process beforehand, official work on this started just over three weeks ago and it took about a week to finalize the first mappings draft. To give you an idea of how complex this is, the current mapping set which we are working with consists of around 4,700 unique entries, all reversed from obfuscated names such as aa.b(x, y, z, int, int) into usable namings.
   • As this is the first time we have completed such a task and other parties have all kept their tools quite close to their chest, we faced a variety of challenges in producing workable source code. In particular, we have begun a rewrite of SpecialSource into SpecialSource-2 and applied a series of handwritten bytecode patches to Fernflower, our decompiler of choice.
2. After spending a week or so deobfuscating the server jar, we needed to integrate this updated code back into CraftBukkit. Ideally the best way to do this would be to use merge tools in order to merge the changesets in, however [frustratingly], after spending nearly 8 hours straight attempting this, the result was completely unworkable! Due to the scale of this update and scope of internal changes, we instead decided that we would need to apply all the CraftBukkit patches / changes manually, essentially completing a rewrite from the ground up! One of the main advantages of this method is that we could work in stages and get the main server code integrated so that we could compile and run the server. Afterwards, we would then start testing with plugins as individual events and add API features back in. This process took us a couple of weeks.
3. In the process of completing step #2, we encountered usage of members which were not deobfuscated, in part because they were not done so in CraftBukkit, or we simply missed them, or because we were unsure of their meaning / use. As such, we went back and fixed our mappings using the tools and processes outlined in step #1. This process was relatively quick and something which we need to do repeatedly throughout the update process.
   • The first pass of this was slightly more difficult as the decompiled code was quite different to the previous versions as it was completed using ancient tools from the CraftBukkit team rather than our new tools.
   • As another point of interest, our patches to Fernflower accomplish some of the following things:
     • Rename local variables from var0 to the familiar typed system seen in other decompilers (i,j,k,object, etc.)
     • Remove spurious comments / decompile notes
     • Remove unnecessary imports
     • Reference static variables by their imported or qualified name
     • Fix incorrect method references due to omitted casts in inherited methods.
       • This one was of particular interest as it has long been the source of many unknown bugs in CraftBukkit and Spigot.
4. This is the step that we are currently on, and it can be broken down into two discrete tasks that can be worked on concurrently.
   • Addition of new API features
     • Minecraft 1.8 sees the addition of a whole host of new features. Some notable examples of things which require APIs include: the new mobs, banners, and armor stands. Additional elements such as world borders are not a priority as they can be controlled with ease from Vanilla. They are also easy for members of the community to implement and make pull requests!
   • Fixing of bugs
     • Whilst the server now successfully runs and loads plugins, there are still countless bugs to the extent where it is nearly unusable. Our job here is to test as many gameplay and API features as we can and then fix them for release. Although not everything can be caught before release, our goal is certainly to make a splash with a mighty fine piece of software from day one.
     • We have made over 70 bug reports to ourselves within the last week and many of these have since been fixed. They include everything ranging from respawning in the void and furnaces not working to world data being corrupted.
5. Update Spigot! This is something which we are more experienced at as we have done it several times before. Unfortunately for us, the Spigot codebase is always growing in size which means that each update is longer and harder than the last. Fortunately, this time, we can go back and deobfuscate any members we use, leading to faster updates in the future. We have not yet started on this step; however, we hope to do so in the near future. This update will be particularly challenging due to the scope of internal changes, particularly those around our own changes.
6. Test test test. Ok, so now we have an updated CraftBukkit and Spigot, with pretty much all the features required to release. Unfortunately these updates and features have brought with them plenty of bugs which need to be squashed. This step is much the same as #4, however needs to be much more extensive as it is the last hurdle before release.
   • No, builds will not be released at this stage. Please don’t ask. If anything, we may open some test servers for the community to help identify the lesser known bugs.
   • People are inevitably going to respond to this post and complain about software not being released now; however, the reasoning is simple. We do not release updates which we know are buggy, broken, or incomplete. This update is about setting the precedent with a release which works as well as it possibly can right out of the box. Releasing too early would be the same as releasing a half-baked product at the end of step #2. Sure, it compiles, but it certainly isn’t ready for much else.

Thanks for taking the time to read all the way down here, and I hope that the above (abridged) steps have given you a brief idea of the amount and scope of work which we are putting into this release. If anything, I can say that it is much more work than the above steps make it out to be! I also wish to note that comparing the process of a Spigot update to a plugin update is virtually impossible. We measure our time in weeks, our changed files in the hundreds, and our lines of code in tens of thousands. It is particularly frustrating to see remarks along the lines of “<plugin|server> already has full 1.8 support, why is the Spigot team being so slow, hurry up and just add this code!!!” It is part of the reason why we have made this post.

With the mention of time frames, we reach the inevitable question: “When will Spigot 1.8 be released?”. To answer this question, I have quite a clear answer:

SOON™​

But in all seriousness, we are aiming for a release date from as early as the beginning of November to as late as early December. In other words, 1.5 to 2.5 months after we first began our work. Given the sheer size of this update (largest in Minecraft history) and how long it has taken other modding teams to perform major updates in previous versions (and that is WITHOUT incorporating the legal issues we have faced or a full-time education), we feel that this is a very reasonable date.

I also wish to note that I (along with many others) have very important commitments, educational or otherwise for the duration of the month of November, which will certainly impact our ability to work on Spigot. However, we will still be in the capable hands of @Thinkofdeath who has been pulling far more than his share of the work on this update, and by the end, will certainly deserve a beer… or a slab of them.

Finally, I would like to give my thanks to everyone who has donated to the Spigot project, and in particular, @LMC and @hfmx from buycraft.net who will be joining us as sponsors. You can be assured that your money has been put to good use. Additionally, I would like to note that @andrewkm has stepped down as owner of EcoCityCraft. At their request, we have removed their official presence on the forums, but they will be continuing to sponsor us until at least Q1 of next year. Without @andrewkm, SpigotMC would have never even existed (little known fact: Spigot was created as a custom project just for him!), and I hope that you will all join me in thanking him for the years of advice and support which he has provided for this project. Andrew will retain his Sponsor badge and continue to hang around with the staff and community.

Once again, thank you for your continued support, and an extra special shoutout to @Thinkofdeath who has been doing the majority of the work for this update!

Long Live Spigot!
~Spigot Team

 

We aren't planning that for this update. More details will be revealed when it comes time to release.

 

We have built a 1.8 CraftBukkit, fixed about 75 bugs we encountered / created in the process, and will begin work on step 5 (update Spigot) some time this week.