Server performance guide

The following is Ras' guide to performance for Pixelmon servers. It is mostly written in first person to acknowldege the fact this is trial and tested information from a Pixelmon teammember's experience. This guide covers world pre-generation, launch Java arguments, recommended settings, commonly encountered issues, and lastly, discusses tools by which you can keep track of your server's performance, including reading and sharing logs. If you are installing Pixelmon on your own server, see the Server Installation guide instead.

This guide assumes that you understand the basic principles of Minecraft server hosting, and specifically, current Pixelmon server environments, including Sponge and Forge. If you do not understand these environments, or have never used them (or head of them), we strongly recommend you read our Server Installation guide first, and if you need further help, please use #server-support in the Pixelmon Discord.

Notes

Pixelmon servers are currently most commonly using Sponge as a plugin environment. Plugins mentioned in this guide can be found on Sponge Ore, a plugin repository. All other plugins mentioned will be have links provided in their description.

World Pre-generation

Please note that this cannot be avoided. It is of extreme importance for servers to pre-generate their world's chunks as soon as possible.

1. Find, download and install Nucleus, Sponge's leading Essentials-like plugin.
2. In your server.properties (located in the server's root folder), set the setting max-tick-time to -1. This will prevent your server from timing out during this process. This setting is used to tweak the Server Watchdog, a Minecraft ‘feature’ that forcefully crashes your server if a single tick takes too long. Disabling it removes the risk of the Watchdog crashing your server while pre-generating.
3. If not already the case, whitelist your server and ensure all players have logged off. The process takes a few hours (depending on your server's hardware), and runs smoother if there are no players online.
4. Use /world border set [world] [center-x] [center-z] [diameter-in-blocks], replacing each variable with the desired configuration. The bigger the [diameter-in-blocks], the bigger the border, the bigger the map, the longer the pre-generating process will take. Once you have decided on your border's size, add 2000 blocks more to its count. If you want 15000, then 17000 will be your border size for pregenerating.
   1. The desired choice will vastly depend on your particular case. For starting, public servers, we recommend anything between 15000 and 20000.
   2. In general, anything above 30K in diameter will be a difficult to manage, as worlds of that size can go upwards of 50GB after 10 months of use.
   3. This pre-generation process has to be repeated for every world that will be used by players. Vanilla Pixelmon servers have 4 by default - overworld, DIM-1, DIM1, and ultraspace. Note, each world can have it's own specific size, but all will weigh heavily on disk space.
   4. When deciding your world's size, please take into account the need for fast, reliable and varied backups. Most servers will need at least one current, zipped backup of the world folder, as a safety net.
5. Use /world border gen [world] to start pre-generating. You will see messages in console indicating the progress. Don't worry, it'll stop once it reaches your border, so you can basically leave your server until it is done.
   1. To speed-up the process, you can use the -a flag in the pre-generation command to turn on 'agressive' mode. This will usually allow pre-generation to move along faster, but it can also severely immobilize the server. We do not recommend usig this flag if you have players online. To use, insert the flag like so, /world border gen -a [world].
   2. You can also increase save time to accelerate the generation process, it will however imply you will lose more time if the server ends up crashing. For example, /world border gen --save 5m [world]. You can combine this with the <code-a flag to generate at the fastest rate possible.
6. Expect the pre-generating process to take a long time. Make yourself a coffee, or two, or three. Wait.
7. Once the completion message is show in console, you will need to use /world border set [world] [center-x] [center-z] [diameter-in-blocks] again. This time however, the [diameter-in-blocks] will be your initial size, without the additional buffer recommended above. This will prevent players at the border from generating past it, and act as a buffer to ensure no more generation takes place.
8. Use Minecraft's native world border damage command, /worldborder damage buffer <sizeInBlocks>, to turn on world border damage higher than default.
9. Repeat steps 2 through 8 for every existing world.
10. Once you are done, go back to your server.properties and set the max-tick-time to 18000. This re-enabled Minecraft's native Watchdog, ensuring that if your server stalls because of a leak, it will crash and not remain frozen without recovery.

Java Launch Arguments

1. In our Server Installation guide, we covered start.bat scripts. In that script, replace with the following:

java -Xms6G -Xmx6G -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:MaxGCPauseMillis=100 -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:G1NewSizePercent=50 -XX:G1MaxNewSizePercent=80 -XX:G1MixedGCLiveThresholdPercent=50 -XX:+AlwaysPreTouch -jar server.jar

1. Replace server.jar with your running jar. For example, if running a Forge server, your running jar is named similarly to forge-1.12.2-14.23.5.2838-universal.jar.
2. Most Pixelmon servers will need at the bare minimum 4GB of RAM to run properly. Processes like pre-generation can exacerbate memory problems, which is why it should be done before your server's launch. The more players you have online, the more 'heavy' your server will be, so plan accordingly. For the average server with a small player-base, 6GB of RAM will suffice.
   1. More RAM does not mean more performance. In fact, on the contrary - it will worsen Java's garbage collection. Do not assign over 14GB, unless otherwise indicated by your particular circumstances and hosting experience.
   2. For a longer, and more technical read on Java launch arguments, we recommend studying Aikar's Garbage Collection guide, which applies to Forge, Sponge and Spigot environments alike.

Reduced View-Distance

1. Open your server.properties file, located at the base (root) folder of your server).
2. By default, Minecraft's view-distance is of 10. This means that around the player, a radius of 10 chunks will be loaded as they move/idle. That's a lot of chunks.
3. The more chunks are loaded, the worse it is for your server. Decrease the value of the view-distance to 6, or 4.

Tile-Entity and Entity Activation Ranges

Sponge's configuration file, the global.conf (located under /config/sponge) allows you to change activation ranges of both entities and tileentities. This helps the server process less tasks per tick, and should greatly improve your performance. Note: an ‘activation range’ is the radius around the tile-entity (for example, a furnace) or an entity (for example, a creeper) for which it will tick when in proximity to the player. If you set your radius for example to 1, it will mean that your furnaces will only update when at 1 block around the player; of course, that also means your players will be very annoyed when their farms don’t update because the radius is too low. Similarly, if you have a creeper with an activation range of 0, chances are it will never update, and just explode when the player is standing right on top of it.

1. Open the global.conf.
2. Under "entity-activation", turn on the `auto-populate` feature. Change the default range to 32.
3. Find "tileentity-activation", and do the same - turn on the `auto-populate` argument, and set the default range to 32.
4. Save and upload the file to your server, making sure to restart.
5. Periodically check that file over the next couple of hours or days (depending how active your players are). The config will start populating, and more options will appear, under the name of entities (minecraft:pig) or tile-entities (pixelmon:fossil_cleaner).

Now, you can lower or raise the ranges of the blocks defined in the global.conf, under "entity-activation" and "tile-entity-activation". You usually should lower most to 16/8/6. The lower the value, the less strain those items have on your server, but that doesn't mean you should lower it to almost nothing. That would cause your entities/tile-entities to malfunction. We don't want that. Sponge’s config files are kept alive during runtime, meaning that the server prevents you from editing them. On shutdown, these files revert. If you are editing values and your server is running, please do /sponge -g reload to reload the configuration and save your changes. Alternatively, you can edit these files while offline.

Choosing your Plugins or Mods

This comes down to the quality of what your server is running. Not all plugins, or mods, have the same impact on your performance. Some, like Pixelmon, are unavoidable costs on our networks, but it might not be the case with plugins, or mods. In comparing Polis to GriefPrevention, the former is heavy on the server because it logs everything into a single file, whereas the latter divides the checks into many files that are managed by Luckperms. Choosing the right plugins (and mods) for your server will be one of the ways that you can greatly hinder, or improve, your performance. If you have questions about something, ask around, see if anyone has commentary on its performance on a server.

The golden rule is this: nothing is free. If you have a plugin with an inventory GUI, or block-logging features, or even regeneration, that stuff CAN be expensive. You have to be prepared for features added to impact your server’s performance. Remember, Minecraft is a single-core game, and no amount of additional RAM will make it go any faster on the main thread. Choosing your plugins and mods wisely will help you decide what you keep, and what is too costly to run.

Known Performance Plugin/Mod Configs

There's always new features for you to test and experiment with. The best way you can keep a good server performance is to watchout for new methods of lowering your server's strain. Of course, please ask around if you are going to edit something you have no idea about. Here are a few I've found.

1. Lighting async values can be found in Sponge's global.conf. You can assign more threads to this, and it should help with the lighting updates within loaded chunks. If you don’t know what this implies, or what it can potentially do, please do not use this option.
2. Real-time TPS is an experimental feature in Sponge's global.conf as well. In that file, under "modules", set real-time to true. It'll try to replicate as much as possible 20 TPS (ticks per second) for the client, even when the server is experiencing lag.
3. Sometimes, GriefPrevention (or GriefDefender) causes issues with the buildup of dropped items. Use /cf collide-block minecraft:item minecraft:any true override in your wilderness to lower their impact on the server.
4. Use CatClearLag, or a command scheduler, to queue /pokekill and dropped-item despawning. Both are notoriously fickle, so I recommend giving them the extra help needed to make sure those are recycled.
   1. CatClearLag, along with other lag-fixing plugins are known to work off of basic config fixes that you could be doing yourself, like the spawning rate in Pixelmon, or base MC mobs. I personally would not recommend using it as an ‘improving’ plugin, as it does not actually clear your lag. That said, if you need the entity clearing scheduling because, for instance, entity despawning is an issue, use it. It’s a good way to schedule clearing tasks, but don’t expect it to be magic. It won’t replace anything in this guide.
5. World saves are a real issue on any server. By default on Sponge, they happen every 45 seconds. In your global.conf, change the tick count between player-data and world save to the same amount - I usually do every 5 minutes, so 6000 ticks.
   1. You could be scheduling a longer interval, it’s a matter of personal decision. Just remember to think of this interval as “the maximum amount of timed progress my players are willing to lose in the instance of a crash”. If the server crashes, the data is reverted to the last world save - so you’d want these as frequently as can be managed.
6. In Sponge’s global.conf, enable structure saving and let it auto-populate to true. It’ll help saving mineshafts.
7. Sponge also has a use-real-time tick option, as well as use-panda-redstone and eigen-redstone fix option. All three are recommended.
   1. If you experience weird behaviour with redstone, please try first disabling the two fixes mentioned above to see if the issue persists. If it fixes the glitches you see, please report your findings to Sponge's tracker.

Pixelmon Performance Settings

Pixelmon Reforged itself has made a few configuration settings to lessen the performance impact of the mod's processes in high-density, high-population servers. Below is an explanation of a few changes made for this purpose. Please note, Pixelmon's configuration file can be found under ./config/pixelmon.hocon.

1. Pixelmon’s hocon config has a great config option to lower issues related to saves: async pokemon saving. Search for useAsyncSaving and set it to true. The interval's default is 90, and it should remain at default value unless you are interested in experimenting. From experience, this default value is fine.
   1. Async saving is fully tested by now, and it does not impact the safety of player pokemon saving.
   2. Once you have grown accustomed to Sponge's Aikar Timings, carefully scour through the data provided. If you see "ComputerManager" or "PokeballManager" as the highest two entries in your timings, this means you did not follow the guide fully. Remember that reading and applying the wiki are two very different things.
2. Pixelmon Reforged's Better Spawner is a treasure for performance. Play around with the different options, and see what works best for you. All servers are different, and all players want different things. It's up to you to find a good balance.

Keeping Track of Performance

Performance is never assured, and it will vary constantly, so you have to be prepared to recognize and ascertain issues as they arise. You can do this by monitoring your server, and keeping an eye on errors, or values out of the ordinary. Below is a list of the resources I personally recommend, and it includes how exactly I use them in my daily monitoring.

1. Sponge Timings: In your Sponge global.conf, under the "timings" section, set the history-length to 18000, and the interval to 2000.
   1. This will allow more data to collect in your Sponge Timings, which you can view by using /sponge timings paste. Though this might look complicated at first, Sponge Timings are a useful tool to find out performance issues. Make sure that the right-corner header is set to "AVG" and "ALL". Intuitively, the graph should be mostly green, with no major fluctuations. Pictured on the right-hand side is an example of this.
      

      Pictured: Sponge's Aikar Timings catching classic world-save pokemon performance issues.
   2. The data under it, sorted into a table, should not contain values out of the ordinary. If you see "doChunkMap", "chunkGenerator" or "chunkPopulate", you didn't fully pre-generate your map.
2. Nucleus’ GC Command: Nucleus’ /gc -c command is a great way to quickly assess performance in the last session. Always note the average "loaded chunk" count, which will vary depending on the number of players.
3. Forge & Sponge’s TPS Command: Both Forge and Sponge have a command to monitor TPS per world. Use /forge tps or /sponge tps to quickly check the server's current performance. Minecraft counts 20 ticks per second (TPS), so if the resulting output shows it to be below 20 TPS, run a timings and check your console for clues as to what is happening.
4. Server Console: the most primitive tool left, the server console, is only ever useful when you see a recurring error appear as your TPS decreases. Most, but not all, spamming errors will affect your performance, so make sure that you have checked for errors in the console before further investigating.
   1. If you see "moved too quickly!" messages, then your server is suffering from chunk loading. Make sure your map is pre-generated to lessen the impact of chunk loading.
5. TickProfiler: pretty simple tool to use, it is installed as simple jar. You can find the wiki and it’s open source on MinimallyCorrect’s github. It will be a useful tool that can help when timings do not seem to give you a good idea of what exactly is causing all that TPS loss. It is used to track entity and tile-entity usage. Thanks to merjilin for this tip.
6. WarmRoast: Recommended only as a profiling tool NOT to leave online at all times, you can download and install it from its author’s Github, sk89q. This is mostly useful when you have a memory leak and timings cannot seem to decide what’s wrong.
7. Spark: this plugin for Sponge was developed by Luck, the same person that made Luckperms. A more user-friendly option to WarmRoast, this allows process profiling in-game (which WarmRoast does not). The added accessibility makes this my preferred option, as it is easy to setup and has a discord if ever you need more help. Download and read the wiki here. A simple command to use is /spark sampler --thread * on. This sampler will have to be stopped once you’ve neared 3-5 minutes of runtime during lag spikes.
8. Sampler: arguably the most useful out of the bunch in my personal experience, Sampler allows server owners to set a TPS timer and catch an all-threads, spark-like memory snapshot that is registered to a file. It also automatically will generate stall-reports, useful for never-ending spikes, and effectively replacing that ‘watchdog’ crash report that is pure garbage (sorry to burst anyone’s bubble). It’s a free, public plugin. Simply drop it to your /mods folder to install.
   1. To get a sampling report, run /sampler trigger ticktime 30000. This command will therefore take a snapshot nps when the ticktime reaches 30000. If you want to be more aggressive, you can always set this lower. You will have to do this every restart (and note, as all sampling plugins, running it does impact your performance slightly, so I wouldn’t run it just for fun). That command will generate a .nps file in your server’s root directory.
   2. To open it, you will have to download VisualVM (free software). Once you do, you can unzip to your desktop the downloaded file, and run the .exe.
   3. Once you have VisualVM open, you can open your .nps file, and start poking at the data collected.
   4. My personal trick is to go into ‘Select Threads’, uncheck everything in the dropdown list and keep ‘Server_Thread’ checked. The data will then be re-sorted automatically.
   5. Just like a Spark, you can then collapse and expand tabs to look at the possible lag culprits, like this.
   6. As mentioned previously (and if you didn’t see this, read back up), stall-reports are created automatically. They are located in the ./stall-reports folder in your server’s directory, and they are simple text files, like crash-logs or debug logs.
   7. Download and start opening them (personally recommend Notepad++), and check for the first sections of the stall report to see the possible sources.
   8. Though this is slightly less accurate than sampling reports, it can be useful in determining a slowdown in console, and usually will be created in batches. For better accuracy, consider using the first stall-report out of a single ‘batch’, since all the subsequent stall-reports are usually slowdowns caused by the initial performance issue.

Work in progress