TakeV/Paper
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
Paper/Spigot-Server-Patches/0529-Implement-Chunk-Priority-Urgency-System-for-Chunks.patch

843 lines
44 KiB
Diff
Raw Normal View History

Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Aikar <aikar@aikar.co>
Date: Sat, 11 Apr 2020 03:56:07 -0400
Subject: [PATCH] Implement Chunk Priority / Urgency System for Chunks
Mark chunks that are blocking main thread for world generation as urgent
Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.
Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.
This massively reduces the lag spikes from sync chunk gens.
Then we further prioritize loading order so nearby chunks have higher
priority than distant chunks, reducing the pressure a high no tick
view distance holds on you.
Chunks in front of the player have higher priority, to help with
fast traveling players keep up with their movement.
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
diff --git a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
index b5c2e1f4a2b5fdcaa6bb01f4b3b6847cd5b73ae8..6209b33d8497ec56bbde507e523db0649c66f590 100644
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
--- a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java
+++ b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java
@@ -4,7 +4,10 @@ import com.destroystokyo.paper.io.PaperFileIOThread;
import com.destroystokyo.paper.io.IOUtil;
import com.destroystokyo.paper.io.PrioritizedTaskQueue;
import com.destroystokyo.paper.io.QueueExecutorThread;
+import it.unimi.dsi.fastutil.longs.Long2ObjectMap;
+import net.minecraft.server.ChunkCoordIntPair;
import net.minecraft.server.ChunkRegionLoader;
+import net.minecraft.server.ChunkStatus;
import net.minecraft.server.IAsyncTaskHandler;
import net.minecraft.server.IChunkAccess;
import net.minecraft.server.MinecraftServer;
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -106,7 +109,7 @@ public final class ChunkTaskManager {
}
static void dumpChunkInfo(Set<PlayerChunk> seenChunks, PlayerChunk chunkHolder, int x, int z) {
- dumpChunkInfo(seenChunks, chunkHolder, x, z, 0, 1);
+ dumpChunkInfo(seenChunks, chunkHolder, x, z, 0, 4);
}
static void dumpChunkInfo(Set<PlayerChunk> seenChunks, PlayerChunk chunkHolder, int x, int z, int indent, int maxDepth) {
@@ -127,6 +130,30 @@ public final class ChunkTaskManager {
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Status - " + ((chunk == null) ? "null chunk" : chunk.getChunkStatus().toString()));
PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Ticket Status - " + PlayerChunk.getChunkStatus(chunkHolder.getTicketLevel()));
PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Holder Status - " + ((holderStatus == null) ? "null" : holderStatus.toString()));
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Holder Priority - " + chunkHolder.getCurrentPriority());
+
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ if (!chunkHolder.neighbors.isEmpty()) {
+ if (indent >= maxDepth) {
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Neighbors: (Can't show, too deeply nested)");
+ return;
+ }
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Neighbors: ");
+ for (PlayerChunk neighbor : chunkHolder.neighbors.keySet()) {
+ ChunkStatus status = neighbor.getChunkHolderStatus();
+ if (status != null && status.isAtLeastStatus(PlayerChunk.getChunkStatus(neighbor.getTicketLevel()))) {
+ continue;
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ }
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ int nx = neighbor.location.x;
+ int nz = neighbor.location.z;
+ if (seenChunks.contains(neighbor)) {
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + " " + nx + "," + nz + " in " + chunkHolder.getWorld().getWorld().getName() + " (CIRCULAR)");
+ continue;
+ }
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + " " + nx + "," + nz + " in " + chunkHolder.getWorld().getWorld().getName() + ":");
+ dumpChunkInfo(seenChunks, neighbor, nx, nz, indent + 1, maxDepth);
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ }
+ }
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
}
}
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
diff --git a/src/main/java/net/minecraft/server/ChunkMapDistance.java b/src/main/java/net/minecraft/server/ChunkMapDistance.java
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
index 586a20fe5c77c2ad5fa26f337a94a16e21d8b5e2..9805361e2d49fa1cfecf0c5811187fc503d0ad8e 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/ChunkMapDistance.java
+++ b/src/main/java/net/minecraft/server/ChunkMapDistance.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -23,6 +23,7 @@ import java.util.concurrent.Executor;
import javax.annotation.Nullable;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import org.spigotmc.AsyncCatcher; // Paper
public abstract class ChunkMapDistance {
@@ -84,6 +85,7 @@ public abstract class ChunkMapDistance {
}
private static int a(ArraySetSorted<Ticket<?>> arraysetsorted) {
+ AsyncCatcher.catchOp("ChunkMapDistance::getHighestTicketLevel"); // Paper
return !arraysetsorted.isEmpty() ? ((Ticket) arraysetsorted.b()).b() : PlayerChunkMap.GOLDEN_TICKET + 1;
}
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -97,6 +99,7 @@ public abstract class ChunkMapDistance {
public boolean a(PlayerChunkMap playerchunkmap) {
//this.f.a(); // Paper - no longer used
+ AsyncCatcher.catchOp("DistanceManagerTick");
this.g.a();
int i = Integer.MAX_VALUE - this.e.a(Integer.MAX_VALUE);
boolean flag = i != 0;
@@ -107,11 +110,13 @@ public abstract class ChunkMapDistance {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
// Paper start
if (!this.pendingChunkUpdates.isEmpty()) {
+ this.pollingPendingChunkUpdates = true;
while(!this.pendingChunkUpdates.isEmpty()) {
PlayerChunk remove = this.pendingChunkUpdates.remove();
remove.isUpdateQueued = false;
remove.a(playerchunkmap);
}
+ this.pollingPendingChunkUpdates = false;
// Paper end
return true;
} else {
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
@@ -147,21 +152,31 @@ public abstract class ChunkMapDistance {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
return flag;
}
}
+ boolean pollingPendingChunkUpdates = false; // Paper
private boolean addTicket(long i, Ticket<?> ticket) { // CraftBukkit - void -> boolean
+ AsyncCatcher.catchOp("ChunkMapDistance::addTicket"); // Paper
ArraySetSorted<Ticket<?>> arraysetsorted = this.e(i);
int j = a(arraysetsorted);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
Ticket<?> ticket1 = (Ticket) arraysetsorted.a(ticket); // CraftBukkit - decompile error
ticket1.a(this.currentTick);
- if (ticket.b() < j) {
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
+ if (ticket.getTicketLevel() < j) {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.e.b(i, ticket.b(), true);
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
+ // Paper start - queue update if priority ticket add
+ } else if (ticket.getTicketType() == TicketType.URGENT || ticket.getTicketType() == TicketType.PRIORITY) {
+ PlayerChunk updatingChunk = chunkMap.getUpdatingChunk(i);
+ if (updatingChunk != null) {
+ pendingChunkUpdates.add(updatingChunk);
+ }
+ // Paper end
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
return ticket == ticket1; // CraftBukkit
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
}
private boolean removeTicket(long i, Ticket<?> ticket) { // CraftBukkit - void -> boolean
+ AsyncCatcher.catchOp("ChunkMapDistance::removeTicket"); // Paper
ArraySetSorted<Ticket<?>> arraysetsorted = this.e(i);
boolean removed = false; // CraftBukkit
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
@@ -182,6 +197,82 @@ public abstract class ChunkMapDistance {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.addTicketAtLevel(tickettype, chunkcoordintpair, i, t0);
}
+ // Paper start
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ public static final int PRIORITY_TICKET_LEVEL = 33;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ public boolean markUrgent(ChunkCoordIntPair coords) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return addPriorityTicket(coords, TicketType.URGENT, 30);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+ public boolean markHighPriority(ChunkCoordIntPair coords, int priority) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ priority = Math.min(28, Math.max(1, priority));
+ return addPriorityTicket(coords, TicketType.PRIORITY, priority);
+ }
+
+ private boolean addPriorityTicket(ChunkCoordIntPair coords, TicketType<ChunkCoordIntPair> ticketType, int priority) {
+ AsyncCatcher.catchOp("ChunkMapDistance::addPriorityTicket");
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ long pair = coords.pair();
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ boolean success;
+ if (!(success = updatePriorityTicket(coords, ticketType, priority))) {
+ Ticket<ChunkCoordIntPair> ticket = new Ticket<ChunkCoordIntPair>(ticketType, PRIORITY_TICKET_LEVEL, coords);
+ ticket.priority = priority;
+ success = this.addTicket(pair, ticket);
+ }
+
+ chunkMap.world.getChunkProvider().tickDistanceManager();
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ PlayerChunk updatingChunk = chunkMap.getUpdatingChunk(pair);
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ if (updatingChunk != null && updatingChunk.priorityBoost < priority) {
+ // May not be enqueued, enqueue it if not and tick distance manager
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ chunkMap.queueHolderUpdate(updatingChunk);
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ chunkMap.world.getChunkProvider().tickDistanceManager();
+ }
+ return success;
+ }
+
+ private boolean updatePriorityTicket(ChunkCoordIntPair coords, TicketType<ChunkCoordIntPair> type, int priority) {
+ ArraySetSorted<Ticket<?>> tickets = this.tickets.get(coords.pair());
+ if (tickets == null) {
+ return false;
+ }
+ for (Ticket<?> ticket : tickets) {
+ if (ticket.getTicketType() == type) {
+ // We only support increasing, not decreasing, too complicated
+ ticket.priority = Math.max(ticket.priority, priority);
+ return true;
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ }
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+
+ return false;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ public int getChunkPriority(ChunkCoordIntPair coords) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ AsyncCatcher.catchOp("ChunkMapDistance::getChunkPriority");
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ ArraySetSorted<Ticket<?>> tickets = this.tickets.get(coords.pair());
+ if (tickets == null) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return 0;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+ for (Ticket<?> ticket : tickets) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ if (ticket.getTicketType() == TicketType.URGENT) {
+ return 30;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+ }
+ for (Ticket<?> ticket : tickets) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ if (ticket.getTicketType() == TicketType.PRIORITY && ticket.priority > 0) {
+ return ticket.priority;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return 0;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ public void clearPriorityTickets(ChunkCoordIntPair coords) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ AsyncCatcher.catchOp("ChunkMapDistance::clearPriority");
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ this.removeTicket(coords.pair(), new Ticket<ChunkCoordIntPair>(TicketType.PRIORITY, PRIORITY_TICKET_LEVEL, coords));
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ }
+
+ public void clearUrgent(ChunkCoordIntPair coords) {
+ AsyncCatcher.catchOp("ChunkMapDistance::clearUrgent");
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ this.removeTicket(coords.pair(), new Ticket<ChunkCoordIntPair>(TicketType.URGENT, PRIORITY_TICKET_LEVEL, coords));
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ }
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ // Paper end
public <T> boolean addTicketAtLevel(TicketType<T> ticketType, ChunkCoordIntPair chunkcoordintpair, int level, T identifier) {
return this.addTicket(chunkcoordintpair.pair(), new Ticket<>(ticketType, level, identifier));
// CraftBukkit end
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
@@ -397,12 +488,14 @@ public abstract class ChunkMapDistance {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
});
}, i, () -> {
- return j;
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return Math.min(PlayerChunkMap.GOLDEN_TICKET, j + 15); // Paper - this is based on distance to player for priority,
+ // ensure new no tick tickets arent higher priority than high priority tickets...
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}));
} else {
ChunkMapDistance.this.k.a(ChunkTaskQueueSorter.a(() -> { // CraftBukkit - decompile error
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
ChunkMapDistance.this.m.execute(() -> {
ChunkMapDistance.this.removeTicket(i, ticket);
+ ChunkMapDistance.this.clearPriorityTickets(new ChunkCoordIntPair(i)); // Paper
});
}, i, true));
}
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
diff --git a/src/main/java/net/minecraft/server/ChunkProviderServer.java b/src/main/java/net/minecraft/server/ChunkProviderServer.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
index 7a275bf3260f9fbefc41883c5ebdc1eb2196daf0..54e89c9cc6c47ff2c4f4dd5d4c22a391f8a3d6e0 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/ChunkProviderServer.java
+++ b/src/main/java/net/minecraft/server/ChunkProviderServer.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -432,6 +432,18 @@ public class ChunkProviderServer extends IChunkProvider {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
public <T> void removeTicketAtLevel(TicketType<T> ticketType, ChunkCoordIntPair chunkPos, int ticketLevel, T identifier) {
this.chunkMapDistance.removeTicketAtLevel(ticketType, chunkPos, ticketLevel, identifier);
}
+
+ public boolean markUrgent(ChunkCoordIntPair coords) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return this.chunkMapDistance.markUrgent(coords);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ public boolean markHighPriority(ChunkCoordIntPair coords, int priority) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return this.chunkMapDistance.markHighPriority(coords, priority);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ public void clearPriorityTickets(ChunkCoordIntPair coords) {
+ this.chunkMapDistance.clearPriorityTickets(coords);
+ }
// Paper end
@Nullable
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -470,6 +482,8 @@ public class ChunkProviderServer extends IChunkProvider {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
if (!completablefuture.isDone()) { // Paper
// Paper start - async chunk io/loading
+ ChunkCoordIntPair pair = new ChunkCoordIntPair(x, z);
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ this.chunkMapDistance.markUrgent(pair);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.world.asyncChunkTaskManager.raisePriority(x, z, com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHEST_PRIORITY);
com.destroystokyo.paper.io.chunk.ChunkTaskManager.pushChunkWait(this.world, x, z);
// Paper end
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -478,6 +492,8 @@ public class ChunkProviderServer extends IChunkProvider {
this.serverThreadQueue.awaitTasks(completablefuture::isDone);
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
com.destroystokyo.paper.io.chunk.ChunkTaskManager.popChunkWait(); // Paper - async chunk debug
this.world.timings.syncChunkLoad.stopTiming(); // Paper
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ this.chunkMapDistance.clearPriorityTickets(pair); // Paper
+ this.chunkMapDistance.clearUrgent(pair); // Paper
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
} // Paper
ichunkaccess = (IChunkAccess) ((Either) completablefuture.join()).map((ichunkaccess1) -> {
return ichunkaccess1;
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -530,6 +546,7 @@ public class ChunkProviderServer extends IChunkProvider {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
if (flag && !currentlyUnloading) {
// CraftBukkit end
this.chunkMapDistance.a(TicketType.UNKNOWN, chunkcoordintpair, l, chunkcoordintpair);
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ if (isUrgent) this.chunkMapDistance.markUrgent(chunkcoordintpair); // Paper
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
if (this.a(playerchunk, l)) {
GameProfilerFiller gameprofilerfiller = this.world.getMethodProfiler();
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -542,8 +559,13 @@ public class ChunkProviderServer extends IChunkProvider {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}
}
}
-
- return this.a(playerchunk, l) ? PlayerChunk.UNLOADED_CHUNK_ACCESS_FUTURE : playerchunk.a(chunkstatus, this.playerChunkMap);
+ // Paper start
+ CompletableFuture<Either<IChunkAccess, PlayerChunk.Failure>> future = this.a(playerchunk, l) ? PlayerChunk.UNLOADED_CHUNK_ACCESS_FUTURE : playerchunk.a(chunkstatus, this.playerChunkMap);
+ if (isUrgent) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ future.thenAccept(either -> this.chunkMapDistance.clearUrgent(chunkcoordintpair));
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+ return future;
+ // Paper end
}
private boolean a(@Nullable PlayerChunk playerchunk, int i) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -593,7 +615,7 @@ public class ChunkProviderServer extends IChunkProvider {
return this.serverThreadQueue.executeNext();
}
- private boolean tickDistanceManager() {
+ public boolean tickDistanceManager() { // Paper - public
boolean flag = this.chunkMapDistance.a(this.playerChunkMap);
boolean flag1 = this.playerChunkMap.b();
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
diff --git a/src/main/java/net/minecraft/server/EntityPlayer.java b/src/main/java/net/minecraft/server/EntityPlayer.java
index 07a6fc3d88e7d44bfab7f3d6a0eef7dc132ab422..d60f659b368500e3a8c3305f99e60ffc643e2fbd 100644
--- a/src/main/java/net/minecraft/server/EntityPlayer.java
+++ b/src/main/java/net/minecraft/server/EntityPlayer.java
@@ -441,6 +441,7 @@ public class EntityPlayer extends EntityHuman implements ICrafting {
if (valid && (!this.isSpectator() || this.world.isLoaded(new BlockPosition(this)))) { // Paper - don't tick dead players that are not in the world currently (pending respawn)
super.tick();
}
+ if (valid && isAlive() && this.ticksLived % 20 == 0) ((WorldServer)world).getChunkProvider().playerChunkMap.checkHighPriorityChunks(this); // Paper
for (int i = 0; i < this.inventory.getSize(); ++i) {
ItemStack itemstack = this.inventory.getItem(i);
diff --git a/src/main/java/net/minecraft/server/MCUtil.java b/src/main/java/net/minecraft/server/MCUtil.java
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
index 0d1065688b19ceca9440bc8bf2bf65910f03fa46..8a349964578e07e5ed13f801c57de68459220da9 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/MCUtil.java
+++ b/src/main/java/net/minecraft/server/MCUtil.java
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -684,6 +684,7 @@ public final class MCUtil {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
chunkData.addProperty("x", playerChunk.location.x);
chunkData.addProperty("z", playerChunk.location.z);
chunkData.addProperty("ticket-level", playerChunk.getTicketLevel());
+ chunkData.addProperty("priority", playerChunk.getCurrentPriority());
chunkData.addProperty("state", PlayerChunk.getChunkState(playerChunk.getTicketLevel()).toString());
chunkData.addProperty("queued-for-unload", chunkMap.unloadQueue.contains(playerChunk.location.pair()));
chunkData.addProperty("status", status == null ? "unloaded" : status.toString());
diff --git a/src/main/java/net/minecraft/server/PlayerChunk.java b/src/main/java/net/minecraft/server/PlayerChunk.java
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
index aeca6b2b9d5d73aeb6dc639b5cad2f2533a2de44..b8fe42e8123e972b1ec97b048c35d90118076e66 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/PlayerChunk.java
+++ b/src/main/java/net/minecraft/server/PlayerChunk.java
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
@@ -26,8 +26,8 @@ public class PlayerChunk {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
private CompletableFuture<IChunkAccess> chunkSave;
public int oldTicketLevel;
private int ticketLevel;
- private int n;
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
- final ChunkCoordIntPair location; // Paper - private -> package
+ volatile int n; public final int getCurrentPriority() { return n; } // Paper - OBFHELPER - make volatile since this is concurrently accessed
+ public final ChunkCoordIntPair location; // Paper - private -> public
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
private final short[] dirtyBlocks;
private int dirtyCount;
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
private int r;
@@ -40,6 +40,7 @@ public class PlayerChunk {
private boolean hasBeenLoaded;
private final PlayerChunkMap chunkMap; // Paper
+ public WorldServer getWorld() { return chunkMap.world; } // Paper
long lastAutoSaveTime; // Paper - incremental autosave
long inactiveTimeStart; // Paper - incremental autosave
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -67,6 +68,128 @@ public class PlayerChunk {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
return null;
}
// Paper end - no-tick view distance
+ // Paper start - Chunk gen/load priority system
+ volatile int neighborPriority = -1;
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ volatile int priorityBoost = 0;
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ public final java.util.concurrent.ConcurrentHashMap<PlayerChunk, ChunkStatus> neighbors = new java.util.concurrent.ConcurrentHashMap<>();
+ public final it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap<Integer> neighborPriorities = new it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap<>();
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ private int getDemandedPriority() {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ int priority = neighborPriority; // if we have a neighbor priority, use it
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ int myPriority = getMyPriority();
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ if (priority == -1 || (ticketLevel <= 33 && priority > myPriority)) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ priority = myPriority;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+
+ return Math.max(1, Math.min(PlayerChunkMap.GOLDEN_TICKET, priority));
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
+ private int getMyPriority() {
+ if (priorityBoost == 30) {
+ return 1; // Urgent - ticket level isn't always 31 so 33-30 = 3
+ }
+ int basePriority = ticketLevel - priorityBoost;
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ if (ticketLevel >= 33 && priorityBoost == 0 && (neighborPriority >= 34 || neighborPriorities.isEmpty())) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ basePriority += 5;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ return basePriority;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ private int getNeighborsPriority() {
+ return neighborPriorities.isEmpty() ? getMyPriority() : getDemandedPriority();
+ }
+
+ public void onNeighborRequest(PlayerChunk neighbor, ChunkStatus status) {
+ neighbor.setNeighborPriority(this, getNeighborsPriority());
+ this.neighbors.compute(neighbor, (playerChunk, currentWantedStatus) -> {
+ if (currentWantedStatus == null || !currentWantedStatus.isAtLeastStatus(status)) {
+ //System.out.println(this + " request " + neighbor + " at " + status + " currently " + currentWantedStatus);
+ return status;
+ } else {
+ //System.out.println(this + " requested " + neighbor + " at " + status + " but thats lower than other wanted status " + currentWantedStatus);
+ return currentWantedStatus;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ });
+
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ public void onNeighborDone(PlayerChunk neighbor, ChunkStatus chunkstatus, IChunkAccess chunk) {
+ this.neighbors.compute(neighbor, (playerChunk, wantedStatus) -> {
+ if (wantedStatus != null && chunkstatus.isAtLeastStatus(wantedStatus)) {
+ //System.out.println(this + " neighbor done at " + neighbor + " for status " + chunkstatus + " wanted " + wantedStatus);
+ neighbor.removeNeighborPriority(this);
+ return null;
+ } else {
+ //System.out.println(this + " neighbor finished our previous request at " + neighbor + " for status " + chunkstatus + " but we now want instead " + wantedStatus);
+ return wantedStatus;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ });
+ }
+
+ private void removeNeighborPriority(PlayerChunk requester) {
+ synchronized (neighborPriorities) {
+ neighborPriorities.remove(requester.location.pair());
+ recalcNeighborPriority();
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ checkPriority();
+ }
+
+
+ private void setNeighborPriority(PlayerChunk requester, int priority) {
+ synchronized (neighborPriorities) {
+ neighborPriorities.put(requester.location.pair(), Integer.valueOf(priority));
+ recalcNeighborPriority();
+ }
+ checkPriority();
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
+
+ private void recalcNeighborPriority() {
+ neighborPriority = -1;
+ if (!neighborPriorities.isEmpty()) {
+ synchronized (neighborPriorities) {
+ for (Integer neighbor : neighborPriorities.values()) {
+ if (neighbor < neighborPriority || neighborPriority == -1) {
+ neighborPriority = neighbor;
+ }
+ }
+ }
+ }
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ private void checkPriority() {
+ if (getCurrentPriority() != getDemandedPriority()) this.chunkMap.queueHolderUpdate(this);
+ }
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+
+ public final double getDistanceFromPointInFront(EntityPlayer player, int dist) {
+ int inFront = dist * 16;
+ final float yaw = MCUtil.normalizeYaw(player.yaw);
+ double rads = Math.toRadians(yaw);
+ final double x = player.locX() + inFront * Math.cos(rads);
+ final double z = player.locZ() + inFront * Math.sin(rads);
+ return getDistance(x, z);
+ }
+
+ public final double getDistance(EntityPlayer player) {
+ return getDistance(player.locX(), player.locZ());
+ }
+ public final double getDistance(double blockX, double blockZ) {
+ int cx = MCUtil.fastFloor(blockX) >> 4;
+ int cz = MCUtil.fastFloor(blockZ) >> 4;
+ final double x = location.x - cx;
+ final double z = location.z - cz;
+ return (x * x) + (z * z);
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ @Override
+ public String toString() {
+ return "PlayerChunk{" +
+ "location=" + location +
+ ", ticketLevel=" + ticketLevel + "/" + getChunkStatus(this.ticketLevel) +
+ ", chunkHolderStatus=" + getChunkHolderStatus() +
+ ", neighborPriority=" + getNeighborsPriority() +
+ ", priority=(" + ticketLevel + " - " + priorityBoost +" vs N " + neighborPriority + ") = " + getDemandedPriority() + " A " + getCurrentPriority() +
+ '}';
+ }
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ // Paper end
public PlayerChunk(ChunkCoordIntPair chunkcoordintpair, int i, LightEngine lightengine, PlayerChunk.c playerchunk_c, PlayerChunk.d playerchunk_d) {
this.statusFutures = new AtomicReferenceArray(PlayerChunk.CHUNK_STATUSES.size());
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -165,6 +288,15 @@ public class PlayerChunk {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}
return null;
}
+ public static ChunkStatus getNextStatus(ChunkStatus status) {
+ if (status == ChunkStatus.FULL) {
+ return status;
+ }
+ return CHUNK_STATUSES.get(status.getStatusIndex() + 1);
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ }
+ public CompletableFuture<Either<IChunkAccess, PlayerChunk.Failure>> getStatusFutureUncheckedMain(ChunkStatus chunkstatus) {
+ return MCUtil.ensureMain(getStatusFutureUnchecked(chunkstatus));
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ }
// Paper end
public CompletableFuture<Either<IChunkAccess, PlayerChunk.Failure>> getStatusFutureUnchecked(ChunkStatus chunkstatus) {
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -418,6 +550,7 @@ public class PlayerChunk {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
return this.n;
}
+ private void setPriority(int i) { d(i); } // Paper - OBFHELPER
private void d(int i) {
this.n = i;
}
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -436,7 +569,7 @@ public class PlayerChunk {
// CraftBukkit start
// ChunkUnloadEvent: Called before the chunk is unloaded: isChunkLoaded is still true and chunk can still be modified by plugins.
if (playerchunk_state.isAtLeast(PlayerChunk.State.BORDER) && !playerchunk_state1.isAtLeast(PlayerChunk.State.BORDER)) {
- this.getStatusFutureUnchecked(ChunkStatus.FULL).thenAccept((either) -> {
+ this.getStatusFutureUncheckedMain(ChunkStatus.FULL).thenAccept((either) -> { // Paper - ensure main
Chunk chunk = (Chunk)either.left().orElse(null);
if (chunk != null) {
playerchunkmap.callbackExecutor.execute(() -> {
@@ -501,12 +634,13 @@ public class PlayerChunk {
if (!flag2 && flag3) {
// Paper start - cache ticking ready status
int expectCreateCount = ++this.fullChunkCreateCount;
- this.fullChunkFuture = playerchunkmap.b(this); this.fullChunkFuture.thenAccept((either) -> {
+ this.fullChunkFuture = playerchunkmap.b(this); MCUtil.ensureMain(this.fullChunkFuture).thenAccept((either) -> { // Paper - ensure main
if (either.left().isPresent() && PlayerChunk.this.fullChunkCreateCount == expectCreateCount) {
// note: Here is a very good place to add callbacks to logic waiting on this.
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
Chunk fullChunk = either.left().get();
PlayerChunk.this.isFullChunkReady = true;
fullChunk.playerChunk = PlayerChunk.this;
+ this.chunkMap.chunkDistanceManager.clearPriorityTickets(location);
}
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -531,7 +665,7 @@ public class PlayerChunk {
if (!flag4 && flag5) {
// Paper start - cache ticking ready status
- this.tickingFuture = playerchunkmap.a(this); this.tickingFuture.thenAccept((either) -> {
+ this.tickingFuture = playerchunkmap.a(this); MCUtil.ensureMain(this.tickingFuture).thenAccept((either) -> { // Paper - ensure main
if (either.left().isPresent()) {
// note: Here is a very good place to add callbacks to logic waiting on this.
Chunk tickingChunk = either.left().get();
@@ -562,7 +696,7 @@ public class PlayerChunk {
}
// Paper start - cache ticking ready status
- this.entityTickingFuture = playerchunkmap.b(this.location); this.entityTickingFuture.thenAccept((either) -> {
+ this.entityTickingFuture = playerchunkmap.b(this.location); MCUtil.ensureMain(this.entityTickingFuture).thenAccept((either) -> { // Paper ensureMain
if (either.left().isPresent()) {
// note: Here is a very good place to add callbacks to logic waiting on this.
Chunk entityTickingChunk = either.left().get();
@@ -581,13 +715,29 @@ public class PlayerChunk {
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
this.entityTickingFuture.complete(PlayerChunk.UNLOADED_CHUNK); this.isEntityTickingReady = false; // Paper - cache chunk ticking stage
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.entityTickingFuture = PlayerChunk.UNLOADED_CHUNK_FUTURE;
}
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
-
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
- this.w.a(this.location, this::k, this.ticketLevel, this::d);
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ // Paper start - raise IO/load priority if priority changes, use our preferred priority
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ priorityBoost = chunkMap.chunkDistanceManager.getChunkPriority(location);
+ int priority = getDemandedPriority();
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ if (getCurrentPriority() > priority) {
+ int ioPriority = com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY;
+ if (priority <= 10) {
+ ioPriority = com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHEST_PRIORITY;
+ } else if (priority <= 20) {
+ ioPriority = com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY;
+ }
+ chunkMap.world.asyncChunkTaskManager.raisePriority(location.x, location.z, ioPriority);
+ }
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ if (getCurrentPriority() != priority) {
+ this.w.a(this.location, this::getCurrentPriority, priority, this::setPriority); // use preferred priority
+ int neighborsPriority = getNeighborsPriority();
+ this.neighbors.forEach((neighbor, neighborDesired) -> neighbor.setNeighborPriority(this, neighborsPriority));
+ }
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
+ // Paper end
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.oldTicketLevel = this.ticketLevel;
// CraftBukkit start
// ChunkLoadEvent: Called after the chunk is loaded: isChunkLoaded returns true and chunk is ready to be modified by plugins.
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
if (!playerchunk_state.isAtLeast(PlayerChunk.State.BORDER) && playerchunk_state1.isAtLeast(PlayerChunk.State.BORDER)) {
- this.getStatusFutureUnchecked(ChunkStatus.FULL).thenAccept((either) -> {
+ this.getStatusFutureUncheckedMain(ChunkStatus.FULL).thenAccept((either) -> { // Paper - ensure main
Chunk chunk = (Chunk)either.left().orElse(null);
if (chunk != null) {
playerchunkmap.callbackExecutor.execute(() -> {
@@ -669,6 +819,7 @@ public class PlayerChunk {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
public interface c {
+ default void changePriority(ChunkCoordIntPair chunkcoordintpair, IntSupplier intsupplier, int i, IntConsumer intconsumer) { a(chunkcoordintpair, intsupplier, i, intconsumer); } // Paper - OBFHELPER
void a(ChunkCoordIntPair chunkcoordintpair, IntSupplier intsupplier, int i, IntConsumer intconsumer);
}
diff --git a/src/main/java/net/minecraft/server/PlayerChunkMap.java b/src/main/java/net/minecraft/server/PlayerChunkMap.java
Fix Chunks unloading immediately for priority access - Fixes #3461 Adding a priority ticket was apparently causing the level tracker to immediately unload the chunk. That code is scary
2020-05-29 02:57:27 +00:00
index 0aa14bfca6e1845eb6e9f5bd4e0e36335fa7f532..f42507f5a17f9388db738218f58ca76f863274ff 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/PlayerChunkMap.java
+++ b/src/main/java/net/minecraft/server/PlayerChunkMap.java
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -50,6 +50,7 @@ import org.apache.commons.lang3.mutable.MutableBoolean;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bukkit.entity.Player; // CraftBukkit
+import org.spigotmc.AsyncCatcher;
public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
@@ -123,6 +124,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
@Override
public void execute(Runnable runnable) {
+ AsyncCatcher.catchOp("Callback Executor execute");
if (queued == null) {
queued = new java.util.ArrayDeque<>();
}
@@ -131,6 +133,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
@Override
public void run() {
+ AsyncCatcher.catchOp("Callback Executor run");
if (queued == null) {
return;
}
@@ -375,6 +378,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
this.playerViewDistanceTickMap = new com.destroystokyo.paper.util.misc.PlayerAreaMap(this.pooledLinkedPlayerHashSets,
(EntityPlayer player, int rangeX, int rangeZ, int currPosX, int currPosZ, int prevPosX, int prevPosZ,
com.destroystokyo.paper.util.misc.PooledLinkedHashSets.PooledObjectLinkedOpenHashSet<EntityPlayer> newState) -> {
+ checkHighPriorityChunks(player);
if (newState.size() != 1) {
return;
}
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -393,7 +397,8 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}
ChunkCoordIntPair chunkPos = new ChunkCoordIntPair(rangeX, rangeZ);
PlayerChunkMap.this.world.getChunkProvider().removeTicketAtLevel(TicketType.PLAYER, chunkPos, 31, chunkPos); // entity ticking level, TODO check on update
- });
+ PlayerChunkMap.this.world.getChunkProvider().clearPriorityTickets(chunkPos);
+ }, (player, prevPos, newPos) -> checkHighPriorityChunks(player));
this.playerViewDistanceNoTickMap = new com.destroystokyo.paper.util.misc.PlayerAreaMap(this.pooledLinkedPlayerHashSets);
this.playerViewDistanceBroadcastMap = new com.destroystokyo.paper.util.misc.PlayerAreaMap(this.pooledLinkedPlayerHashSets,
(EntityPlayer player, int rangeX, int rangeZ, int currPosX, int currPosZ, int prevPosX, int prevPosZ,
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -410,6 +415,85 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
});
// Paper end - no-tick view distance
}
+ // Paper start - Chunk Prioritization
+ private static final int[][] neighborMatrix = {{-1, 0}, {0, -1}, {0, 1}, {1, 0}};
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ public void queueHolderUpdate(PlayerChunk playerchunk) {
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ Runnable runnable = () -> {
+ if (isUnloading(playerchunk)) {
+ return; // unloaded
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ chunkDistanceManager.pendingChunkUpdates.add(playerchunk);
+ if (!chunkDistanceManager.pollingPendingChunkUpdates) {
+ world.getChunkProvider().tickDistanceManager();
+ }
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
+ };
+ if (MCUtil.isMainThread()) {
+ // We can't use executor here because it will not execute tasks if its currently in the middle of executing tasks...
+ runnable.run();
+ } else {
+ executor.execute(runnable);
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ }
+
+ public boolean isUnloading(PlayerChunk playerchunk) {
+ return playerchunk == null || MCUtil.getChunkStatus(playerchunk) == null || unloadQueue.contains(playerchunk.location.pair());
+ }
+
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ public void checkHighPriorityChunks(EntityPlayer player) {
+ MCUtil.getSpiralOutChunks(new BlockPosition(player), Math.min(7, getLoadViewDistance())).forEach(coord -> {
+ PlayerChunk chunk = getUpdatingChunk(coord.pair());
+ if (chunk == null || chunk.isFullChunkReady() || chunk.getTicketLevel() >= 34 ||
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ !world.getWorldBorder().isInBounds(coord) || isUnloading(chunk)
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ ) {
+ return;
+ }
+
+ double dist = chunk.getDistance(player);
+ // Prioritize immediate
+ if (dist <= 5) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ chunkDistanceManager.markHighPriority(coord, (int) (28 - dist));
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ return;
+ }
+ // Prioritize Frustum near
+ double distFront1 = chunk.getDistanceFromPointInFront(player, 2);
+ if (distFront1 <= (4*4)) {
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ if (distFront1 <= (3 * 3)) {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ chunkDistanceManager.markHighPriority(coord, 24);
+ } else {
+ chunkDistanceManager.markHighPriority(coord, 22);
+ }
+ return;
+ }
+ // Prioritize Frustum far
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ double distFront2 = chunk.getDistanceFromPointInFront(player, 5);
+ if (distFront2 <= (4*4)) {
+ if (distFront2 <= (3 * 3)) {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ chunkDistanceManager.markHighPriority(coord, 23);
+ } else {
+ chunkDistanceManager.markHighPriority(coord, 20);
+ }
+ return;
+ }
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+
+ boolean hasNeighbor = false;
+ /*for (int[] matrix : neighborMatrix) {
+ long neighborKey = MCUtil.getCoordinateKey(coord.x + matrix[0], coord.x + matrix[1]);
+ PlayerChunk neighbor = getUpdatingChunk(neighborKey);
+ if (neighbor != null && neighbor.isFullChunkReady()) {
+ hasNeighbor = true;
+ break;
+ }
+ }
+ if (!hasNeighbor) {
+ return;
+ }*/
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
+ // Prioritize nearby chunks
+ if (dist <= (5*5)) {
+ chunkDistanceManager.markHighPriority(coord, (int) (16 - Math.sqrt(dist*(4D/5D))));
+ }
+ });
+ }
+ // Paper end
public void updatePlayerMobTypeMap(Entity entity) {
if (!this.world.paperConfig.perPlayerMobSpawns) {
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -539,6 +623,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
List<CompletableFuture<Either<IChunkAccess, PlayerChunk.Failure>>> list = Lists.newArrayList();
int j = chunkcoordintpair.x;
int k = chunkcoordintpair.z;
+ PlayerChunk requestingNeighbor = getUpdatingChunk(chunkcoordintpair.pair()); // Paper
for (int l = -i; l <= i; ++l) {
for (int i1 = -i; i1 <= i; ++i1) {
More Improvements to Chunks Fixed issues where urgent and prioritized chunks didn't actually always get their priority boosted correctly.... Properly deprioritize non ticking chunks. Limit recursion on watchdog prints to stop flooding as much Remove neighbor priorities from watchdog to reduce information reduce synchronization duration so that watch dog won't block main should main actually wake up probably fixed a deadlock risk in watchdog printing also that was leading to crashes fixed chunk holder enqueues not being processed correctly added async catchers in some locations that should not be ran async Fixed upstream bug where VITAL callbacks that must run on main actually could sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately! This build will provide massive improvements to stability as well as even faster sync chunk load/gens now that priority is correctly set. Fixes #3435
2020-05-22 23:03:48 +00:00
@@ -557,6 +642,14 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
ChunkStatus chunkstatus = (ChunkStatus) intfunction.apply(j1);
CompletableFuture<Either<IChunkAccess, PlayerChunk.Failure>> completablefuture = playerchunk.a(chunkstatus, this);
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ // Paper start
+ if (requestingNeighbor != null && requestingNeighbor != playerchunk && !completablefuture.isDone()) {
+ requestingNeighbor.onNeighborRequest(playerchunk, chunkstatus);
+ completablefuture.thenAccept(either -> {
+ requestingNeighbor.onNeighborDone(playerchunk, chunkstatus, either.left().orElse(null));
+ });
+ }
+ // Paper end
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
list.add(completablefuture);
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
}
Unload leaked Cached Chunks Due to some complexity in mojangs complicated chain of juggling whether or not a chunk should be unloaded when the last ticket is removed, many chunks are remaining around in the cache. These chunks are never being targetted for unload because they are vastly out of view distance range and have no reason to be looked at. This is a huge issue for performance because we have to iterate these chunks EVERY TICK... This is what's been leading to high SELF time in Ticking Chunks timings/profiler results. We will now detect these chunks in that iteration, and automatically add it to the unload queue when the chunk is found without any tickets.
2020-05-25 15:12:22 +00:00
@@ -1022,14 +1115,22 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
};
CompletableFuture<NBTTagCompound> chunkSaveFuture = this.world.asyncChunkTaskManager.getChunkSaveFuture(chunkcoordintpair.x, chunkcoordintpair.z);
+ PlayerChunk playerChunk = getUpdatingChunk(chunkcoordintpair.pair());
+ int chunkPriority = playerChunk != null ? playerChunk.getCurrentPriority() : 33;
+ int priority = com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY;
+
+ if (chunkPriority <= 10) {
+ priority = com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHEST_PRIORITY;
+ } else if (chunkPriority <= 20) {
+ priority = com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY;
+ }
+ boolean isHighestPriority = priority == com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHEST_PRIORITY;
if (chunkSaveFuture != null) {
- this.world.asyncChunkTaskManager.scheduleChunkLoad(chunkcoordintpair.x, chunkcoordintpair.z,
- com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY, chunkHolderConsumer, false, chunkSaveFuture);
- this.world.asyncChunkTaskManager.raisePriority(chunkcoordintpair.x, chunkcoordintpair.z, com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY);
+ this.world.asyncChunkTaskManager.scheduleChunkLoad(chunkcoordintpair.x, chunkcoordintpair.z, priority, chunkHolderConsumer, isHighestPriority, chunkSaveFuture);
} else {
- this.world.asyncChunkTaskManager.scheduleChunkLoad(chunkcoordintpair.x, chunkcoordintpair.z,
- com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY, chunkHolderConsumer, false);
+ this.world.asyncChunkTaskManager.scheduleChunkLoad(chunkcoordintpair.x, chunkcoordintpair.z, priority, chunkHolderConsumer, isHighestPriority);
}
+ this.world.asyncChunkTaskManager.raisePriority(chunkcoordintpair.x, chunkcoordintpair.z, priority);
return ret;
// Paper end
}
Unload leaked Cached Chunks Due to some complexity in mojangs complicated chain of juggling whether or not a chunk should be unloaded when the last ticket is removed, many chunks are remaining around in the cache. These chunks are never being targetted for unload because they are vastly out of view distance range and have no reason to be looked at. This is a huge issue for performance because we have to iterate these chunks EVERY TICK... This is what's been leading to high SELF time in Ticking Chunks timings/profiler results. We will now detect these chunks in that iteration, and automatically add it to the unload queue when the chunk is found without any tickets.
2020-05-25 15:12:22 +00:00
@@ -1158,7 +1259,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
long i = playerchunk.i().pair();
playerchunk.getClass();
- mailbox.a(ChunkTaskQueueSorter.a(runnable, i, playerchunk::getTicketLevel)); // CraftBukkit - decompile error
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ mailbox.a(ChunkTaskQueueSorter.a(runnable, i, () -> 1)); // CraftBukkit - decompile error // Paper - final loads are always urgent!
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
});
}
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
diff --git a/src/main/java/net/minecraft/server/Ticket.java b/src/main/java/net/minecraft/server/Ticket.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
index 7a8397815a5b7f79f3e3a0348aeedf63fe879f8f..0d6e0f2ddaa85c04e626980591e9a78ac27fb42d 100644
Many improvements to chunk prioritization and bug fixes Fixed a few bugs, and made numerous improvements. Fixed issue where a sync chunk load could have its ticket removed and the priority ticket could expire... Still not perfect there but better than before. Also fixed few other misc issues such as watchdog cpu usage, chunk queue update had risk of double enqueue due to it no longer being a set. Added much more information about chunk state to watchdog prints. I see some more room for improvement even, but this is much better than before. Fixes #3407 Fixes #3411 Fixes #3395 Fixes #3389
2020-05-20 09:11:57 +00:00
--- a/src/main/java/net/minecraft/server/Ticket.java
+++ b/src/main/java/net/minecraft/server/Ticket.java
@@ -8,6 +8,7 @@ public final class Ticket<T> implements Comparable<Ticket<?>> {
private final int b;
public final T identifier; public final T getObjectReason() { return this.identifier; } // Paper - OBFHELPER
private long d; public final long getCreationTick() { return this.d; } // Paper - OBFHELPER
+ public int priority = 0; // Paper
protected Ticket(TicketType<T> tickettype, int i, T t0) {
this.a = tickettype;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
diff --git a/src/main/java/net/minecraft/server/TicketType.java b/src/main/java/net/minecraft/server/TicketType.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
index 8055f5998213ab1c6c10d03d88d2b14d220a5e40..24ec5d77ca7fdf12585c1bb7442554380f0c1918 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/net/minecraft/server/TicketType.java
+++ b/src/main/java/net/minecraft/server/TicketType.java
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
@@ -23,6 +23,8 @@ public class TicketType<T> {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
public static final TicketType<org.bukkit.plugin.Plugin> PLUGIN_TICKET = a("plugin_ticket", (plugin1, plugin2) -> plugin1.getClass().getName().compareTo(plugin2.getClass().getName())); // CraftBukkit
public static final TicketType<Long> FUTURE_AWAIT = a("future_await", Long::compareTo); // Paper
public static final TicketType<Long> ASYNC_LOAD = a("async_load", Long::compareTo); // Paper
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ public static final TicketType<ChunkCoordIntPair> PRIORITY = a("priority", Comparator.comparingLong(ChunkCoordIntPair::pair), 300); // Paper
+ public static final TicketType<ChunkCoordIntPair> URGENT = a("urgent", Comparator.comparingLong(ChunkCoordIntPair::pair), 300); // Paper
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
public static <T> TicketType<T> a(String s, Comparator<T> comparator) {
return new TicketType<>(s, comparator, 0L);
diff --git a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
Fix plugins calling getChunkAtAsync asynchronously While this method has async in it's name, it's not actually meant to be called asynchronously.... It just means IT will load the chunk asynchronously without blocking main. So fix this so that if a plugin calls it async, it forces the request back to main thread.
2020-05-23 22:38:29 +00:00
index 5f180bb77b736220c848357b2cac4d0d2b99e3df..d3c5b7d1904a6cbd65db406639ed2ba90ec9fd2a 100644
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
--- a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
+++ b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
Fix plugins calling getChunkAtAsync asynchronously While this method has async in it's name, it's not actually meant to be called asynchronously.... It just means IT will load the chunk asynchronously without blocking main. So fix this so that if a plugin calls it async, it forces the request back to main thread.
2020-05-23 22:38:29 +00:00
@@ -2484,6 +2484,10 @@ public class CraftWorld implements World {
return future;
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
}
Many fixes and improvements to chunk prioritization I believe this brings us back to stable. A lot of complexity was learned about juggling priorities. We were essentially promoting more chunks to urgent than really needed to be urgent. So this commit adds a lot more logic to juggle neighbor priorities and demote their priority once they meet the requirements needed of them. This greatly improves the performance of "urgent" chunks". Fixes #3410 Fixes #3426 Fixes #3425 Fixes #3416
2020-05-22 04:46:44 +00:00
+ if (!urgent) {
+ // if not urgent, at least use a slightly boosted priority
+ world.getChunkProvider().markHighPriority(new ChunkCoordIntPair(x, z), 1);
+ }
return this.world.getChunkProvider().getChunkAtAsynchronously(x, z, gen, urgent).thenComposeAsync((either) -> {
Implement Chunk Priority / Urgency System for Chunks Mark chunks that are blocking main thread for world generation as urgent Implements a general priority system so that chunks that are sorted in the generator queues can prioritize certain chunks over another. Urgent chunks will jump to the front of the line, ensuring that a sync chunk load on an ungenerated chunk does not lag the server for a long period of time if the servers generator queues are filled with lots of chunks already. This massively reduces the lag spikes from sync chunk gens. Then we further prioritize loading order so nearby chunks have higher priority than distant chunks, reducing the pressure a high no tick view distance holds on you. Chunks in front of the player have higher priority, to help with fast traveling players keep up with their movement. This commit also improves single core cpu scenarios in that we will now automatically disable Async Chunks as well as Minecrafts thread pool. It is never recommended to use async chunks on a single CPU as context switching will be slower than just running it all on main. This also bumps the number of server worker threads by default too. Mojang does not utilize the workers in an effecient manner, resulting in them using barely any sustained CPU. So give it more workers so more chunks can be processed concurrently This change also improves urgent chunk loading, so players flying into unloaded chunks will hurt a little bit less (but still hurt) Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 08:01:53 +00:00
net.minecraft.server.Chunk chunk = (net.minecraft.server.Chunk) either.left().orElse(null);
return CompletableFuture.completedFuture(chunk == null ? null : chunk.getBukkitChunk());
Reference in a new issue Copy permalink