leafanvil-ADR-002: In-Memory .lca Chunk Representation, $O(1)$ Dirty-Overlay Buffer & Non-Overlapping Slab Compaction¶
- Status: Accepted
- Date: 2026-04-19 (Updated 2026-11-13)
- Scope:
:resource:LeafAnvil(leafanvil-common,leafanvil-paper,leafanvil-fabric,leafanvil-neoforge) &:anvil-api(io.github.dailystruggle.anvil) - Related ADRs & Specs:
leafanvil-ADR-001,leafanvil-ADR-003,leafanvil-ADR-005,leafanvil-ADR-006,leafshard-ADR-001,leafshard-ADR-002,anvil-api-ADR-002,anvil-api-ADR-003,leafrtp-ADR-072,LCA-ICD-1.0(api/anvil-api/docs/ICD.md)
Context¶
leafanvil-ADR-001 intercepts chunk persistence at the RegionFileStorage / RegionFile stream boundary (NmsRegionStorageInterceptor, LcaRegionFileStorage, LcaLiveArchive), while api/anvil-api (anvil-api-ADR-002, anvil-api-ADR-003) provides the .lca (LCA_V001, LCA-ICD-1.0) binary codec and AnvilChunkView / PaletteSection reader models.
While .lca eliminates 4 KiB .mca sector slack (-39.3% to -50.2% disk footprint) and .linear whole-region Zstd bucket stalls (15.5x faster shutdown saves), profiling live servers and chunk I/O pipelines exposes three distinct memory-pressure and copy-amplification boundaries:
1. Boundary A — The Mojang NBT Serialization Round-Trip (LevelChunk $\leftrightarrow$ CompoundTag $\leftrightarrow$ byte[] $\leftrightarrow$ Map<String, Object> $\leftrightarrow$ .lca ChunkFrame)¶
- On Chunk Save (
LcaPreEncodingOutputStream/onWriteByteBuffer): - Minecraft (
ChunkSerializer#writein1.20.4–1.21.1orSerializableChunkData#writein1.21.4+/26.1+) copies everyLevelChunkSection'sPalettedContainer<BlockState>(long[]bit-storage + palette),PalettedContainer<Holder<Biome>>, andDataLayer/ StarlightSWMRNibbleArray(byte[2048]) into a heap tree of MojangCompoundTag/ListTag/LongArrayTag/ByteArrayTagobjects (~40–90 KiBof short-lived heap allocations per chunk). NbtIo#writeserializes thatCompoundTagtree into an uncompressed binary NBTbyte[]insideLcaPreEncodingOutputStream(or Paper/Folia Moonrise'sRegionFile#write(ChunkPos, ByteBuffer)).LcaRegionFileStorage#encodeChunkNbtForRegionSlabcallsNbt.readRootCompound(uncompressedNbtBytes), parsing the binary NBT stream back into a second heap tree ofLinkedHashMap<String, Object>,ArrayList, boxed primitives,long[], andbyte[](~60–120 KiBper chunk save) beforeLcaSectionCodecencodes the.lcaChunkFrame.- In
NmsRegionStorageInterceptor#onWriteByteBuffer,ByteBuffer#get(byte[])allocates an additionalbyte[payloadLen]copy beforecommitPreEncodedChunkSlab. - On Chunk Load (
LazyChunkNbtInputStream/readChunkNbtBytes): LcaRegionReaderdecodes the compact.lcaChunkFrame(~3–4.5 KiB) intoScratchArena.nbtOutBuf(128 KiBthread-local buffer) and returnsArrays.copyOf(nbtBuf, len)(~35–80 KiBuncompressed binary Mojang NBTbyte[]) wrapped in aByteArrayInputStream.- Vanilla, Moonrise, or
C2MEparses that byte stream viaNbtIo#readto allocate a full MojangCompoundTagtree in heap, andChunkSerializer#read/SerializableChunkData#parsecopies those arrays a third time intoLevelChunkSection(PalettedContainer+SimpleBitStorage).
2. Boundary B — anvil-api's In-Memory AnvilChunkView / PaletteSection Inflation¶
- When
LcaRegionReader#readChunkViewdecodes a chunk forLeafRTPorLeafMap, it materializes aPaletteSection(int sectionY, List<String> palette, long[] data)per non-uniform section wheredatais eagerly expanded into vanilla-styleYZXlong[256..512](2–4 KiBper section,~30–64 KiBper chunk), plusList<String>palette copies—even though the underlying.lcaSectionFrameis only~150–300 Bof Delta-Roaring runs (anvil-api-ADR-003Section 5.2).
3. Boundary C — NMS Live In-Memory LevelChunkSection Footprint & Naive Copy-on-Write (CoW) Spikes¶
- Steady-State Heap Overhead: For every loaded chunk on the server, Minecraft's
LevelChunkSectionholds aPalettedContainer<BlockState>backed bySimpleBitStorage. VanillaSimpleBitStorageenforces a minimum floor of 4 bits per entry (long[256]=2,048 B) even when a section contains only 2–4 distinct block states (e.g.,stone+deepslate, orwater+air, which represent~20%of all sections), and jumps tolong[342](2,736 B) as soon as a 17th block state enters the section, alongside per-sectionLinearPalette/HashMapPalette/CrudIncrementalIntIdentityHashBiMapinstances (~200–600 B) andbyte[2048]light arrays. Across10,000loaded chunks (240,000sections), livePalettedContainerandDataLayerobjects consume~500 MB–1.2 GBof old-generation heap, despite~75%–90%of loaded sections never being mutated while in memory and66.4%of all sections being completely uniform (|P| = 1). - Why Naive Section-Level Copy-on-Write (
CoW) Fails Under Scattered Edits:.lca'sBLOCK_MODE_DELTA_ROARING(TAG_DELTA_RUN) packs minority block states as contiguous variable-length delta-run byte streams. Splitting a run in-place when a block is placed changes the container byte length (+1 to +4 bytes), which shifts subsequent minority state containersp+1 .. |P|-1in thatSectionFrame. Conversely, if a cold.lcasection naively unpacked to a fulllong[256..342]SimpleBitStorageon the first single-block mutation (section-levelCoW), an explosion, redstone contraption, orWorldEditbrush touching just 1 block across each of 500 sections would trigger 500 full 4,096-voxel unpacks andlong[256..342]allocations (~1–1.5 MBof burstmemcpyand permanent heap expansion) for only 500 modified voxels.
Decision¶
We propose a three-tier architectural evolution for LeafAnvil and anvil-api, culminating in an Immutable .lca baseFrame + $O(1)$ dirtyOverlay In-Memory Section Representation paired with Region-Sharded, Non-Overlapping Background Slab Compaction built directly on top of the LCA_V001 (LCA-ICD-1.0) forward-compatible patch primitives (HAS_CHUNK_PATCH_TAIL = 0x4000, order-independent sectionOffsets[], and append-only Slab-Local Micro-Palette delta entries).
graph LR
subgraph Live_Tick["Tick Thread (Zero-Memcpy Path)"]
SetBlock["setBlockState(x,y,z)"] -->|"O(1) int write"| Overlay["dirtyOverlay int[16..64] + dirtyOctantMask64"]
GetBlock["getBlockState(x,y,z)"] -->|"Check dirtyOctantMask64"| Overlay
Overlay -->|"Miss / Bit=0"| Base["Immutable .lca baseFrame Slice"]
end
subgraph Flush_Planner["Region-Sharded Background Flush Worker (rx, rz)"]
Overlay -->|"Quiet window expired: freeze snapshot"| Frozen["(baseFrame, frozenOverlay)"]
Frozen -->|"Fast path: fits in 256B slack"| PatchTail["HAS_CHUNK_PATCH_TAIL (0x4000) / Appended SectionFrame"]
Frozen -->|"Compaction path: O(R+D) stream-merge"| Splice["New SectionFrame + Verbatim Copy of 23 Clean Sections"]
PatchTail --> NewSlab["New 256B-Aligned .lca ChunkFrame Slab"]
Splice --> NewSlab
NewSlab -->|"VarHandle CAS"| Base
NewSlab -->|"Zero-copy positional write"| Disk["r.rx.rz.lca (LcaLiveArchive)"]
end
1. Tier 1: Zero-Copy Streaming NBT Transcoder & Flyweight AnvilChunkView (:anvil-api & :leafanvil-common)¶
Target: Eliminate intermediate Map<String, Object> trees, duplicate byte[] copies in NmsRegionStorageInterceptor, and expanded long[] allocations in PaletteSection with zero NMS field coupling.
- Stream-To-Slab Direct Binary NBT Transcoding (
LcaStreamingNbtEncoder): - Replace
Nbt.readRootCompound(byte[])inLcaRegionFileStorage#encodeChunkNbtForRegionSlabwith a single-pass streaming binary NBT visitor (LcaStreamingNbtEncoder) inLcaSectionCodec. - The visitor scans the uncompressed binary Mojang NBT
ByteBuffer/byte[]directly into thread-localScratchArenabuffers (yzxPaletteIdx,skyLightNibbles,blkLightNibbles,packedBlockLongs), copying unknown non-section tags as raw byte slices straight intoresidualNbtPayloadwithout instantiatingLinkedHashMap<String, Object>,ArrayList, or boxed primitives (~60–120 KiBsaved per chunk save). - Zero-Copy
ByteBufferSlice Handoff inNmsRegionStorageInterceptor: - Expose the internal buffer of
LcaPreEncodingOutputStream(and Moonrise'sonWriteByteBufferparameter) as a zero-copyByteBufferslice passed directly toFileChannel#write(ByteBuffer, long)inLcaLiveArchive#commitPreEncodedChunkCoW, eliminatingtoByteArray()andnew byte[payloadLen]. - In
LazyChunkNbtInputStream, stream synthesized Mojang binary NBT lazily section-by-section from the~4 KiB.lcaChunkFrameinstead of allocating a fullArrays.copyOf(nbtBuf, len)(~35–80 KiB) upfront. - Flyweight
.lca-BackedPaletteSection/AnvilChunkView: - Allow
AnvilChunkViewandPaletteSectionto wrap the.lcaChunkFrameslice (byte[]orByteBuffer) directly, evaluatingblockIdAt(x, y, z)and vertical column probes over theuint16[sectionCount]directory,octreeOccupancyMask64, and Delta-Roaring containers without eagerly inflating non-uniform sections intolong[256..512].
2. Tier 2: Direct LevelChunkSection $\leftrightarrow$ .lca ChunkFrame Bridge (Bypassing CompoundTag & NbtIo)¶
Target: Eliminate Mojang CompoundTag tree construction and NbtIo byte streams on both chunk load and chunk save.
- Direct Section Extractor on Save (
LevelChunkSection$\to$LcaSectionCodec.ScratchArena): - Hook
SerializableChunkData#fromChunk(ServerLevel, ChunkAccess)(1.21.4+/26.1+) andChunkSerializer#write(1.20.4–1.21.1). - Extract each
LevelChunkSection'sPalettedContainer.Data<BlockState>directly: mapPalette<BlockState>entries to.lcadictionary IDs through a fast globalBlock.BLOCK_STATE_REGISTRYID $\to$.lcadictIdlookup table (zero string formatting orName/PropertiesCompoundTagallocation), unpackBitStorage#getRaw()intoScratchArena.yzxPaletteIdx[4096], and feedDataLayer#getData()(byte[2048]) directly into the Gray-code bitplane encoder. - Serialize only non-voxel residual fields (
block_entities,block_ticks,fluid_ticks,structures, BukkitBukkitValuesPDC) to a minimal residualCompoundTag(< 200 Bfor 95%+ of chunks). - Direct Section Hydrator on Load (
.lcaChunkFrame$\to$LevelChunkSection): - Resolve
.lcaTier-1 (STATIC_CORE), Tier-2 (region shared dictionary), and Tier-3 (slab-local micro-palette, including appended patch entries) IDs via a cachedint[] lcaIdToMojangBlockStateIdtable directly to singleton NMSBlockStatereferences, bypassing string allocations and MojangCodec/DataFixerUpperparsing wheneverchunkDataVersion == currentDataVersion.
3. Tier 3: Immutable .lca baseFrame + $O(1)$ dirtyOverlay In-Memory Section Representation¶
Target: Maintain an always-ready .lca baseFrame + $O(1)$ dirtyOverlay representation per loaded section (LcaSectionBacking, LcaInMemoryChunkCoordinator) that absorbs random single-block edits in $O(1)$ (~1–2 ns) with zero baseFrame unpacking, zero 4,096-voxel memcpy, and zero-translation network/inter-server delta export (leafanvil-ADR-003, leafshard-ADR-002).
Instead of eagerly unpacking every loaded LevelChunkSection into a 2,048 B long[256] SimpleBitStorage (or triggering a full 4,096-voxel memcpy on the first block edit), each in-memory section pairs an immutable .lca base slice with a tiny, fixed-capacity open-addressed delta overlay:
A. Base Layer (baseFrame — Immutable .lca SectionFrame Slice, ~0–300 B)¶
- Inline Uniform Sections (
66.4%of sections): Encoded entirely in the 2-byteuint16 sectionOffsets[i]header word (0x8000 | (skyLight2 << 13) | (biomeTemplate2 << 11) | localDictId11,LCA-ICD-1.0Section 6.4).baseFrameisnull(0body bytes in RAM) and backed byZeroBitStorage+SingleValuePalette. - Non-Uniform Cold Sections:
baseFramepoints directly to the section's[relStart_i .. relEnd_i)byte slice (~150–300 B) inside the chunk's256-byte-aligned.lcaChunkFrameslab in RAM (or a sub-4-bit1b/2b/octreeOccupancyMask64sparse $4 \times 4 \times 4$ octant view where only non-dominant octants with bit $k = 1$ allocate a32-bytelong[4]cell, indexed in $O(1)$ viaLong.bitCount(mask & ((1L << octantIdx) - 1))). - Shared Immutable Palettes: Deduplicate immutable
Palette<BlockState>instances across the region archive for common 2- to 4-state strata combinations (e.g.,[stone, deepslate],[air, grass_block, dirt]), eliminating per-sectionLinearPalette/HashMapPaletteobject overhead (~200–600 Bper section).
B. Delta Layer (dirtyOverlay & dirtyOctantMask64 — $O(1)$ Zero-memcpy Block Writes)¶
When a block is mutated in a loaded section (setBlockState(x, y, z, newState)):
1. 32-Bit Packed Voxel Delta Word (4 bytes), Section-Local Coordinate Masking & Zero-Sentinel Disambiguation:
- A voxel's intra-section coordinate is kept in raw native YZX order (pos12 = ((y & 15) << 8) | ((z & 15) << 4) | (x & 15), 0..4095) on the tick thread so zero space-filling curve permutation (H_3, Y+H_2, XZY) ever runs on the game thread.
- Section-Local $Y$ Masking Invariant ($Y \in [-64, 319]$): Downstream hooks and mixins (especially those intercepting world-level coordinate methods where $Y$ is absolute world space) must mask (& 15) or shift $Y$ relative to section.bottomBlockY() before computing pos12 and octantIdx = (((y & 15) >>> 2) << 4) | (((z & 15) >>> 2) << 2) | ((x & 15) >>> 2) (0..63). Passing an unmasked world-space $Y$ into pos12 would overflow the 12-bit coordinate boundary and corrupt the 13-bit (pos12 + 1) sentinel field.
- Because pos12 = 0 ((0, 0, 0)) and stateId = 0 (air) are valid values, packing stores (pos12 + 1) (1..4096) in the upper 13 bits and stateId19 (0..524,287 global BlockState ID or .lca dictionary ID) in the lower 19 bits:
$$\mathtt{entry32} = ((\mathtt{pos12} + 1) \ll 19) \mid (\mathtt{stateId19} \ \& \ \mathtt{0x7FFFF})$$
- This guarantees that entry32 == 0 is strictly the empty-slot sentinel without requiring a separate occupancy bitmask lookup during probing.
2. Two-Stage L1-Resident int[16..64] Overlay (64–256 bytes) & Load-Factor Ceiling:
- For $D \le 16$ (64 B, a single CPU L1 cache line), dirtyOverlay uses a flat unhashed int[16] scanned branchlessly in L1 cache (~1–2 ns, zero hash-collision branch mispredictions).
- When $16 < D \le 48$ (256 B, capped at a $75\%$ load factor on int[64]), dirtyOverlay promotes to a power-of-two open-addressed int[64] table indexed by upper-bit 32-bit Fibonacci hashing:
$$\mathtt{slot} = ((\mathtt{pos12} \times \mathtt{0x9E3779B9}) \ggg 26) \ \& \ 63$$
using entry32 == 0 as the empty-slot sentinel and linear probing (slot = (slot + 1) & 63) for collision resolution. (Extracting the top 6 bits via >>> 26 is mandatory: because (y << 8) is a multiple of $2^8 = 256$, a low-bit hash (pos12 * 0x9E37) & 63 would receive zero entropy from $Y$, causing vertical column edits at identical $(x, z)$—falling sand, pillar building, strip mining, fluid columns—to collide on the exact same initial slot.)
- Open-Addressed Load-Factor Invariant ($\le 75\%$ on int[64] / $\le 50\%$ on int[128]): Open-addressed linear probing degrades sharply in probe cluster length above a 75% load factor (reaching a worst-case 63-slot probe as $D \to 63$). To guarantee $O(1)$ tail-probe latency without cluster degradation, int[64] never exceeds $D = 48$ (75% load factor) before triggering Section 3.D promotion (or, if configured for a full 64-entry overlay capacity, sizes the stage-2 table to int[128] / 512 B indexed by ((pos12 * 0x9E3779B9) >>> 25) & 127 at a $\le 50\%$ load factor).
- In-Place Update & Unique-Voxel dirtyCount Invariant: When a coordinate pos12 is written more than once within the same overlay epoch (e.g., water placed and then overwritten with stone at the same (x, y, z)), the int[16] linear scan or int[64] linear probe tests (entry32 >>> 19) == (pos12 + 1) and overwrites the existing slot in-place rather than appending or inserting a duplicate pos12 entry. dirtyCount (D) is incremented only when claiming a previously empty (entry32 == 0) slot, ensuring D strictly counts unique modified voxels in the section rather than total write operations.
- dirtyOctantMask64 is a long bitmask where bit octantIdx = (((y & 15) >>> 2) << 4) | (((z & 15) >>> 2) << 2) | ((x & 15) >>> 2) (0..63, corresponding to the $4 \times 4 \times 4 = 64$-voxel sub-cube) is set to 1 whenever any voxel in that octant is written.
- Write Cost & Inline Counter Maintenance: setBlockState(x, y, z, newState) resolves oldState (via the octant-filtered read path below), updates LevelChunkSection's nonEmptyBlockCount, tickingBlockCount, and tickingFluidCount (plus Paper/Moonrise bitset-indexed ticking queues) from the oldState -> newState property delta, writes one 4-byte int into dirtyOverlay, sets dirtyOctantMask64 |= (1L << octantIdx), and records lastMutatedTick = currentTick—with zero baseFrame unpacking and zero 4,096-voxel memcpy.
C. Read Path (getBlockState(x, y, z)) & 3-Layer Compaction View¶
- Clean Fast Path (
dirtyCount == 0andfrozenOverlay == null,~85%–95%of loaded sections): Read directly frombaseFrame(or $O(1)$ inline constant / octant cell). - Octant-Filtered Dirty Path (
dirtyCount > 0orfrozenOverlay != null): - Compute
octantIdx(0..63) and test((dirtyOctantMask64 | frozenOctantMask64) & (1L << octantIdx)) == 0L. - If
0(~90%+of reads in a lightly edited section): Skip both overlays in 1 CPU cycle (1bitwiseOR+AND) and read directly frombaseFrame. - If
1(3-Layer Precedence —activeOverlay -> frozenOverlay -> baseFrame):- If
(dirtyOctantMask64 & (1L << octantIdx)) != 0L, probe activedirtyOverlay(overlayB) in L1 cache (O(1)). If(entry32 >>> 19) == (pos12 + 1), returnstateId19 = entry32 & 0x7FFFF. - Otherwise, if a background compaction is in flight (
frozenOverlay != nulland(frozenOctantMask64 & (1L << octantIdx)) != 0L), probe read-onlyfrozenOverlay(overlayA). If(entry32 >>> 19) == (pos12 + 1), returnstateId19 = entry32 & 0x7FFFF. - Otherwise, fall through to
baseFrame.
- If
- Retaining
frozenOverlayas a read-only secondary reference untilVarHandle#compareAndSetatomically publishes(newBaseFrame, null, epoch + 1)guarantees that neithergetBlockStatenor a concurrent unload/autosave snapshot (SerializableChunkData#fromChunk/ChunkSerializer#write) ever observes a split-horizon gap whilefrozenOverlayis mid-compaction.
D. Adaptive Bulk-Edit Threshold (Explosions / WorldEdit)¶
- Normal gameplay edits (player building, mining, pistons, fluid flow) keep
dirtyCount <= 48(75%load factor onint[64],64–256 B; ordirtyCount <= 64at50%load factor onint[128],512 B) inside the L1-resident overlay without ever unpacking the section. - If a bulk operation (TNT explosion,
WorldEditbrush) mutates> 48distinct voxels onint[64](or> 64onint[128]) within the same section before the background flush window triggers, promote either only the affected $4 \times 4 \times 4$ sub-octants (long[4]=32 Bper octant) or upgrade that single section toBLOCK_MODE_FLAT_BITPACK(leafrtp-ADR-072,LCA-ICD-1.0Section 7.3) until the next background slab compaction.
4. Non-Overlapping Background Slab Compaction & LCA_V001 Patch Integration¶
To fold dirtyOverlay mutations back into compact .lca frames without contending with server tick threads or rewriting untouched sections, LeafAnvil schedules background flushes using spatial region sharding, temporal quiet-window coalescing, and lock-free double-buffered CAS swaps:
A. Cooldown / "Quiet-Window" Write Coalescing (lastMutatedTick)¶
- Active sections undergoing rapid consecutive mutations (redstone clocks, pistons, water/lava spread, active player building) must not trigger background re-encoding on every tick.
- Each dirty section tracks
lastMutatedTick. WhilecurrentTick - lastMutatedTick < quietWindowTicks(default20–40 ticks/1–2 seconds) anddirtyCount <= 48(75%load factor onint[64]), all edits coalesce in-place inside the L1-residentdirtyOverlay. - Only after a section remains quiet for
quietWindowTicks(or whendirtyCountcrosses the48-entry load-factor high-water mark or world save is requested) is the chunk enqueued to the regional flush planner.
B. Lock-Free Double-Buffered Snapshot (VarHandle#compareAndSet), Tombstone & Unload/Save Epoch Fence¶
- When a chunk is selected for background flush, the planner snapshots the immutable tuple
(baseFrame, frozenOverlay, epoch)—retainingfrozenOverlay(overlayA) as a read-only secondary layer on the section—and atomically switches the live section's active delta pointer to a fresh secondarydirtyOverlayslot (overlayB, an LSM-style memtable switch). - The background flush worker operates exclusively on the immutable
(baseFrame, frozenOverlay, epoch)snapshot using its own thread-localLcaSectionCodec.ScratchArena(0heap allocations,0locks held on the liveLevelChunkSection). - Authoritative Tombstone / Epoch Race Invariant: While
frozenOverlay(overlayA) is being compacted in the background (epoch), any concurrent block mutation on the tick thread writes into the active secondaryoverlayB. Entries inoverlayBare strictly authoritative overwrites/tombstones and must never be elided or removed simply becausenewStatematchesoldBaseFrame(for example, if a voxel togglesair -> stoneinoverlayAand revertsstone -> airinoverlayBwhileoverlayAis mid-compaction). ElidingairfromoverlayBagainstoldBaseFramewould lose the revert oncenewBaseFrame(containingstone) is published viaVarHandle#compareAndSet. Deduplication againstbaseFrameis permitted only when no background flush is in flight (frozenOverlay == null) or afternewBaseFramepublication completes and the activeepochadvances. - Wait-Free Unload / Save Coalescing (
REQ-RTP-S-005Compliance):ChunkMap#save,LevelChunk#unloadCallback, andSerializableChunkData#fromChunk/ChunkSerializer#writeexecute on the main server thread (or Folia owning region thread) and must never block (CompletableFuture#join()) waiting on a background compaction worker: - Queued Task Cancellation & Single-Pass Fold: If a background flush task for
epochis still queued in the regional(rx, rz)FIFO queue when save/unload is triggered, the tick thread atomically cancels/dequeues the pending task and folds the coherent 3-layer state(baseFrame, frozenOverlay, overlayB)in a single save pass. - Mid-Flight Task & Unload Tombstone Fence (
UNLOADEDBit): Each chunk holder tracks a monotonicchunkEpoch(long) where bit63(Long.MIN_VALUE) is the terminalUNLOADEDtombstone flag. When a chunk unloads while its background compaction task is already mid-flight:- The save path extracts from the coherent 3-layer view
(baseFrame, frozenOverlay, overlayB)(or chains the final(+ overlayB + residualNbt)commit onto the same single-worker(rx, rz)regional queue behind the in-flight task) and setschunkEpoch |= UNLOADEDviaVarHandle. - If the in-flight background worker finishes after unload, its
VarHandle#compareAndSetonchunkEpochobserves theUNLOADEDbit, aborts in-memory publication, and recycles its scratch buffers without touching the unloadedLevelChunkSection.
- The save path extracts from the coherent 3-layer view
- Monotonic
lastCommittedEpoch[1024]Guard inLcaLiveArchive:LcaLiveArchive#commitPreEncodedChunkCoWtrackslong lastCommittedEpoch[1024]per chunk slot(cx & 31, cz & 31). Any commit carryingtaskEpoch <= lastCommittedEpoch[slotIdx]is rejected as stale, guaranteeing that a background compaction task (epoch N) can never overwrite a newer unload or autosave slab (epoch N + 1) inr.<rx>.<rz>.lca.
C. Two-Mode Flush Leveraging LCA-ICD-1.0 (LCA_V001) Patch Primitives¶
Because LCA-ICD-1.0 (anvil-api-ADR-003) bakes patch-level primitives directly into LCA_V001, the background flush worker has two zero-waste flush strategies:
- Mode 1 — Fast Tail-Slack Patch (
HAS_CHUNK_PATCH_TAIL = 0x4000& Out-of-OrdersectionOffsets[]): - Every
.lcaChunkFrameis padded to256-bytesub-slabs (slabShift = 8), leaving an average of~128 bytesof natural tail slack inside the chunk's existing slab allocation. -
Furthermore, the in-memory
dirtyOverlayis structurally isomorphic toLCA-ICD-1.0Section 8.3 (boilerplateFlags16bit14=0x4000,HAS_CHUNK_PATCH_TAIL):- New block states or biome templates not yet in the chunk's
Slab-Local Micro-Paletteare appended toappendedPartACount/appendedPartBCount(assigned to local indicesK .. K + M - 1so existing local indices0 .. K - 1in the 23 untouched sections never shift). - For sparse edits (
1..32voxels), the worker serializes the sorted(pos12, localStateIdx)delta pairs directly intoChunkPatchTail(protected bypatchParity8,CBIT T6b) inside the trailing256Bsub-slab slack. - Alternatively, when a single section
iis re-encoded into a newSectionFrame(~150–300 B), order-independent section bounding (LCA-ICD-1.0Section 6.4: $\mathtt{relEnd}_i = \min{\mathtt{relStart}_j > \mathtt{relStart}_i, \mathtt{residualRelStart}, \mathtt{frameLen}}$) allows appending the newSectionFrame[i]at the tail of the chunk frame and updating eithersectionOffsets[i](2 bytes) orPATCH_FLAG_REPLACEMENT_OFFSET(0x02) without shifting or touching the other 23SectionFrames.
- New block states or biome templates not yet in the chunk's
-
Mode 2 — Off-Thread Curve Permutation & $O(R + D)$ Stream-Merge Section-Splice Compaction:
- When merging
frozenOverlayintobaseFrame:- Because
dirtyOverlaystorespos12in rawYZXcoordinates to keep space-filling curve math off the tick thread, the background flush worker first permutes the $D \le 64$frozenOverlaycoordinates into the section's active curve order (00:YZX,01:H_3,10:Y+H_2, or11:XZY) and sorts those $\le 64$ integers inside its thread-localScratchArena. - If
baseFrameusesBLOCK_MODE_DELTA_ROARING(TAG_DELTA_RUN), the worker stream-merges the curve-sorted delta coordinates directly into the existingTAG_DELTA_RUNcontainers in $O(R + D)$ time (< 500 ns, where $R \approx 30$ runs) without decoding all 4,096 voxels intoyzxPaletteIdx[4096]. - For the 20–23 unmodified sections in that chunk (
dirtyCount == 0), the worker copies their existing compressedSectionFramebyte slices ([relStart_j .. relEnd_j),~3 KiBtotal) verbatim viaSystem.arraycopywithout decoding or re-encoding their blocks, biomes, or Gray-code light bitplanes.
- Because
- The resulting
256B-alignedChunkFrameslab simultaneously serves as:- The new in-memory
baseFramebacking slab for all 24 sections of the live chunk (published viaVarHandle#compareAndSet, releasing the old slab and recyclingfrozenOverlayto the pool), and - The exact byte-for-byte pre-encoded
.lcadisk slab staged for zero-copy positional write (FileChannel#write(ByteBuffer, slabOffset)) inLcaLiveArchiveon world autosave.
- The new in-memory
D. Spatial Non-Overlapping Region Sharding (rx, rz)¶
- Flush queues are partitioned by
.lcaregion coordinates(rx, rz)(r.<rx>.<rz>.lca), aligning 1-to-1 with Folia's regional tick ownership (Rule F-001..F-003) andLcaLiveArchiveinstances. - At most one background flush worker processes a given
(rx, rz)region archive at any time, ensuring zero cross-thread lock contention on the region'sTier-2 Shared Dictionary(0x2040), theuint64[1024]atomic pointer directory, and the256-bytesub-slabTreeMapfree-list allocator.
5. Lighting Lifecycle & Paper / Moonrise / Folia / Mod Compatibility¶
A. Lighting (LIGHT_MODE_CONST & LIGHT_MODE_GRAY_PLANES)¶
- Cold Uniform Light (
~75%+of sections): - Sections with uniform
SkyLight(e.g.,0underground or15open sky) and uniformBlockLight(0) are represented in.lcabyLIGHT_MODE_CONST = 01(1 byte:uint8 constLightNibbles = (blkConst4 << 4) | skyConst4, or inlineskyLight2insectionOffsets[i]). - In RAM, these sections stay unallocated (
0 Bbyte[]array), mapping directly to Paper/Starlight's constant/uninitializedSWMRNibbleArraystate (or a shared read-only constant table on Fabric/NeoForge with Copy-on-Write clone onset(...)), saving2,048–4,096 Bper section. - Active Light Propagation Wave (Hot State & Atomic CAS Coordination):
- Running BFS flood-fill light updates (
Starlight/ThreadedLevelLightEngine) directly on RLE-compressedLIGHT_MODE_GRAY_PLANESbitplanes would re-encode bitplanes hundreds of times per torch placement. - Therefore, when a light propagation wave enters a section, it operates on a hot mutable
byte[2048](SWMRNibbleArray) or raw 4-bitplanelong[64]bitmap (11 = Raw 512B BitmapinLCA-ICD-1.0Section 7.4, where nibble updates are $O(1)$ bitwise word operations). - Concurrent Light Thread Coordination (
VarHandle/AtomicReferenceCAS): BecauseThreadedLevelLightEngineand Paper/Folia Starlight execute light updates on asynchronous worker threads decoupled from the region tick thread runningsetBlockState, both promotion fromLIGHT_MODE_CONSTto the mutable hot buffer and demotion/compaction back toLIGHT_MODE_CONST/LIGHT_MODE_GRAY_PLANESmust use an atomic compare-and-set (VarHandle#compareAndSet/AtomicReference<byte[]>, or Starlight'sSWMRNibbleArraystate lock/CAS protocol) so concurrent block mutations, light propagation workers, and background slab compaction threads never race when allocating or reclaiming the2,048 Blight buffer. - Once the section's
lastMutatedTickquiet window expires (1–2 secondsafter light propagation settles), the background flush worker encodes the settled light array intoLIGHT_MODE_CONST(1 B) orLIGHT_MODE_GRAY_PLANES(~30–120 Bin the chunk slab) and atomically swaps the cold representation back in before releasing the2,048 Bbyte[]buffer to the pool.
B. Paper / Moonrise (BlockCountingBitStorage), Lithium & Network Packet Streaming¶
- Inline Ticking & Non-Empty Block Counter Deltas (
LevelChunkSection#setBlockState&hasOnlyAir()): - Vanilla and Paper/Folia's Moonrise engine (
ca.spottedleaf.moonrise.patches.block_counting, which replaces vanilla ticking lists with bitset-indexed linked queues) maintain live per-section counters (nonEmptyBlockCount,tickingBlockCount,tickingFluidCount) and ticking coordinate bitsets insideLevelChunkSection/PalettedContainerto drive random block ticks, fluid ticks, and mob spawning. - When
dirtyOverlayinterceptssetBlockState(x, y, z, newState)in Tier 3 without materializingPalettedContainer, it must resolveoldState(viagetBlockState(x, y, z)) and apply the exactoldState -> newStatedelta tononEmptyBlockCount,tickingBlockCount,tickingFluidCount, and Moonrise's bitset-indexed ticking queues so ticking and spawning behavior never drift. - Entry Coordinate Masking & Release-Store Visibility on
nonEmptyBlockCount(hasOnlyAir()): AtLevelChunkSectionandPalettedContainerhook entrypoints, coordinates shall always be masked (x & 15,y & 15,z & 15) before computingpos12oroctantIdx. On Vanilla, Fabric, NeoForge, and Paper/Folia,LevelChunkSection#hasOnlyAir()(nonEmptyBlockCount == 0) is checked as a fast-path by entity collision and AI pathfinding workers executing across off-thread / asynchronous worker pools. Updates tononEmptyBlockCount,tickingBlockCount, andtickingFluidCountduringdirtyOverlaywrites must therefore publish with volatile or release-store semantics (VarHandle#setRelease/setVolatile) strictly after writingdirtyOverlayanddirtyOctantMask64(and invokingTickingCoordinateCallbackfor Paper/FoliaMoonriseSimpleRandomTickList) so off-thread pathfinders and collision checks never observe a non-empty count before the overlay entry is visible. - Network Chunk Packet Streaming (
ClientboundLevelChunkWithLightPacket&leafanvil-ADR-003Negotiated.lcaStream): - For unmodified vanilla clients, hook
LevelChunkSection#write(FriendlyByteBuf)/PalettedContainer#writeto stream the standard Mojang network wire format directly from(baseFrame, dirtyOverlay)usingScratchArena's thread-locallong[1024]scratch buffer—avoiding permanent heap inflation of cold sections when players walk across chunks. - For negotiated Fabric and NeoForge clients running
LeafAnvil(leafanvil-ADR-003),PlayerChunkSenderslices(baseFrame, dirtyOverlay)directly intoClientboundLcaChunkBatchPayload(O(1)System.arraycopy, zero curve-to-YZXunpacking on the server) and attaches the.lcaslab directly to the client'sClientLevelLevelChunkSection. - Non-Pinning Ephemeral Copy-on-Read for Client Meshers (
SodiumWorldSlice) &PalettedContainerShell Interception: - Client chunk meshers (Sodium
WorldSlice/ChunkRenderRebuildTaskand vanillaSectionCompiler) callPalettedContainer#copy()on the client main thread to hand an isolated section snapshot to background meshing worker threads, while multi-block update packets (ClientboundSectionBlocksUpdatePacket) mutateLevelChunkSection/PalettedContainerentries in batches. - Hooking both
LevelChunkSection(MixinLevelChunkSection) andPalettedContainer(MixinPalettedContainer) ensures that multi-block updates always write intoLcaSectionBacking, whilePalettedContainer#copy()materializes a transient snapshot viaLcaSectionBacking#materializeEphemeralStateIds(THREAD_LOCAL_EPHEMERAL_VOXELS) without permanently pinning an inflatedSimpleBitStoragelong[256..342]back onto the liveLevelChunkSection.
Consequences & Empirical Production Outcome¶
Empirical Outcome: Why It Did Not Reduce Server RAM in Practice — and How It Enabled LeafShard¶
- No Net Server JVM Heap Reduction in Practice (
README.mdTable 3): - Although an isolated
LcaSectionBackinginstance (0 Bbody for66.4%uniform sections;~150–300 BbaseFrame+64–256 BdirtyOverlayfor non-uniform sections) is smaller than a standalone2,048–2,736 BSimpleBitStorage, in live multi-platform server benchmarks (120runs across Paper, Folia, Fabric+ C2ME, and NeoForge+ C2ME) overall JVM heap usage did not decrease (~10.4–14.0 GBpeak heap across.lcaarms vs~10.3–15.5 GBon.mca). - In a running Minecraft server, NMS/Moonrise
LevelChunkSectioninstances, lighting engines (Starlight/ThreadedLevelLightEngine), entity/POI trackers, and Copy-on-Read callers (BitStorage#getRaw(),PalettedContainer#copy()) either retain livePalettedContainerstate alongsideLcaInMemoryChunkCoordinator's trackedLcaSectionBacking[]columns or dominate G1GC region occupancy between collection cycles. - The Decisive Win — Zero-Translation Network Factor Enabling
LeafShard(leafanvil-ADR-003,leafshard-ADR-002): - What
(baseFrame, dirtyOverlay)(LcaSectionBacking) andLcaInMemoryChunkCoordinatordid unlock in practice is keeping every loaded chunk continuously staged in wire-ready.lcadelta form:- Every block mutation is already captured as a single
4-byteentry32 = ((pos12 + 1) << 19) | (stateId19 & 0x7FFFF)word insidedirtyOverlay, flagged indirtySectionBitmask32, and anchored by a 64-bit Extended Hamming(72, 64)baselineSecDed64syndrome. - Exporting a dirty chunk delta (
LcaInMemoryChunkCoordinator#exportInterServerDeltaEntry) or full chunk (toWireFullChunkEntry) requires zero main-threadPalettedContaineriteration, zero MojangCompoundTag/NbtIoserialization, and zero curve re-encoding—just an $O(1)$ slice ofbaseFrameplusexportMergedDirtyOverlayEntries(). - This network factor directly made
LeafShard(leafshard-ADR-001..003) viable: authoritative shards stream4-byte-per-blockLCAS(0x4C434153)EntryPatchDeltaframes and self-healingEntryFullChunkrefills over peerLSHD(0x4C534844) channels to keep10-chunk(160-block) cross-border Halo rings synchronized in real time across multiple backend servers without saturating network links or server tick threads.
- Every block mutation is already captured as a single
Positive¶
- Eliminates Save/Load & Wire Serialization Churn (Tiers 1 & 2): Cuts chunk load, save, and network/inter-server transfer temporary heap allocations by
85%–95%by removing theCompoundTag$\leftrightarrow$byte[]$\leftrightarrow$LinkedHashMapround-trip and string palette parsing. - Enables Sub-Kilobyte
LCASInter-Server Halo Streaming forLeafShard(Tier 3): BecauseLcaInMemoryChunkCoordinatortracks(baseFrame, dirtyOverlay),dirtySectionBitmask32, andbaselineSecDed64per loaded chunk column,LeafShardreplicates cross-border10-chunkHalo rings across peer server nodes at4 bytesper mutated voxel (EntryPatchDelta) with hardware-styleSEC-DEDsyndrome verification (leafshard-ADR-002). - $O(1)$ (
~1–2 ns) Single-Block Mutations with Zero Unpackmemcpy: Scattered single-voxel edits write a single4-byte((pos12 + 1) << 19) | (stateId19 & 0x7FFFF)intintodirtyOverlayand set 1 bit indirtyOctantMask64, completely avoiding the1–1.5 MBburstmemcpyspikes of naive section-level Copy-on-Write. - Near-Zero Autosave CPU via Unified In-Memory / On-Disk
.lcaSlabs: Background flush workers compact only the1–2dirty sections per chunk (< 500 nsvia $O(R + D)$TAG_DELTA_RUNstream-merge orHAS_CHUNK_PATCH_TAIL = 0x4000tail-slack append) and copy the20–23cleanSectionFrames verbatim (System.arraycopy), producing the exact256B-aligned.lcaslab needed for disk persistence off-thread before autosave even begins. - Cold-by-Default 3D Slab Stratification & Bounded AIMD 3D Cache (
leafanvil-ADR-006): Unmodified16x16x163D chunk slabs immediately drop unpackedCompactBaseIndextables after encoding their.lcabaseFrame, while edited 3D slabs are retained inside a bounded $N$-slab LRU cache scored by Additive Increase / Multiplicative Decrease (AIMD) edit counts. - Zero
.lcaFormat Breakage: Relies entirely onLCA_V001(LCA-ICD-1.0) primitives already locked intoanvil-api-ADR-003(HAS_CHUNK_PATCH_TAIL = 0x4000, order-independentsectionOffsets[], and append-onlySlab-Local Micro-Paletteentries).
Trade-Offs & Risks¶
- Heap-Neutral Coexistence with Live NMS
LevelChunkSectionState: On live servers,LcaInMemoryChunkCoordinatortracksLcaSectionBackingcolumns alongside NMS/Moonrise chunk holders rather than shrinking total JVM heap; the architectural payoff is network bandwidth reduction (leafanvil-ADR-003) and zero-translation inter-server sharding (LeafShard), not server RAM reduction. - Loader & NMS Hook Surface (Tiers 2 & 3): While Tier 1 is 100% internal to
:anvil-apiand:leafanvil-common, Tiers 2 and 3 hookLevelChunkSection,PalettedContainer, andBitStorageacross Paper/Folia (1.20.4–26.1+), Fabric (26.1+), and NeoForge (26.1+). Every NMS hook must degrade gracefully to Tier 1 if an unknown server fork replacesPalettedContainerinternals. - External
BitStorage#getRaw()Pinning: Mods or server patches that unconditionally callBitStorage#getRaw()on every section at chunk load time trigger the Copy-on-Read fallback unless routed throughLeafAnvil'sBlockCountingBitStorage/ collision bridge; Moonrise block-counting integration therefore computes non-air / ticking block counts directly from.lca'sSectionFramerun totals anddirtyOverlaytransitions during hydration. - Double-Buffered Overlay Lifecycle: Background compaction workers and tick threads must strictly honor the
VarHandle#compareAndSetepoch contract on(baseFrame, frozenOverlay)so concurrent block writes on the owning region thread are never lost or reordered across a compaction boundary.