Skip to content

leafconfig-ADR-002 - Selective Cross-Server Fleet Config Synchronization and Third-Party Plugin Lifecycle Control

Status: Accepted Date: 2026-10-27 Formerly: ADR-128 Extends: leafconfig-ADR-001, leafrtp-ADR-022, leafrtp-ADR-055, leafrtp-ADR-079, leafrtp-ADR-080, leafrtp-ADR-082

Context

Operators running multi-server Minecraft networks (Velocity or BungeeCord fronting multiple Paper, Folia, Fabric, or NeoForge backends such as lobby-1..2, survival-1..4, resource-1..2) must keep dozens of server engine and third-party plugin configurations synchronized across hosts without clobbering host-specific settings.

Existing operational approaches fail in three ways: 1. Manual SFTP Drift vs. Blind Shared Mounts: Copying files manually across 10+ hosts causes configuration drift, whereas sharing plugins/ over NFS/rsync overwrites host-specific keys (serverId, proxyId, local world names, ports, bStats UUIDs, or role-specific limits like lobby vs. survival view distances). 2. Third-Party onDisable { saveConfig() } Clobbering: Many third-party plugins hold an in-memory FileConfiguration and unconditionally call saveConfig() during onDisable or periodic autosaves, silently overwriting any external configuration updates written to disk while the server was running. 3. Inconsistent Reload Lifecycles & Offline Hosts: Third-party plugins expose non-standard reload commands (/ess reload, /wg reload, /lp sync) or require a full JVM restart, and hosts that are offline or restarting during a fleet update miss transient pub/sub broadcasts unless target state is durably versioned in a shared database.

Decision

Extend the leafconfig-ADR-001 Web IDE and database-api / SqlNetworkStateBinding persistence layer with Selective 3-Layer Fleet Configuration Synchronization and External Third-Party Lifecycle Control, allowing an operator to inspect, diff, validate, and update an entire multi-server network from a single Web Panel.

1. Three-Layer Selective Sync Model (FLEET_GLOBAL, SELECTIVE_GROUP, PINNED_LOCAL)

Every configuration key (artifact_id, file_path, key_path) across the fleet is governed by one of three synchronization modes in network_sync_policies:

  1. FLEET_GLOBAL: Synced identically across all hosts in the network that have the artifact installed (e.g., messages.yml branding, shared network schema versions, global permission mappings).
  2. SELECTIVE_GROUP (TARGET_SET): Synced across an explicit subset of hosts or role tags (e.g., role:survival covering survival-1..4 vs. role:lobby covering lobby-1..2).
  3. PINNED_LOCAL (Host-Locked): Locked to each individual host_id and never overwritten by fleet-wide or group commits (e.g., network.serverId, network.proxyId, local database credentials, server ports, bStats serverUuid, or per-host world names).

2. Automatic PINNED_LOCAL & Split-Key Protection for Third-Party Plugins

Because third-party plugins do not declare # @sync: local directives, LeafConfig protects host-specific settings automatically during database ingestion and in the Web Panel UI:

  1. Cross-Host Entropy & Pattern Scan on Initial Import:
  2. When hosts publish their initial config snapshots to the database, any key whose value matches a UUID pattern, IP/port, hostname, credential key (password, secret, token, jdbc:, url), or matches the host's folder/world identifier is automatically classified as PINNED_LOCAL.
  3. Implicit Split-Key Preservation in the Web Panel:
  4. When an operator opens a third-party configuration file in the Fleet Web Panel, keys that already have identical values across the selected target group default to group-synced editing.
  5. Keys whose live values differ across hosts in the selected group are badged as Split Across Hosts — Kept Separate Unless Unified, displaying an inline per-host value matrix under the YAML line so the operator never overwrites a host-specific value unless they explicitly click Unify Value Across Selected Hosts.

3. Shared SQL Schema for Per-Server State & Signed Fleet Commits

Persist per-host configuration snapshots, selective sync rules, and transactional fleet commits in the shared SQL database (PostgreSQL, MySQL, MariaDB, H2) alongside backend_state and proxy_state:

CREATE TABLE IF NOT EXISTS network_server_configs (
    host_id TEXT NOT NULL,                 -- e.g. 'survival-1', 'lobby-1', 'proxy-1'
    host_role TEXT NOT NULL,               -- e.g. 'backend:paper', 'backend:folia', 'proxy:velocity'
    artifact_id TEXT NOT NULL,             -- e.g. 'plugin:leafrtp', 'engine:paper', 'plugin:essentialsx'
    file_path TEXT NOT NULL,               -- e.g. 'config.yml', 'paper-world-defaults.yml'
    revision_seq BIGINT NOT NULL,          -- Monotonic version counter
    content_sha256 TEXT NOT NULL,          -- Fast fleet drift detection
    yaml_content TEXT NOT NULL,            -- Comment-preserved YAML via RtpYamlConfig
    flat_keys_json TEXT NOT NULL,          -- Flattened {"key.path": value} for SQL/UI matrix diffing
    updated_at_epoch_ms BIGINT NOT NULL,
    PRIMARY KEY (host_id, artifact_id, file_path)
);

CREATE TABLE IF NOT EXISTS network_sync_policies (
    artifact_id TEXT NOT NULL,
    file_path TEXT NOT NULL,
    key_path TEXT NOT NULL,
    sync_mode TEXT NOT NULL,               -- 'PINNED_LOCAL', 'FLEET_GLOBAL', 'TARGET_SET'
    target_hosts_json TEXT NOT NULL,       -- '["*"]', '["survival-1","survival-2"]', or '["role:survival"]'
    canonical_value_json TEXT,
    updated_by TEXT,
    updated_at_epoch_ms BIGINT NOT NULL,
    PRIMARY KEY (artifact_id, file_path, key_path)
);

CREATE TABLE IF NOT EXISTS network_config_commits (
    commit_id TEXT PRIMARY KEY,
    author TEXT NOT NULL,
    summary TEXT NOT NULL,
    created_at_epoch_ms BIGINT NOT NULL,
    target_hosts_json TEXT NOT NULL,
    key_deltas_json TEXT NOT NULL,         -- [{artifact_id, file_path, key_path, old_val, new_val}]
    hmac_signature TEXT NOT NULL           -- Signed with RTP_NET_SECRET (rtp-proxy-ADR-010)
);

4. Fleet Web Panel Workflow & Cross-Server Topology Validation

  1. Server Target Selector & Drift Matrix:
  2. The leafrtp-ADR-080 Web Panel header renders a Server Matrix Pill Bar (All Hosts, Proxies, Lobbies, Survival, or individual hosts) with real-time drift badges (Synced N/N, Split, Pinned Per-Host) computed from network_server_configs.flat_keys_json.
  3. Cross-Server Pre-Commit Validation (NetworkConfigCheck):
  4. Before committing a multi-server transaction, config-bit-api validates cross-host references against backend_state and config_artifacts rows in the database (for example, verifying that a world or region referenced in a proxy or lobby routing rule exists in the target backend server's worldsLoaded[] / regionsAvailable[], and preventing Folia-only keys from being pushed to a Fabric host).

5. Third-Party Reload Orchestration & Anti-Clobber Shutdown Guard

When a targeted host observes a new HMAC-signed commit (revision_seq > local_revision) via Redis PUBSUB, database notification, or the SqlNetworkStateBinding poll loop—or when an offline host boots and catches up against network_server_configs:

  1. Comment-Preserving Selective Merge:
  2. Off the main thread, the host writes a .bak backup of the target file, merges only the committed key deltas via RtpYamlConfig while preserving all local comments and PINNED_LOCAL keys, and records the expected content_sha256.
  3. Per-Artifact reload_strategy Execution:
  4. NATIVE_SPI: For Leaf* plugins and config-bit-api consumers, invokes the native transactional reload hook directly.
  5. COMMAND_DISPATCH: For third-party plugins with a known or Plane-1-discovered reload subcommand (<main-command> reload), dispatches the reload command on the global/main thread and captures console feedback.
  6. BUKKIT_FILECONFIG_INJECT: For standard Bukkit JavaPlugin instances without a dedicated reload command, invokes plugin.reloadConfig() on the main thread.
  7. DEFER_UNTIL_SHUTDOWN (Anti-Clobber Post-Disable Hook): For plugins that do not support hot reload or that call saveConfig() inside onDisable(), registers a post-PluginDisableEvent listener and JVM shutdown hook that re-applies the authoritative network_server_configs YAML after the third-party plugin finishes onDisable(), ensuring in-memory FileConfiguration dumps cannot overwrite synced fleet changes.

Alternatives Considered

Alternative Why Rejected
Shared network filesystem (NFS / rsync) across plugins/ Overwrites PINNED_LOCAL identity keys (serverId, ports, UUIDs), cannot express role:lobby vs. role:survival selective overrides, and corrupts files when multiple JVMs write simultaneously.
Ephemeral pub/sub broadcast only (no SQL state table) Offline or restarting servers miss configuration updates and boot with stale configs; operators cannot view cross-server drift matrices in the Web Panel without querying every host live.
Write disk files without post-PluginDisableEvent guard Third-party plugins that call saveConfig() inside onDisable() silently revert synced changes on the next server restart.

Consequences

  • Positive:
  • Operators can configure an entire Velocity/BungeeCord + Paper/Folia/Fabric/NeoForge network from a single Web Panel, even for third-party plugins whose source code they do not control.
  • Offline servers automatically catch up to the latest revision_seq on boot before enabling listeners or regions.
  • Host-specific keys (PINNED_LOCAL) and intentional per-server differences (Split Across Hosts) are protected from accidental fleet-wide overwrites.
  • Negative / Trade-offs:
  • Requires a shared SQL database (PostgreSQL, MySQL, MariaDB, or networked H2) across participating hosts, reusing the database-api connection pool (leafrtp-ADR-079).
  • Third-party plugins classified as DEFER_UNTIL_SHUTDOWN stage their updated YAML in the database immediately but require a scheduled restart for the runtime JVM state to reflect the change.

References