Deep Dive

Near caching and synthetic patches: warm reads and deltas at the edge

How the optional near cache on every gateway keeps a warm copy, how to prime and stop it, and how synthetic patch generation turns plain writes into deltas in the cluster and at the edge.

Why it matters

Every gateway is a client of the cluster. Without help, each GET is a round trip to the RAFT core, and each new subscriber that asks for a snapshot is one more snapshot the cluster has to build and send. Reads dominate most workloads, and the core is deliberately a fixed-size consistency tier (see scaling the gateways), so asking it to answer every read is the wrong place to spend its capacity.

The near cache moves that work to the edge. A gateway keeps a warm copy of the caches it serves, keeps it current from the cluster’s own update stream, and answers from it. Synthetic patch generation is the companion idea for subscribers: even when a writer only knows how to put a whole document, patch-mode subscribers can still receive just the fields that changed.

Both are transparent. The URIs, request shapes and client code do not change, so you turn them on with a deployment option, not a code change.

It is a switch on every gateway

The near cache is not a feature of one interface. It is available on all four gateways: HTTP REST, WebSocket (one-directional and bidirectional), SSE, and the Aeron SBE gateway. Each reads the same NEAR_CACHE_ENABLED setting (Helm: nearCacheEnabled; default false in the app), and each keeps its own independent copy. A replica with it on holds the caches it has warmed; a replica with it off simply relays to the cluster.

The rest of this page uses the HTTP gateway as the worked example because its behaviour is the easiest to follow, and calls out where the streaming gateways differ. The mechanism, the admin surface and the metrics are the same everywhere.

How a cache gets warm

On HTTP, with the flag on, the regular GET read routes are bound to near-cache handlers instead of the strong cluster-read handlers. There are no separate /near/ read URIs. Paraphrased, the item read looks like this:

if (nearCacheManager.contains(cacheId)) {
    var local = nearCacheManager.get(cacheId);
    if (local.containsKey(key)) {
        return local.get(key);                    // served from the warm copy
    }
}
subscriptionService.primeNearCache(cluster, cacheId);  // start warming this cache
return unknownKey(cacheId, key);                       // the near cache never reads through to the cluster

Three things follow:

  • A near-cache read never goes to the cluster. A key the copy holds is returned from it. Anything else returns an unknown key (or unknown cache) response, and the read starts warming the cache in the background, so a later read can succeed.
  • Priming fills every key. The internal subscription requests a snapshot that arrives asynchronously and fills the copy for every key, not just the one you asked for. You can also prime explicitly, as described below.
  • Updates keep it warm. That one internal subscription delivers every later ADD_ITEM, REMOVE_ITEM, PATCH_ITEM and clear or delete event, and a NearCacheManager (ConcurrentNearCacheManager) applies each to the local copy.

Whole-cache reads (GET /api/v1/cache/{cacheId}) work the same way: a warm cache returns its local snapshot, a cold one starts warming and returns an unknown cache response. Writes, cache creation and deletes always go to the cluster. The near cache never accepts a write, it only observes them through its subscription.

On the streaming gateways

WebSocket, SSE, bidirectional and Aeron gateways warm the cache from hydrating subscriptions, the ones that ask for a snapshot (sendSnapshot). The first whole-cache subscriber to a cold cache hydrates from a single cluster snapshot, and the gateway tees that snapshot into the near cache while forwarding it. A single internal subscription then keeps the copy warm even after that client disconnects.

Later hydrating subscribers are served entirely from the local copy, with no per-subscriber cluster snapshot:

  • Whole-cache subscribers get the full copy.
  • Key-based subscribers get just the keys they asked for, replayed from the copy and then kept in sync, filtered to those keys.
  • A key-based subscription to a cold cache takes the plain cluster path. A snapshot of a few keys can never warm a full local copy, so warm the cache first with a whole-cache subscription or an explicit prime.

The Aeron gateway keeps its allocation-free dispatch on the hot path (a reused flyweight) and feeds the near cache through an allocation-free overload, so enabling it does not add per-message garbage there.

Same client, either way

Because nothing about the call changes, the same embedded client works against a gateway with or without a near cache:

var client = new AeronCacheClient("http://localhost:7070", "ws://localhost:7071");
EmbeddedAeronCache cache = client.getCache("quotes");

cache.subscribe(event -> {}, true);   // hydrate: from the warm copy if the gateway has one
cache.put("EURUSD", "1.0871");        // writes always go to the cluster
cache.get("EURUSD");             // in-process mirror, no network call

Point it at a gateway with NEAR_CACHE_ENABLED=true and the hydration snapshot comes from the gateway’s copy. Point it at one without and the cluster supplies it. The two layers compose: the embedded mirror lives in your process, the near cache lives in the gateway, and the first can hydrate from the second. The architecture page shows the three places a read can land.

Admin: prime, list, stop

Sometimes you want a cache warm before traffic arrives, or you want to drop a copy you no longer need. Every gateway with near caching on exposes the same management routes over HTTP on its own port (on the Aeron gateway, its health server port):

ActionHTTP gateway routeWhat it does
ListGET /api/v1/near/cachesThe caches this gateway holds warm, with local item counts
PrimePOST /api/v1/near/cache/{id}Start the internal subscription and request the snapshot, so the copy is warm before any reader arrives
StopDELETE /api/v1/near/cache/{id}Drop the subscription and the local copy

Counter caches have GET /api/v1/near/counters-caches and /api/v1/near/counters/{id}. The same routes exist on the WebSocket, SSE and Aeron gateways at their own addresses.

Both mutating calls are idempotent. Priming an already-warm cache reports wasAlreadyWarm=true and does not re-subscribe, which avoids a duplicate subscription. Stopping a cache that is not warm succeeds with wasCached=false. Because the snapshot populates the copy asynchronously, a successful prime means priming was started: the cache appears on the list immediately, and its item count catches up shortly after.

You do not have to call these by hand. The web console has a Near Cache page that lists, primes and stops caches per gateway for both caches and counter caches, and ac-cli has prime-near-cache, stop-near-cache and list-near-caches (plus counter equivalents) for the HTTP gateway. The demo videos show both.

Synthetic patches: deltas without a patch write

Patch-native caching normally starts with a client sending a patch. But a lot of writers will never do that: they put a whole document each time, and the changed field is one in fifty. Synthetic patch generation lets patch-mode subscribers receive a small delta anyway, by computing it from the old and new values. It happens in two places, and a subscriber cannot tell which produced the delta.

In the clustered service

When a plain put overwrites an existing value and a patch-mode subscriber is listening for that cache and key, the cluster service computes the RFC 7386 merge patch between the stored and the new value, and sends it as a PATCH_ITEM. If nobody is listening, it does not compute one. The gate is a cheap check before any diffing starts:

boolean producePatch = patchSubscriptionService != null && mergePatchOut != null
        && patchSubscriptionService.hasSubscriber(cacheId, key);

Computing a patch can fail, for example when the stored or incoming value is not parseable JSON. Rather than send nothing, which would leave the patch subscriber holding stale state, the service falls back to sending the whole value as a full ADD_ITEM. A first write has nothing to diff, and a write that changes nothing needs no patch event.

At the edge, from the near cache

When a patch-mode (?mode=patch) hydrating subscriber arrives at a warm near cache, the gateway serves it and computes the deltas itself. The subscriber gets a full ADD_ITEM baseline on hydration (and for any key it has not seen before), and from then on the gateway diffs each incoming full value against the one it already holds and forwards only the difference as a PATCH_ITEM.

The internal near-cache subscription stays in full mode, because it needs whole values to diff against. No per-subscriber patch subscription is held on the cluster, so the delta work is offloaded from the core to the edge. This runs on the WebSocket, SSE, bidirectional and Aeron gateways, and on the Aeron gateway the delta is computed on the egress thread, only when a patch subscriber is present, so the no-patch path stays allocation-free. Patch subscriptions to a cold cache take the plain cluster path, where the first mechanism applies instead.

A worked example

The cache holds {"bid":1.0871,"ask":1.0873} for EURUSD, and a writer does a plain put of {"bid":1.0875,"ask":1.0873}:

  • A full-mode subscriber receives ADD_ITEM with the whole new value.
  • A patch-mode subscriber receives PATCH_ITEM with {"bid":1.0875}. ask did not change, so it never crosses the wire.
  • If the writer repeats the same value, the delta is empty, so the gateway drops it and the subscriber is not woken.

An embedded EmbeddedObjectCache merges that PATCH_ITEM into the value it holds using the same RFC 7386 rules, so it reconstructs the full object with no extra code. Both the compute and apply directions of the merge live in one self-contained class, so the near-cache layer does not depend on the cluster’s write-side patch provider. See JSON Merge Patch for the merge rules themselves.

Patches are only generated for JSON string values. Scalar counters are not diffed.

Consistency and cost

Be clear-eyed about the trade:

  • Warm reads are eventually consistent. A read of an already-warm key returns the local copy, which can briefly lag a concurrent write until its update event arrives. Leave near caching off where you need strongly consistent reads, or read through a gateway that has it off.
  • Memory per replica. Each replica holds its own copy of the caches it has warmed, so adding replicas multiplies read and hydration capacity and also the RAM. This is the usual edge-cache trade: memory for read throughput.
  • Warming is whole-cache. A warm copy holds every key, not just the ones you touch. That is the point, and also the cost for a very large cache you only touch lightly. Use stop to drop a copy you do not want to keep.

Seeing it work

Each synthetic-patch decision is counted by outcome:

OutcomeMeaning
deltaA merge patch was computed and forwarded as PATCH_ITEM
baselineA first-seen key with nothing to diff, sent as a full ADD_ITEM
suppressedThe delta was empty, a no-op write, and was dropped
full_fallbackA valid patch could not be built, so the whole value was sent

The delta to suppressed split shows how much wake-up and bandwidth local patching saves. A patch-subscribers gauge shows how many client sessions hold a patch-mode subscription.

These are Prometheus metrics, and how they are exported depends on the gateway. The HTTP, WebSocket and SSE gateways are ordinary web servers, so they serve aeroncache_near_patches_total{outcome} from their own /prometheus endpoint using Micrometer. The Aeron gateway records aeron_gateway_near_patches_total{outcome} as an Aeron media driver counter, which a sidecar converts to Prometheus text. The architecture page walks through both paths.

Takeaway

Near caching is one switch on each gateway that moves reads and subscriber hydration to the edge, with prime, list and stop admin to manage what is warm. Synthetic patches mean subscribers can receive deltas even when writers only know put, computed in the cluster for plain overwrites and at the edge from the warm copy. Neither asks anything of your clients, and both trade a little consistency and some memory for load taken off the core. See the API reference for the routes and the configuration guide for the settings.