Configuration guide

How configuration works

Aeron Cache is configured with environment variables. Everything else is a delivery mechanism for them:

The same option has the same name everywhere. cacheMode in a Helm chart becomes CACHE_MODE in the pod, and is the CACHE_MODE line in backend.env. So once you know an option from one place, you know it in all of them.

If you have not installed anything yet, start with Getting started.

Pick a deployment mode

The biggest decision is which shape you deploy. It sets CACHE_MODE and decides which other options matter.

All three expose the same command surface and the same APIs, so moving from one to another does not change your client code.

Homebrew (all-in-one)

brew tap bhf/aeroncache
brew install aeron-cache
aeron-cache

On first run a default config is written to ~/.aeron-cache/backend.env. Edit it and restart aeron-cache. Lines are plain NAME=value.

Choosing interfaces

The REST API and cluster tools are always on. The streaming gateways are switchable:

A disabled gateway is not started at all, so the matching UI streaming features will not connect.

For example, to enable the Aeron gateway over IPC and turn SSE off:

AERON_GATEWAY_ENABLED=true
GATEWAY_TRANSPORT_MEDIA=ipc
SSE_GATEWAY_ENABLED=false

Near cache

An optional read-ahead cache inside the HTTP, WebSocket (including the bidirectional socket), SSE and Aeron gateway interfaces. When on, the interface subscribes to the cluster, keeps a warm local copy of the caches it touches, and serves reads from it. It is transparent to clients (same URIs, no client change); see the API page for what is served from it and the management endpoints.

Because it is read-ahead, a read of a warm key can briefly lag a concurrent write. Leave it off if you need strongly consistent reads.

Other all-in-one options

JVM options and heap size

To pass JVM flags such as a larger heap, set JAVA_TOOL_OPTIONS in backend.env:

JAVA_TOOL_OPTIONS=-Xmx4g

Several flags go in one value, separated by spaces, for example JAVA_TOOL_OPTIONS=-Xms4g -Xmx4g. Restart aeron-cache after editing.

• Why this works: the launcher exports every line of backend.env as an environment variable before starting Java, and the JVM reads JAVA_TOOL_OPTIONS by itself.
• Not JAVA_OPTS: the launcher sets that variable itself, so a value in backend.env is overwritten.
• Not command-line arguments: aeron-cache <args> passes them to the application, not to the JVM.
• Default: with no flag, the JVM sizes the heap at a quarter of physical RAM.
• Check it applied: the JVM logs Picked up JAVA_TOOL_OPTIONS: ... at the top of ~/.aeron-cache/aeron-cache.log.
• Shell variable: a JAVA_TOOL_OPTIONS in your shell is replaced by the value in backend.env.
• Native memory: Aeron’s term buffers live outside the heap, so a larger -Xmx does not change them. See term length.

Core options (all deployments)

These are shared by the cluster, interface apps and all-in-one install.

Term length and maximum value size

The maximum Aeron message size is termLength / 8. The charts set 128m so values up to roughly 16 MB work. Leave it blank to fall back to the application default of 64k, which suits small values only. The buffers are sparse, so the resident memory cost of a larger term length is small.

Idle strategy: latency versus CPU

The idle strategy decides what a thread does when it has no work. It is the main latency-versus-CPU dial.

IDLE_STRATEGY applies to every role. To tune one role, set IDLE_STRATEGY_<ROLE>, which takes priority over the global value. For example the HTTP app reads IDLE_STRATEGY_BLOCKING_PUBLISHER, IDLE_STRATEGY_CLUSTER_CLIENT_AGENT, IDLE_STRATEGY_CLUSTER_MESSAGE_PUBLISHER, IDLE_STRATEGY_AGENT_RUNNER and IDLE_STRATEGY_UNCLUSTERED. An invalid value fails fast at startup rather than when the first agent is created.

Snapshots and log purging

A RAFT cluster appends every command to a log. Snapshots let a node restart quickly, and purging discards log history older than the retained snapshots. The on-demand snapshot-and-purge endpoint always works. The options below control the scheduled run.

Choose the check strategy by what you want to avoid:

Scheduling only applies to clustered (RAFT) runs where CLUSTER_FOLDER is set. In SINGLE mode the options are inert. Durations use ISO-8601 form, for example PT30M for 30 minutes and PT6H for 6 hours.

Kubernetes and Helm

The charts live in k8s/helm/ in the main repository. Each component is its own chart, and the Makefile there wires them together.

git clone https://github.com/bhf/aeron-cache
cd aeron-cache/
make all                  # build images into Minikube
cd k8s/helm/
make install-all          # cluster + interfaces + UI

Which chart do I need?

Near cache in the charts

Near caching is a switch on the interface charts, not a separate chart (the old aeroncache-http-near chart and image have been removed).

Each sets NEAR_CACHE_ENABLED on the container. For example: helm upgrade --install aeroncache-ws ./aeroncache-ws --set nearCacheEnabled=true. Each replica holds its own copy of the caches it has warmed, so it costs memory per replica.

Install targets

All targets run from k8s/helm/, and every install-* has a matching uninstall-*.

make install-backend              # cluster + HTTP, WS, SSE, gateway
make install-frontend             # UI only
make install-all                  # backend + UI
make install-backend-ephemeral    # single-node backend
make install-all-ephemeral        # single-node backend + UI
make install-monolith-all         # all-in-one: node + gateways + UI
make uninstall-all                # tear down

Individual components work too, for example make install-http or make install-ui.

Make flags

Flags go on the end of any install target.

make install-all GHCR=true GHCR_TAG=1.2.3
make install-backend-ephemeral GHCR=true EXTERNAL_MEDIA_DRIVER=true

Common values

These appear in most charts. Defaults below are from the chart values.yaml files.

Cluster chart specifics

The cluster needs /dev/shm for Aeron’s shared memory. The chart mounts a 1 GiB in-memory emptyDir there by default. Raise sizeLimit if you raise term lengths or run many streams.

All-in-one chart specifics

External media driver

By default each node embeds a Java media driver. Setting externalMediaDriver.enabled=true runs the C aeronmd driver as a sidecar and attaches the node to it as a pure client (LAUNCH_EMBEDDED=false).

Build the driver image with make build-media-driver, which is part of make build-backend.

Observability

Deploy Prometheus and Grafana with make install-observability. It installs into the aeroncache namespace and ships a pre-provisioned “Aeron Cache - Interface Apps” dashboard covering JVM memory, CPU, GC and HTTP request rate and latency.

There are two sources of metrics:

• Clustertools sidecar. Cluster and ephemeral pods carry prometheus.io/* annotations, and the sidecar serves the shared media driver counters on /prometheus at port 7080. This works with no extra flag.
• OpenTelemetry Java agent. Optional and off by default. Pass JAVAAGENT=true to an interface-app install target, or set javaagent.enabled=true, and the app serves metrics on javaagent.metricsPort (default 9464) at javaagent.metricsPath (default /metrics). Traces and logs exporters default to none.

Sending telemetry to an external collector (OTLP)

The agent is the standard OpenTelemetry Java agent, so it reads the standard OTEL_* environment variables. The chart sets a few of them for you and passes anything in javaagent.extraEnv straight through, which is how you point it at your own collector.

To ship traces and logs to a collector while keeping the Prometheus metrics endpoint:

# values override for an interface app, e.g. aeroncache-http
javaagent:
  enabled: true
  serviceName: aeroncache-http
  tracesExporter: otlp
  logsExporter: otlp
  extraEnv:
    - name: OTEL_EXPORTER_OTLP_ENDPOINT
      value: http://otel-collector.observability.svc.cluster.local:4318
    - name: OTEL_EXPORTER_OTLP_PROTOCOL
      value: http/protobuf

Use port 4317 with grpc instead if your collector expects gRPC. Per-signal variables such as OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS (for authenticated backends) work the same way. If the endpoint is unreachable, the agent logs export failures, so leave tracesExporter and logsExporter at none until the collector is up.

The metrics exporter is fixed to prometheus by the chart. To push metrics over OTLP as well, add OTEL_METRICS_EXPORTER to extraEnv with the value otlp or prometheus,otlp. It is appended after the chart’s own entry, so it takes precedence, but check the rendered pod spec with helm template before relying on it.

This applies to the interface apps only. The clustertools sidecar serves Prometheus metrics on port 7080 and is not configured through these values. Scrape it, or have your collector scrape it with a Prometheus receiver.

Overriding values

Use standard Helm. Either edit the chart’s environments/values.dev.yaml, or pass overrides:

helm upgrade --install aeroncache-cluster ./aeroncache-cluster \
  --namespace aeroncache \
  --set idleStrategy=backoff \
  --set persistence.size=10Gi \
  --set clustertools.snapshot.scheduleEnabled=true \
  --set clustertools.snapshot.check.strategy=RECORDING_GROWTH

Recipes

Local, minimal

Install with Homebrew and leave everything at its defaults. You get REST, WebSocket and SSE, plus the UI.

Local, native Aeron access

# ~/.aeron-cache/backend.env
AERON_GATEWAY_ENABLED=true
GATEWAY_TRANSPORT_MEDIA=ipc

IPC works when the client runs on the same machine. Use udp when it does not.

Cluster with bounded disk

# values override for aeroncache-cluster
persistence:
  size: 20Gi
clustertools:
  snapshot:
    scheduleEnabled: true
    retentionCount: 2
    check:
      strategy: RECORDING_GROWTH
      archiveGrowthBytes: 536870912   # 512 MiB

Cluster tuned for latency

idleStrategy: "spin"        # a full core per thread, so give the pods dedicated CPU
externalMediaDriver:
  enabled: true
  threadingMode: DEDICATED
resources:
  requests: { cpu: "4", memory: 4Gi }
  limits:   { cpu: "4", memory: 4Gi }

Measure before and after. Spinning trades CPU for latency, and the right answer depends on your hardware and load. See Low-GC design for the design reasoning.

Large values

Keep aeronCacheTermLength: "128m" for values up to about 16 MB. Leaving it blank drops back to 64k, which is far smaller.

Troubleshooting

Where the source of truth lives

Defaults can change between releases. The main repository is authoritative: the config/*.env files for environment variables, and k8s/helm/<chart>/values.yaml for every Helm option, each annotated with comments.