Running fastcached as a compile cache¶

Serving fastcache-cc needs no special mode: the compile-cache protocol is detected from the first byte, so any listener serves it.

Check the value cap¶

The per-value cap defaults to 256 MiB, chosen for exactly this workload, so most deployments need no flag here. It matters because a value above the cap is rejected outright — that translation unit then never caches, which looks like a poor hit rate rather than an error.

The rejection is at least diagnosable: the daemon answers with a typed payload-too-large error naming the declared size and the cap, so it appears in fastcache-cc --show-stats under fall-back reasons and in the daemon log, rather than as a dropped connection.

Object files in a large C++ codebase routinely exceed 16 MiB; one measured codebase peaked at ~122 MB for a single object. If your largest object goes past 256 MiB, raise it:

fastcached --storage-max-value=512M

--storage-max-value also raises the wire frame-payload cap, so a single flag covers both limits — for the compile-cache and Redis protocols. The two memcached framings keep a fixed 16 MiB ceiling that this flag does not move.

To find your largest object:

find build -name '*.o' -printf '%s %p\n' | sort -rn | head -5

Persist the cache¶

Without --storage the cache is memory-only and evaporates on restart, which for a compile cache usually means throwing away hours of compiles.

fastcached --storage=/var/lib/fastcached/cache.cow \
           --max-memory=8g

--max-memory sizes the in-memory L1 tier; the on-disk L2 grows as the workload needs. Reads consult L1 first and fall through to disk on a miss, so a warm working set is served from RAM while the long tail stays durable.

Cap the disk footprint with --storage-max-disk; the tree then evicts its LRU tail to fit rather than growing without bound.

Scale it¶

For a shared cache serving many builders concurrently:

fastcached --bind=0.0.0.0 \
           --storage=/var/lib/fastcached/cache \
           --storage-shards=16 \
           --threads=16 \
           --requirepass=<secret> \
           --metrics --metrics-bind=0.0.0.0

--storage-shards=N (N>1) makes --storage a directory of shard-NN.cow files, so writes to different shards never block each other.
--threads=N runs N pinned single-threaded reactors.
Binding 0.0.0.0 exposes the cache to the network — always pair it with --requirepass. See Deployment.

Compression¶

The on-disk tier compresses values by default (--compression=zstd). Object files compress well, so this is usually worth keeping; reads always return plaintext because each record decodes by its own tag. Lower --compression-level (default 3) if CPU is the constraint rather than disk.

The in-memory L1 tier has a codec of its own, off by default: --memory-compression, --memory-compression-level and --memory-compression-min-bytes. Turning it on makes --max-memory hold more rather than less — the budget counts the bytes a value actually occupies — at a decompress on every read.

lz4 and zstd are only present when the build was configured with FASTCACHED_ENABLE_COMPRESSION. A codec you name on a build without them is refused at startup; the disk half's zstd default is not, since nobody typed it — that tier stores plaintext instead, and a warning says so.

Both codecs are on the startup banner, and both name what the store will actually do rather than what was asked for:

... storage=/var/lib/fastcached/store.cow durability=batched compression=zstd memory-compression=none max-value=256M ...

Without --storage there is no on-disk tier for a codec to describe, so that field reads <no disk tier> — distinct from none, which is the name of a codec.

Prefetch groups¶

Prefetching is automatic and needs no configuration. When a FETCH hits a key that belongs to a group, the rest of that group is warmed into memory in the background, so the compiles that follow in the same build are served from RAM.

Clients control the grouping with FASTCACHE_PREFETCH_GROUP. Since it is not part of the cache key, regrouping never invalidates anything — set it per project and branch (myproject-main) for the tightest prefetch locality.

Monitoring¶

--metrics serves Prometheus metrics on /metrics and a liveness probe on /healthz on a dedicated port (default 9259). For a compile cache the useful signals are the hit ratio and the eviction rate: sustained evictions with a falling hit ratio mean the working set no longer fits and --max-memory (or --storage-max-disk) needs raising.

The client side reports independently — fastcache-cc --show-stats gives the hit rate as each builder sees it, which is what actually determines build times.

fastcache-cli live-stats pointed at the 0xFC port draws the same reading live: the daemon streams its cache subject, the counters and snapshot /metrics renders, pushed on a fixed tick rather than scraped (#1399). The node and fleet subjects belong to a compile node, and the daemon refuses them by name. A stream is held to the connection's credential while it runs, not only when it opens: a connection that never authenticated loses its stream on the tick --requirepass takes effect, and one that did keeps it when the secret rotates. The live counters are the node's rows, described under Live stats.

If you run a compile fleet rather than a lone cache, the leader can show you the whole of it on one page: see Looking at the whole fleet. That is a view of who is a member, how big each machine is and what its cache holds — the thing /metrics cannot give you without a Prometheus in front of it, because a scrape is one process reporting itself. /metrics stays the source of truth for anything you alert on; the page is for the question you ask once, while something is wrong.

A note on client configuration¶

If hit rates are low, check the client before tuning the server. Cache keys incorporate the compiler identity and the relativized command line, so mismatched compiler versions or differing flags between CI and developer machines produce different keys by design. fastcache-cc --show-stats distinguishes a genuine miss from a cache that was never reached.