Performance and Scalability
These are results from the Caracal v1.0.0 production validation: a primary bench on a production cloud VM, and an extended bench covering component costs, failure injection, startup, and storage behavior. Every figure below was observed on the bench described in Test Conditions; none is extrapolated or theoretical.
Treat these as a reference baseline for sizing and evaluation, not a service-level guarantee. Throughput and latency depend on your hardware, workload shape, and upstreams. Establish your own baseline with Scale Capacity before committing to a limit.
Test Conditions
Section titled “Test Conditions”| Attribute | Primary cloud bench | Extended bench |
|---|---|---|
| Deployment | Packaged Docker Compose via caracal up, single host, loopback-bound, stock configuration | Packaged Docker Compose, single host, loopback-bound |
| Host | Azure Standard_D4as_v5: 4 vCPU, 16 GiB RAM, StandardSSD, Ubuntu 24.04 | 16 vCPU, 60 GiB RAM, NVMe SSD, Linux (Fedora 43) |
| Load model | Closed-loop client, published TypeScript SDK, one in-flight governed request per worker, 16 long-lived application identities | Closed-loop client, published TypeScript SDK, one in-flight governed request per worker |
| Upstream | Mock HTTP service on the compose network (isolates Caracal overhead) | Local mock HTTP service (isolates Caracal overhead) |
| Credential path | Long-lived identities: verification served from the credential cache, the steady state for persistent agent fleets | Includes full Argon2id credential derivation |
A governed request performs one single-use, replay-protected mandate mint at the STS plus one Gateway hop. An application transport provisions its session and delegation once per credential lifetime (four control-plane calls), then mints per request.
Throughput and Scaling
Section titled “Throughput and Scaling”On the primary cloud bench, governed throughput scaled linearly with offered concurrency until the host CPU saturated, reaching 33 governed requests per second per host vCPU with credential verification served from cache:
| Concurrent workers | Sustained governed req/s | p50 | p95 |
|---|---|---|---|
| 1 | 8.2 (paced) | 20 ms | 30 ms |
| 4 | 36.5 | 24 ms | 33 ms |
| 8 | 63.2 | 22 ms | 33 ms |
| 16 | 138.2 | 34 ms | 71 ms |
An 8-minute sustained soak at 16 workers held 132.6 governed requests per second — 63,984 requests with zero transport or 5xx errors — for a daily capacity of 11.5 million governed calls on one 4-vCPU host. Peak measured throughput was 139 req/s.
Two ceilings apply together:
- Per resource: a single
(zone, resource, application)pair is capped at 1000 mints/minute (16.6 req/s) by default. The cap is a deployment setting: raiseSTS_MINT_RATE_LIMIT_PER_MINfor a higher ceiling, or set a lower working limit from the web console (Settings → Preferences → Mint rate limit). Spreading load across resources or applications also multiplies the budget; the soak above spread 16 identities. - Per host: CPU-bound, with the STS and Postgres as the first components to saturate. Add cores, or split the stores onto their own hosts, before tuning any Caracal setting.
Session lifecycle is independent of the mint path: on the extended bench the Coordinator sustained 192 session create-and-close cycles per second (p50 20 ms) before lock contention flattened throughput.
Latency
Section titled “Latency”Measured on the primary cloud bench. The governance decision itself is negligible; cost is concentrated in the cryptographic mint, and it degrades gracefully rather than collapsing as the host saturates.
| Operation | p50 | p95 | p99 |
|---|---|---|---|
| Governed request (warm, below saturation) | 17-24 ms | 23-33 ms | 34-43 ms |
| Governed request (at full host saturation) | 45 ms | 85 ms | 113 ms |
| Governed request (cold, first call incl. authority provisioning) | 287 ms | — | — |
| Policy (OPA) evaluation, mean over 128,000 evaluations under load | 1.04 ms | — | — |
| List applications / resources | 4 ms | 6 ms | — |
| Audit query (50 rows) | 6 ms | 11 ms | — |
Component costs measured on the extended bench: Gateway mandate verification 3.7 ms, delegation create 17.5 ms, provider create 8.1 ms p50.
The mint dominates because it verifies identity and traverses the delegation graph cryptographically. Policy evaluation and Gateway verification are sub-millisecond to low-single-digit milliseconds.
Resource Usage and Stability
Section titled “Resource Usage and Stability”Measured during the primary cloud bench’s 8-minute sustained soak at full host saturation: 132.6 req/s, 63,984 requests, zero hard errors.
| Signal | Observation |
|---|---|
| Host memory | 2 GiB used of 16 at full load; memory was never a limiting factor |
| Service restarts during soak | 0 |
| Postgres connections | 27-43 of the 100 ceiling; no connection leak |
| Audit events dropped | 0 — dead-letter queue empty at soak end, hash chain continuous |
| Policy evaluation errors | 0 across 128,000 evaluations |
On the extended bench, STS memory under sustained load showed a healthy garbage-collection sawtooth (floor 337 MB / peak 792 MB on a 2 GB limit) with no leak and no unbounded growth.
Failure and Recovery
Section titled “Failure and Recovery”On the extended bench, each dependency was failed in isolation while a continuous governed probe ran. Every failure was fail-closed — no request ever succeeded without valid authority — and every recovery was automatic, with no manual intervention.
| Injected failure | Live governed traffic | Recovery | Data loss |
|---|---|---|---|
| Redis unavailable (brief) | Requests block, then drain on return | Automatic, immediate | None |
| Postgres unavailable (brief) | Requests block, then drain on return | Automatic, immediate | None |
| STS process killed | Immediate fail-closed (connection refused) | Next request after restart; restart-to-healthy 0.6 s | None |
| Gateway process killed | Requests fail closed | Automatic; restart-to-healthy 0.6 s | None |
| Coordinator process killed | Continues unbroken (established transports mint against the STS) | No impact to live traffic | None |
| Audit process killed | Continues unbroken (asynchronous write path) | Events buffer and drain; hash chain verified continuous | None |
This separates the request path from the control and evidence paths:
- Required for live governed traffic: STS, Gateway, Postgres, Redis.
- Not required for live governed traffic: Coordinator (needed only to provision new sessions and delegations) and Audit (asynchronous). Their outages are invisible to established transports.
Brief database or cache interruptions shorter than the request timeout are absorbed as latency, not errors. For diagnosis and recovery order, see Recover from Failures.
Startup and Operational Timings
Section titled “Startup and Operational Timings”Measured on the extended bench.
| Operation | Time |
|---|---|
| Download, checksum-verify, and install the runtime | 9.0 s |
Cold start (caracal up: image pull, migrate, readiness gate) | 17.7 s |
Warm restart (caracal down then up) | 4.1 s + 3.7 s |
| Single service restart to healthy | 0.6 s |
caracal status | 37-87 ms |
| Web console response | 3.6 ms |
Storage and Capacity Growth
Section titled “Storage and Capacity Growth”Measured on the extended bench.
| Metric | Measured |
|---|---|
| Audit storage per event | 1,843 bytes (including indexes) |
| Audit events per governed call | ~1.4 |
| Audit query scaling | Flat with total volume (keyset pagination over monthly partitions) |
| Database footprint (end of run) | 176 MB for 49,498 audit events, 8,450 sessions, 186 applications, 186 providers, 33 resources |
| Postgres connections | 100 ceiling; 50 peak under full load; no lock contention |
| Audit retention | 365-day default ceiling (AUDIT_RETENTION_DAYS), console-adjustable below it; hourly Parquet export to S3-compatible storage for longer archival |
At a sustained governed rate, audit disk grows at approximately rate × 1.4 × 1843 bytes per second — for example, ~62 KB/s (~5.3 GB/day) at 24 req/s. Plan audit storage and retention for your rate.
Known Product Limits
Section titled “Known Product Limits”Defaults are deliberately conservative safety limits. The throughput figures above were reached by raising the capacity-shaping limits for the test workload.
| Limit | Default | Behavior when exceeded |
|---|---|---|
STS mint rate, per (zone, resource, app) | 1000 / minute (STS_MINT_RATE_LIMIT_PER_MIN); console working limit below it | Denied, fail-closed; recorded as rate_limited in audit |
| STS repeated authentication failures, per application | 60 / minute | Temporary block |
| Coordinator requests, per client IP | 600 / minute (COORDINATOR_RATE_LIMIT_PER_MIN) | 429 rate_limited |
| Live sessions per zone | 50 (MAX_AGENTS_PER_ZONE) | 429 session_zone_limit_exceeded |
| Live sessions per application | 200 (MAX_AGENTS_PER_APP) | 429 session_limit_exceeded |
| Control API calls, per client | 60 / minute | 429 |
To operate at the throughput and concurrency in this report, raise MAX_AGENTS_PER_ZONE to your concurrent-agent count and STS_MINT_RATE_LIMIT_PER_MIN to your target rate, then confirm the change with Scale Capacity. Configure limits and container resources in Configure Service Environment; the console-managed working limits live under Settings → Preferences.
Recommended Starting Profiles
Section titled “Recommended Starting Profiles”Sizing is driven by the STS (throughput and memory) and Postgres (durability). These profiles derive from the extended-bench measurements, which include full credential derivation on the load path; fleets of long-lived identities served from the credential cache reach substantially higher throughput on the same profile, as the primary-bench results above show. Size the supporting stores with headroom above the measured footprint and validate before production.
| Workload target | STS | Gateway | Postgres | Redis |
|---|---|---|---|---|
| Small — ≤ 10 req/s, ≤ 50 sessions | 1 vCPU / 1 GB | 1 vCPU / 512 MB | 2 vCPU / 2 GB | 0.5 vCPU / 512 MB |
| Standard — ~25 req/s, hundreds of sessions | 2 vCPU / 2 GB | 2 vCPU / 1 GB | 4 vCPU / 4 GB (SSD) | 1 vCPU / 768 MB |
| Large — ~40 req/s per instance | 4 vCPU / 4 GB | 2 vCPU / 2 GB | 8 vCPU / 8 GB (NVMe) | 1 vCPU / 1 GB |
| High throughput | STS replicas at 4 vCPU / 4 GB, sharded across resources | 4 vCPU / 2 GB | 8 vCPU / 16 GB + read replica | 2 vCPU / 2 GB |
The packaged deployment ships the STS at 2 vCPU / 1 GB, between the Small and Standard profiles. Move to the Standard profile’s 2 GB when sustaining ~25 req/s. Postgres used up to roughly one vCPU-equivalent and 461 MB with 50 connections at 24 req/s; provision it with headroom above that and prefer SSD or NVMe storage.
Production Recommendations
Section titled “Production Recommendations”- Size STS memory at or above the 1 GB packaged default. The credential cache removes Argon2id from the steady state and the verification budget bounds cold bursts; provision 2 GB when raising
STS_SECRET_VERIFY_CONCURRENCYor running many distinct applications. - Raise the per-zone session cap (
MAX_AGENTS_PER_ZONE) to your concurrent-agent count before load; the default is 50. - Raise
STS_MINT_RATE_LIMIT_PER_MINwhen a shared resource needs more than 16.6 req/s, or shard across resources or applications; then add STS replicas to scale beyond one instance. - Provision Postgres as the second bottleneck after STS CPU: SSD or NVMe storage, connection headroom below the 100 default, and roughly twice the STS vCPU.
- Alert on the STS memory limit, Postgres connections, and audit lag, and validate every limit change against your own workload.
Next Step
Section titled “Next Step”Turn these results into a sized deployment with Choose a Deployment Profile, then validate your own baseline in Scale Capacity.