Agentic — envelope capacity matrix
The full envelope-capacity matrix — every tier, every scale, not just the collapsed grid. See the narrative version at Benchmarks → Agentic; this is the complete record.
Full data
Capacity + latency of an agent-memory deployment across hardware envelopes: does each deployment fit a given tier, and how fast does it serve? Corpus A = LongMemEval (bge-large, 1024d; buckets = real conversational sessions). Harness benchmarks/agentic/run_envelope_full.sh + cell_watchdog.py.
Status: complete — 256/256 cells. AWS m6a.xlarge (x86-64-v3 / AVX2), data on the real SSD (/mnt/ssd, bind-mounted), real swap (4 GB swapfile, verified with swapon — an earlier pass ran silently swapless; every number below is from the corrected, swap-true run). xyzDB image xyzdb:0.9.7-x86v3 (budget-governed ingest — the engine bounds its own build against --memory-budget-mb); rivals at their latest images. One engine at a time, per-cell fresh container + wiped data dir, BUILD_TIMEOUT=1200s, hard cell wall 1400s, live fall-detection with reasons (OOM-kill / crashed / thrash).
2.1 Setup
| Tier | CPUs | RAM | Swap | --memory-swap (total) | xyz --memory-budget-mb |
|---|---|---|---|---|---|
| T1 | 1 | 256M | 256M | 512M | 256 |
| T2 | 2 | 512M | 512M | 1,024M | 512 |
| T3 | 2 | 2G | 512M | 2,560M | 2,048 |
| T4 | 2 | 4G | 512M | 4,608M | 4,096 |
Docker --memory-swap is the total (DRAM + swap), so the effective swap is --memory-swap − DRAM: T1 = 256M, T2/T3/T4 = 512M. Host-backed by a real 4G swapfile (EC2 has none by default). The lean store in the 2-container deployments is fixed at 1c / 256M DRAM + 256M swap.
Deployments: xyz (1 container: vector + structured), pg (1: pgvector does both), qdrant+pg / chroma+pg(2: vector engine at the tier envelope + a fixed lean Postgres store, 1c/256M, on :5433). Scales 30k / 100k / 246k (full corpus). RAM peak = combined across the deployment's containers, with the per-container split. Every engine runs its own idiomatic best form.
2.2 Scenarios
Six agent-memory workloads, each the same business question posed to every engine in its idiomatic shape. Corpus A = LongMemEval, bge-large 1024d; the retrieval bucket is question_id, a session is sid.
S1 retrieve-and-expand — “the k nearest memories, then expand each to its full session (the conversation, not a lone turn)”
e2e latency, session_recall (fraction of the oracle top-k found — xyz exact = ceiling, HNSW rivals < 1.0), expand_complete_frac (correctness: exactly the hit sessions' turns, no drop/bleed), roundtrips, disk (payload-dup tax).
S2 live session — “an agent that saves one memory and searches 2–3× per turn (sustained ~30% write / 70% read, deterministic replay of corpus turns)”
online insert latency, query p50/p99 under concurrent writes, degradation (late vs early window — where a maintenance cycle shows: LSM compaction / optimizer), visibility (a write at t is a candidate at t+1). Durable-strict, equalised across engines.
S3 multi-agent fleet — “N agents = N tenants (buckets); the fleet is born, grows, and dies (PURGE), each engine in its best tenant form”
cost to create/destroy the Nth tenant (not the first), RAM vs #tenants curve, filtered-query recall at each step; steps 10/100/1000 (if an engine can't hold 1000 in the tier, that OOM is the result). Run once per tier.
S4 serverless wake — “build the index → restart the container cleanly → TTFQ = restart → first successful query”
cold-start readiness: xyz serves from the LSM (little to warm); rivals load the graph/collections into RAM on first access. Composes with at-rest footprint.
S5 hybrid search — “"of the memories matching topic < T, give the k nearest" — exact structured filter + NEAREST, selectivity swept 50% → 0.1%”
recall@k vs the f64 oracle over the filtered set (tie-aware); deterministic metadata → all engines filter the identical universe (the parity gate).
S6 composite turn — “one engine for the whole agent: write vector+fields → update structured → NEAREST, then the AGGREGATE (count topic X active + avg importance)”
per-op latency, the +store inconsistency window (0 for one-system xyz/pg; non-zero for the 2-container stacks), the aggregate, summed footprint. Cells here are deployments, not binaries.
2.3 Capacity map (S1 — building and serving the base index)
| Deployment | T1:30k | T1:100k | T1:246k | T2:30k | T2:100k | T2:246k | T3:30k | T3:100k | T3:246k | T4:30k | T4:100k | T4:246k |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| xyz | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT |
| pg | FIT | BT | XX | FIT | BT | XX | FIT | FIT | XX | FIT | FIT | XX |
| qdrant+pg | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT | FIT |
| chroma+pg | FIT | TH | TH | FIT | FIT* | XX | FIT | FIT | FIT | FIT | FIT | FIT |
BT = build did not complete in 20 min · XX = crash/OOM during load · TH = OOM-thrash (pinned + no progress, watchdog-verdicted) · FIT* = completes but degraded (see per-tier tables).
2.4 Headline
- xyz is the only single-container deployment that fits everywhere — and its latency is flat across envelopes. At the full corpus (246k), S1 p50 is 15.0 / 15.9 / 19.3 / 17.8 ms across T1→T4 and the build takes 206–220 s whether the box has 256 MB or 4 GB. The envelope changes what xyz uses (RAM peak 251 MB at T1 → 1,539 MB at T4, elastic), not how it behaves. Recall 1.0 in every cell (exact NEAREST, never approximate).
- qdrant+pg also fits everywhere and posts the best roomy-tier scan p50 (4.4–7.9 ms, resident HNSW) — but it pays where an agent feels it: TTFQ is 1.2–8.4 seconds in every cell (xyz: 22–434 ms), the tight tier degrades it (26.9/52.2 ms at T1/246k, worse than xyz), live-session queries drop to 18–29 ms p50 at 246k (xyz 4.5–14), and it is two systems.
- pg never loads the full corpus— 246k OOMs during the HNSW build at every tier, including 4G+swap; 100k builds only at ≥2G (T1/T2 exceed the 20-min budget). Where it fits it is fast and lean on point queries (S2 q50 1.3 ms), and its T1/30k “fit” is heavily degraded: the build pages for 1,043 s vs 95 s on a roomy tier (11×).
- chroma+pg is the slowest at every size (S1 p50 36 → 290 ms as N grows; S5 up to 552 ms; S2 q50 265 ms at 246k) with brutal tails (S1 p99 4.0 s at T3/246k), needs ≥2G for the full corpus, and thrashes at T1 beyond 30k.
- Fleet (S3): creating the 1000th tenant costs xyz/qdrant ~150–190 ms; pg ~1.0 s (DDL per tenant); chroma never reached 1000 tenants at any tier.
- The lean second DB works: the +pg store stays at 23–57 MB in every cell — the stack tax is operational (two systems, double write, TTFQ), not RAM.
2.5 Per-tier tables
Latency in ms. S1 load s = corpus build wall-clock (the fits-degraded signal). S6 near/agg = composite-turn vector read / structured aggregate.
T1 — vector 1c/256M + 256M swap · store 1c/256M + 256M swap
| Deploy | N | S1 p50/p99 | S1 load s | Recall | S2 q50 | S4 TTFQ ms | S5 p50 | S6 near/agg | RAM peak (split) | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| xyz | 30k | 16.2/25.9 | 23.3 | 1.00 | 1.1 | 224.9 | 15.3 | 1.4/1.77 | 155 (xyzdb 155) | FIT |
| xyz | 100k | 17.3/24.8 | 83.8 | 1.00 | 1.4 | 230.8 | 17.1 | 5.3/0.30 | 217 (xyzdb 217) | FIT |
| xyz | 246k | 15.0/26.4 | 220.4 | 1.00 | 14.0 | 433.6 | 17.4 | 5.8/0.36 | 251 (xyzdb 251) | FIT |
| pg | 30k | 14.6/27.9 | 1,043.2 | 1.00 | 1.3 | 277.6 | 13.8 | 1.5/15.67 | 256 (pgvector 256) | FIT |
| pg | 100k | -/- | - | - | - | - | - | -/- | 256 (pgvector 256) | build-timeout |
| pg | 246k | -/- | - | - | - | - | - | -/- | 105 (pgvector 105) | OOM-load |
| qdrant+pg | 30k | 8.9/36.4 | 23.7 | 1.00 | 2.5 | 1,638.9 | 8.0 | 3.0/0.76 | 290 (qdrant 253, store 37) | FIT |
| qdrant+pg | 100k | 12.0/31.8 | 78.0 | 1.00 | 2.8 | 3,083.5 | 24.7 | 13.5/0.85 | 244 (qdrant 221, store 25) | FIT |
| qdrant+pg | 246k | 26.9/52.2 | 192.3 | 1.00 | 29.3 | 8,106.6 | 25.9 | 26.4/1.06 | 274 (qdrant 248, store 26) | FIT |
| chroma+pg | 30k | 42.1/512.0 | 98.1 | 1.00 | 4.8 | 912.9 | 49.3 | 29.4/0.94 | 256 (chroma 230, store 26) | FIT |
| chroma+pg | 100k | -/- | - | - | - | - | - | -/- | 282 (chroma 256, store 26) | thrash |
| chroma+pg | 246k | -/- | - | - | - | - | - | -/- | 282 (chroma 256, store 26) | thrash |
T2 — vector 2c/512M + 512M swap · store 1c/256M + 256M swap
| Deploy | N | S1 p50/p99 | S1 load s | Recall | S2 q50 | S4 TTFQ ms | S5 p50 | S6 near/agg | RAM peak (split) | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| xyz | 30k | 16.1/24.2 | 22.9 | 1.00 | 1.0 | 223.3 | 14.5 | 1.4/0.33 | 223 (xyzdb 223) | FIT |
| xyz | 100k | 18.7/31.7 | 80.8 | 1.00 | 1.3 | 227.9 | 17.1 | 4.6/0.28 | 282 (xyzdb 282) | FIT |
| xyz | 246k | 15.9/25.1 | 212.3 | 1.00 | 12.4 | 231.1 | 17.2 | 5.1/0.30 | 325 (xyzdb 325) | FIT |
| pg | 30k | 15.2/27.5 | 102.4 | 1.00 | 1.3 | 277.3 | 13.8 | 1.4/7.70 | 327 (pgvector 327) | FIT |
| pg | 100k | -/- | - | - | - | - | - | -/- | 512 (pgvector 512) | build-timeout |
| pg | 246k | -/- | - | - | - | - | - | -/- | 179 (pgvector 179) | OOM-load |
| qdrant+pg | 30k | 4.4/7.6 | 26.8 | 1.00 | 2.4 | 2,763.2 | 6.9 | 5.2/1.33 | 214 (qdrant 191, store 23) | FIT |
| qdrant+pg | 100k | 7.4/13.3 | 81.5 | 1.00 | 2.6 | 1,910.9 | 7.0 | 5.2/0.77 | 405 (qdrant 380, store 25) | FIT |
| qdrant+pg | 246k | 7.9/19.1 | 196.9 | 1.00 | 18.1 | 7,179.9 | 8.3 | 10.6/1.29 | 423 (qdrant 398, store 26) | FIT |
| chroma+pg | 30k | 36.0/104.4 | 29.0 | 1.00 | 4.9 | 385.9 | 47.5 | 29.3/0.94 | 271 (chroma 248, store 23) | FIT |
| chroma+pg | 100k | 117.7/2,126.4 | 529.3 | 1.00 | 13.4 | 3,002.1 | 193.0 | 63.6/0.96 | 538 (chroma 512, store 26) | FIT |
| chroma+pg | 246k | -/- | - | - | - | - | - | -/- | 538 (chroma 512, store 26) | OOM-load |
T3 — vector 2c/2G + 512M swap · store 1c/256M + 256M swap
| Deploy | N | S1 p50/p99 | S1 load s | Recall | S2 q50 | S4 TTFQ ms | S5 p50 | S6 near/agg | RAM peak (split) | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| xyz | 30k | 11.2/16.4 | 23.0 | 1.00 | 1.0 | 22.1 | 14.6 | 1.4/0.29 | 340 (xyzdb 340) | FIT |
| xyz | 100k | 16.9/26.2 | 79.2 | 1.00 | 1.3 | 23.6 | 15.1 | 5.0/0.37 | 762 (xyzdb 762) | FIT |
| xyz | 246k | 19.3/29.5 | 209.1 | 1.00 | 5.2 | 418.5 | 17.9 | 5.0/0.35 | 882 (xyzdb 882) | FIT |
| pg | 30k | 5.9/9.2 | 94.4 | 1.00 | 1.3 | 220.3 | 4.4 | 1.4/0.67 | 462 (pgvector 462) | FIT |
| pg | 100k | 15.0/21.5 | 375.1 | 1.00 | 1.4 | 252.1 | - | -/- | 1,153 (pgvector 1,153) | FIT |
| pg | 246k | -/- | - | - | - | - | - | -/- | 618 (pgvector 618) | OOM-load |
| qdrant+pg | 30k | 4.3/7.7 | 26.9 | 1.00 | 2.4 | 2,540.1 | 2.9 | 4.4/1.34 | 219 (qdrant 196, store 23) | FIT |
| qdrant+pg | 100k | 4.4/7.9 | 104.5 | 1.00 | 3.3 | 6,708.0 | 2.7 | 3.6/1.48 | 493 (qdrant 468, store 26) | FIT |
| qdrant+pg | 246k | 7.4/13.4 | 269.8 | 1.00 | 6.3 | 1,183.8 | 2.9 | 3.7/1.67 | 1,305 (qdrant 1,280, store 26) | FIT |
| chroma+pg | 30k | 36.5/104.6 | 25.4 | 1.00 | 4.9 | 392.2 | 48.0 | 29.8/0.93 | 255 (chroma 233, store 23) | FIT |
| chroma+pg | 100k | 114.3/322.5 | 109.4 | 1.00 | 12.5 | 657.6 | 191.1 | 60.0/0.99 | 585 (chroma 560, store 26) | FIT |
| chroma+pg | 246k | 289.3/3,987.4 | 388.5 | 1.00 | 263.4 | 6,115.7 | 544.8 | 148.5/1.11 | 1,296 (chroma 1,271, store 26) | FIT |
T4 — vector 2c/4G + 512M swap · store 1c/256M + 256M swap
| Deploy | N | S1 p50/p99 | S1 load s | Recall | S2 q50 | S4 TTFQ ms | S5 p50 | S6 near/agg | RAM peak (split) | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| xyz | 30k | 15.7/24.4 | 23.0 | 1.00 | 1.0 | 21.6 | 14.8 | 1.4/0.31 | 342 (xyzdb 342) | FIT |
| xyz | 100k | 16.7/25.9 | 80.1 | 1.00 | 1.3 | 226.0 | 15.1 | 4.9/0.30 | 1,403 (xyzdb 1,403) | FIT |
| xyz | 246k | 17.8/24.7 | 206.4 | 1.00 | 4.5 | 412.2 | 16.0 | 5.0/0.38 | 1,539 (xyzdb 1,539) | FIT |
| pg | 30k | 5.9/9.1 | 95.4 | 1.00 | 1.3 | 220.2 | 4.3 | 1.4/0.69 | 472 (pgvector 472) | FIT |
| pg | 100k | 6.3/9.3 | 375.7 | 1.00 | 1.4 | 220.9 | - | -/- | 1,701 (pgvector 1,701) | FIT |
| pg | 246k | -/- | - | - | - | - | - | -/- | 1,164 (pgvector 1,164) | OOM-load |
| qdrant+pg | 30k | 4.4/10.1 | 27.6 | 1.00 | 2.4 | 2,768.8 | 2.8 | 5.5/1.45 | 228 (qdrant 205, store 23) | FIT |
| qdrant+pg | 100k | 4.7/11.1 | 105.3 | 1.00 | 5.5 | 8,387.2 | 2.9 | 3.3/2.79 | 494 (qdrant 469, store 26) | FIT |
| qdrant+pg | 246k | 4.6/10.0 | 277.4 | 1.00 | 5.8 | 2,320.2 | 2.8 | 3.8/1.59 | 634 (qdrant 609, store 26) | FIT |
| chroma+pg | 30k | 36.8/109.8 | 25.9 | 1.00 | 4.9 | 389.9 | 48.0 | 29.8/0.93 | 242 (chroma 219, store 23) | FIT |
| chroma+pg | 100k | 115.4/322.5 | 109.6 | 1.00 | 12.6 | 661.3 | 192.3 | 60.4/0.98 | 586 (chroma 561, store 26) | FIT |
| chroma+pg | 246k | 289.9/1,316.1 | 311.1 | 1.00 | 265.5 | 1,942.3 | 551.9 | 143.0/1.10 | 1,296 (chroma 1,270, store 26) | FIT |
2.6 Fleet (S3) — 10/100/1000 tenants, once per tier
| Deploy | T1 | T2 | T3 | T4 |
|---|---|---|---|---|
| xyz | 177.2 / 199 | 187.9 / 271 | 184.9 / 817 | 190.1 / 1,467 |
| pg | 1,036.8 / 129 | 1,016.5 / 203 | 1,011.6 / 648 | 1,012.6 / 1,213 |
| qdrant+pg | 158.8 / 123 | 156.2 / 311 | 286.7 / 258 | 161.6 / 214 |
| chroma+pg | no t1000 | no t1000 | no t1000 | no t1000 |
create-nth ms / RAM MiB at 1000 tenants. chroma+pg's fleet loop never completed the 1000-tenant step inside the cell wall at any tier (it reaches ~100).
2.7 Notes
- Fits vs fits-degraded. A FIT verdict says the cell completed; the degradation shows in
S1 load sand the latencies. Clear degraded fits: pg T1/30k (build 1,043 s vs 95 s roomy), chroma T2/100k (build 529 s, S1 p99 2.1 s). xyz shows no degraded fit anywhere (load 23/80/210 s at 30k/100k/246k, constant across tiers). - pg S6 cells at 100k/246k appear as “fall (base build)”: the composite scenario rebuilds the base index first, which is exactly what fails at those scales — same root as its S1 verdicts, not an extra failure mode.
- Why xyz fits T1 now: engine 0.9.7 derives a global memtable ceiling from
--memory-budget-mband stalls ingest for background flush at the ceiling — a tight container bounds its own build instead of OOM-ing (0.9.6 died building 246k at T1). - Verdicts are watchdog-attributed, with reasons (e.g. chroma T1:
mem pinned 256/256MiB and cpu <20% sustained 300s -> paging, no progress). A container must stay down >90 s to be declared fallen (the settle restart comes back; a real OOM does not). - Mac/OrbStack runs of this matrix are direction only; this AWS dataset is the publishable one.