Short definition
We define Cache Debt as the hidden cost created when AI systems keep stale context, bloated prompts, outdated embeddings, old model cache, unused artifacts, and forgotten agent state.
Longer definition
Cache Debt builds when AI memory and generated material accumulate faster than teams review, refresh, or remove them. Most tools keep state by default. Hugging Face keeps every downloaded model revision,[5] a routine Docker prune leaves volumes behind,[7] and Claude Code writes every tool result to a plaintext transcript.[15] Each item looks harmless alone. Together they cost disk, slow builds, degrade retrieval, and hold data nobody meant to keep.
In plain English
AI systems remember things. Teams often forget to maintain what those systems remember. The growing gap between retained state and useful state is Cache Debt.
Examples
- Embeddings that still represent deleted or revised source material.
- Old prompt versions that remain active in production workflows.
- Agent logs and tool results with no retention or deletion policy.
- Local model and package caches left behind by AI-assisted builds.
- Saved conversation context carried into later sessions. Longer inputs lower model performance.[2]
Warning signs
- No one can explain which retained state a workflow still needs.
- Prompt behavior differs between environments without a clear version history.
- Vector search returns material that is no longer authoritative.
- Agent workspaces and cache directories grow without review.
- Debugging begins with clearing state because its provenance is unknown.
Prior art
The idea is not new. In 2015 Sculley et al. showed machine learning systems carry hidden maintenance costs, naming undeclared consumers, unstable data dependencies, legacy features, and dead experimental codepaths.[1] Cache Debt narrows that lens to the state modern AI tools keep on disk and in context:
| Sculley et al., 2015 | Cache Debt today |
|---|---|
| Unstable data dependencies | Embeddings whose source documents changed |
| Undeclared consumers | Prompt versions and model files nobody can map to a workflow |
| Dead experimental codepaths | Model weights and caches from finished experiments |
| Configuration debt | Agent memory and saved instructions nobody reviews |
Two research results bound the quality claim. Model performance drops as input length grows,[2] and models use the middle of a long context less reliably than its start or end.[3] Neither study measures stale content directly. They show extra context has a cost, not proof every old instruction causes harm.
Related terms
- Technical debt
- Data drift
- Context rot
- Configuration drift
- Data retention
- Prompt versioning
- Artifact lifecycle management
Cache Debt overlaps with technical debt, data drift, context rot, configuration drift, data retention, prompt versioning, and artifact lifecycle management. It focuses specifically on the cost of AI state that persists beyond its useful or trustworthy life.
A cleanup approach
Start with measurement, not deletion. Run the audit script, then rank what it finds with the Cache Debt Score. Size is a poor guide: in our own worked example, 39 MB of agent transcripts outranked 920 MB of unused browser builds because transcripts can hold credentials.
NextRun the cleanup checklist