Cache Stats & Hit Ratio Health

ELI5

The cache is a fridge with a “best by” date on each bag: Prune empties anything older than N days, the door display shows what fraction of opens found what they wanted (hit ratio), and the fridge politely refuses new bags once it hits its size limit.

Technical Deep Dive

Cache health is computed client-side from a single /api/pellets/stats response:

{ hitRate, utilizationPercent, status }

Mapping (from useCacheHealth):

isHealthy flips at hitRate >= 0.5. utilizationPercent = totalSize / maxCacheMB where maxCacheMB defaults to 1024 (free tier).

Eviction & Pruning

flowchart TD
    A[POST /api/pellets/prune] --> B{body.all?}
    B -->|true| C[remove every pellet]
    B -->|false| D{body.days?}
    D -->|n| E[cutoff = now - n*24h]
    E --> F[for each meta.json]
    F --> G{CreatedAt < cutoff?}
    G -->|yes| H[Remove .pellet + .meta.json]
    G -->|no| I[skip]
    C --> J[respond removed:int]
    H --> J

There is no LRU bookkeeping on the engine side — eviction is purely time-based via Store.Prune(days). Tiered limits (Free/Pro/Enterprise, README) are enforced at the dashboard level by formatting against maxCacheMB; the Go store itself does not refuse writes when full in this revision.

Hit Ratio Source

Engine increments nestr_cache_hits_total / nestr_cache_misses_total (counters) and republishes their ratio as nestr_cache_hit_ratio (gauge). Web polls /api/pellets/stats every 10 s for the same numbers in JSON form.

Key Terms

• Hit ratio → cache_hits / (cache_hits + cache_misses), exposed as both a counter pair and a gauge.
• Utilization → fraction of the configured tier limit currently held in .pellet files.
• Cutoff → time.Now().Add(-days * 24h); any pellet with CreatedAt before this is reaped by Prune.

Q&A

Q: What happens if I call prune with neither days nor all? A: The handler treats missing days as 0 and the cutoff equals “now”; everything older than now is removed — effectively a full wipe by accident. Always send all:true for that intent.

Q: Does the Engine block compression when over the tier limit? A: Not in this revision — limits are advisory. Free/Pro/Enterprise gating is presentational; the maxCacheMB value comes from the client config.

Q: Why does the badge go ‘good’ before utilization peaks? A: The two are independent. A lightly-used cache with frequent reuse hits ‘excellent’ early; a stuffed cache with cold pellets stays ‘poor’ regardless of fill.

Examples

Stats sample: {totalSize: 734003200, pelletCount: 42, hitCount: 380, missCount: 120, hitRate: 0.76, originalSize: 4400000000, compressedSize: 734003200, compressionRatio: 0.167} → status “good”, utilization 70 % of a 1024 MB free tier.