CRUMB a card from devarno-cloud

KV Sync Service: Writes Nobody Reads

smo1 intermediate 6 min read

ELI5

The API was built to copy every link into Cloudflare’s global key-value store so the edge could answer without phoning home. It still makes those copies when it has Cloudflare credentials. But the edge looks them up under different names, expects a different layout, and in production is not even connected to the store. The copies sit there unread.

Technical Deep Dive

Source: purr-api internal/service/kv_sync_service.go:19-138, 140-226, 228-320, internal/service/link_service.go:183-193, 376-390, 416-421, cmd/server/main.go:193-209, internal/config/config.go:144-146 (origin/main bb6cfc2); zoomies-edge wrangler.toml:50-63, src/handlers/redirect.ts:26-42, 85-90, src/index.ts:236-241 (5bdde59), read 2026-10-25.

What it does

flowchart LR
    A[LinkService Create, Update, Delete] --> B[Postgres write]
    B --> C{"kvSyncService set ?"}
    C -->|No| X[nothing]
    C -->|Yes| D[goroutine]
    D --> E{"kvEligible: AuthorizeRedirect allowed and expiry over 120 s away ?"}
    E -->|Yes| F["PUT key = uid, and key = custom_slug"]
    E -->|No| G["DELETE uid and custom_slug keys"]
    F --> H[(Cloudflare KV namespace)]
    G --> H
    H -.->|"never read"| W[zoomies-edge]
  • Enabled only when CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_KV_NAMESPACE_ID and CLOUDFLARE_API_TOKEN are all set (kv_sync_service.go:52-67, main.go:193-209). Whether they are set on Railway production is UNVERIFIED.
  • Triggers: create (link_service.go:191-193), update, which also deletes a stale custom-slug key on rename (376-390), and delete (419-421). Each runs in a fire-and-forget goroutine with context.Background(); errors are dropped.
  • Eligibility reuses the redirect rule: protected, paused and expired links are deleted from KV instead of written, because the value has no protection field (kv_sync_service.go:73-86).
  • Expiry is passed as Cloudflare’s ?expiration= so KV drops the entry when the link expires.
  • Bulk: SyncAllLinks and BulkSync batch 10,000 entries per bulk write. Nothing calls them.

Why it is a no-op at the edge

purr-api writesWorker reads
Keyabc123 (bare uid), my-slug (bare custom slug)link:{host}:{slug} in handleRedirect, link:{slug} in fetchLinkData
Value{url, userId, isActive, expiresAt, utmSource…}{id, slug, destination, is_active, expires_at, protection_type, redirect_mode, …}
BindingREST API to a namespace idenv.LINKS_CACHE, commented out in [env.lion]

The key formats never meet, and a value read under the right key would still be missing destination, id and redirect_mode. wrangler.toml:50-55 states that enabling the binding as-is would be a regression.

  1. Pick one key format, host-scoped (link:{host}:{slug}), because custom slugs are unique only per user and domain.
  2. Make the value the edge’s LinkResponse shape, including id, redirect_mode and protection_type.
  3. Make the proxy probe in index.ts use the same host-scoped key.
  4. Decide whether protected links stay KV-ineligible (today they must, since the prompt needs the link id but never the destination).

Key Terms

  • kvEligible — AuthorizeRedirect == allowed and expiry at least 120 s away
  • putKV / deleteKV — single-key Cloudflare REST calls with the API token
  • bulkWrite — 10,000-entry Cloudflare bulk endpoint; uncalled
  • LinkResponse — the edge’s expected link shape, which KVLinkData does not match

Q&A

Q: What triggers a KV sync? A: Link create, link update (including removing the old custom-slug key on rename) and link delete in LinkService, each only when the KV sync service was enabled at boot. There is no scheduled refresh and no caller for the bulk paths.

Q: What is the Cloudflare KV batch size limit? A: The code batches 10,000 entries per bulk write (batchSize = 10000), Cloudflare’s bulk-write limit, but no code path calls the bulk sync.

Q: Why are KV syncs performed asynchronously? A: So a slow or failing Cloudflare API call never delays or fails the dashboard request; the Postgres write is the durable one. The cost is silent loss: errors from the goroutine are discarded.

Q: Why is the KV sync effectively a no-op at the edge, and what has to change before LINKS_CACHE is bound? A: purr-api writes bare-uid and bare-slug keys with a camelCase value; the Worker reads link:{host}:{slug} or link:{slug} expecting the snake_case LinkResponse, and in production has no KV binding at all. Key format, value shape and the proxy probe’s key must all be reconciled first.

Examples

An owner pauses a link. If KV credentials are set, a goroutine deletes keys abc123 and my-slug from the namespace. The Worker never looked there, so the pause takes effect at the edge immediately anyway, through the next /api/links/resolve call returning is_active: false.

neighbors on the map