KV Sync Service

ELI5

Whenever someone creates, edits, or deletes a link in the dashboard, the backend does not just save it to the database — it also sends a copy to Cloudflare’s global cache. This is like updating the menu at a restaurant: you change the recipe book in the kitchen (database), but you also reprint the menus on every table (edge cache) so customers see the latest options. For big events where hundreds of items change at once, the kitchen sends menus in bundles rather than one at a time.

Technical Deep Dive

Sync Triggers

The KV Sync Service runs asynchronously after these operations:

Async Pattern

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#e8f4f8', 'primaryTextColor': '#2d3748', 'primaryBorderColor': '#90cdf4', 'lineColor': '#718096', 'secondaryColor': '#f0fff4', 'tertiaryColor': '#fefcbf'}}}%%
flowchart LR
    A[LinkService mutation] --> B[UPDATE PostgreSQL]
    B --> C[Commit transaction]
    C --> D[Spawn goroutine]
    D --> E[KV Sync Service]
    E --> F[Call Cloudflare KV REST API]
    F --> G{Success ?}
    G -->|Yes| H[Log success]
    G -->|No| I[Log error<br/>No retry]

Why async?

• Database transaction must commit before KV write (avoid cache poisoning on rollback)
• KV API latency (50–200 ms) should not block the HTTP response to the user
• Failure to sync KV is non-fatal — the edge worker falls back to purr-api on cache miss

Cloudflare KV REST API

PUT /accounts/{account_id}/storage/kv/namespaces/{namespace_id}/values/{key}
Authorization: Bearer {CF_API_TOKEN}
Content-Type: application/json

{
  "url": "https://example.com",
  "userId": "user_abc123",
  "isActive": true,
  "expiresAt": null,
  "utmSource": "twitter",
  "utmMedium": "social",
  "utmCampaign": "launch2025",
  "redirectMode": "redirect",
  "protectionType": "none"
}

Response: 200 OK on success, 4xx/5xx on failure.

KV Data Structure

type KVLinkData struct {
    URL              string    `json:"url"`
    UserID           string    `json:"userId"`
    IsActive         bool      `json:"isActive"`
    ExpiresAt        *time.Time `json:"expiresAt,omitempty"`
    UTMSource        string    `json:"utmSource,omitempty"`
    UTMMedium        string    `json:"utmMedium,omitempty"`
    UTMCampaign      string    `json:"utmCampaign,omitempty"`
    UTMTerm          string    `json:"utmTerm,omitempty"`
    UTMContent       string    `json:"utmContent,omitempty"`
    RedirectMode     string    `json:"redirectMode"`
    ProtectionType   string    `json:"protectionType"`
}

Bulk Sync

For operations affecting many links (e.g., bulk import, tier change enabling all links):

const maxBatchSize = 10000  // Cloudflare KV limit

func (s *KVSyncService) BulkSync(ctx context.Context, links []Link) error {
    for i := 0; i < len(links); i += maxBatchSize {
        batch := links[i:min(i+maxBatchSize, len(links))]
        if err := s.writeBatch(ctx, batch); err != nil {
            return err
        }
    }
    return nil
}

Cloudflare KV bulk writes accept up to 10,000 key-value pairs per request. The service batches accordingly and processes sequentially to avoid rate limiting.

Error Handling

KV sync failures are logged but not retried automatically:

• Reason: KV is a best-effort cache. Stale data is acceptable for short periods.
• Monitoring: Log aggregation should alert on sustained sync failures.
• Recovery: The next cache miss triggers an API fallback, which writes the correct data back to KV.

Race Conditions

Consider this sequence:

1. User updates link URL from A → B
2. KV sync writes B
3. User updates link URL from B → C
4. KV sync for C is delayed
5. Edge worker reads KV and sees B (stale)

Mitigation: KV values do not have versioning. The system accepts eventual consistency. For critical updates (e.g., malicious URL replacement), the custom slug can be changed, which forces a new KV key and invalidates the old one.

Key Terms

• Goroutine → Lightweight Go thread; used to run KV sync without blocking the HTTP response
• Write-through cache → Updating the cache synchronously or asynchronously when the database is updated
• Bulk write → Sending multiple key-value pairs in a single API request for efficiency
• Cache poisoning → Writing stale or incorrect data to the cache before the database transaction commits
• Best-effort → Operation that improves performance but is not strictly required for correctness

Q&A

Q: Why not use Cloudflare Durable Objects instead of KV? A: Durable Objects provide strong consistency but are more expensive and complex. KV’s eventual consistency is acceptable for link shorteners, where a 60-second propagation delay is imperceptible to most users.

Q: How long does a KV write take to propagate globally? A: Typically 1–10 seconds, but Cloudflare documents up to 60 seconds for full global consistency. In practice, most edge locations see updates within seconds.

Q: What happens if the KV namespace is full? A: Cloudflare KV has generous limits (billions of keys, 25 MB per value). Hitting limits would require architectural changes (e.g., sharding by prefix). This is not a concern at SMO1’s current scale.

Q: Can KV sync be disabled for development? A: Yes. If CF_API_TOKEN or CF_ACCOUNT_ID is not configured, the KV Sync Service logs a warning and skips the write. Local development works without Cloudflare credentials.

Examples

Think of KV sync like updating flight information displays at an airport:

• Link creation is a new flight being added to the schedule — the control tower (database) records it, and every display board (KV) is updated
• Link update is a gate change — passengers at the old gate (stale KV) might be confused for a minute, but the system corrects itself quickly
• Bulk sync is a weather cancellation affecting 50 flights — the control tower sends one big update to all displays rather than 50 individual messages
• Async sync is the control tower telling an intern to update the boards — the tower does not wait for the intern to finish before answering the next radio call