Request batching and caching

Request Batching and Caching

Instructions

A well-designed caching strategy lets an app feel instant even on 3G. Batching keeps the radio from waking up for every tiny write. This skill covers both.

1. Cache Layers

┌────────────────────────────────────────────┐
│          View / UI                         │
└───────────────┬────────────────────────────┘
                │
┌───────────────▼────────────────────────────┐
│ In-memory cache (per process, short TTL)   │
│ e.g., MemoryCache, NSCache, LRU            │
└───────────────┬────────────────────────────┘
                │ miss
┌───────────────▼────────────────────────────┐
│ Disk cache (per install, minutes-hours TTL)│
│ OkHttp Cache, URLCache, Drift, Realm       │
└───────────────┬────────────────────────────┘
                │ miss / stale
┌───────────────▼────────────────────────────┐
│ CDN edge (seconds-minutes TTL)             │
└───────────────┬────────────────────────────┘
                │ miss
┌───────────────▼────────────────────────────┐
│ Origin                                     │
└────────────────────────────────────────────┘

Every layer should honor Cache-Control and ETag. Build the client so it asks one layer, falls through on miss, and never bypasses layers silently.

2. HTTP Caching Primitives

Server sends:

Cache-Control: public, max-age=60, stale-while-revalidate=600, stale-if-error=86400
ETag: "abc123"
Last-Modified: Sat, 19 Apr 2026 11:20:01 GMT

Client behavior to enforce:

1. Serve from cache if not expired.
2. If expired but within stale-while-revalidate, serve cached and fire a background revalidation.
3. On revalidation, send If-None-Match: "abc123"; accept 304 Not Modified.
4. If origin is down, serve stale under stale-if-error.

3. Android — OkHttp Cache

val cache = Cache(directory = File(ctx.cacheDir, "http"), maxSize = 20L * 1024 * 1024)
val client = OkHttpClient.Builder()
    .cache(cache)
    .addNetworkInterceptor { chain ->
        val resp = chain.proceed(chain.request())
        resp.newBuilder()
            .removeHeader("Pragma")
            .header("Cache-Control", resp.header("Cache-Control") ?: "public, max-age=60")
            .build()
    }.build()

For offline reads:

val request = Request.Builder()
    .url(url)
    .cacheControl(CacheControl.Builder().maxStale(1, TimeUnit.DAYS).build())
    .build()

4. iOS — URLCache + URLSession

let cache = URLCache(memoryCapacity: 16 * 1024 * 1024,
                     diskCapacity: 128 * 1024 * 1024,
                     directory: nil)
let config = URLSessionConfiguration.default
config.urlCache = cache
config.requestCachePolicy = .useProtocolCachePolicy
let session = URLSession(configuration: config)

Use .returnCacheDataElseLoad for an explicit stale-first fetch when rendering a cached screen.

5. Flutter — dio_cache_interceptor

final cacheOptions = CacheOptions(
  store: HiveCacheStore(cachePath),
  policy: CachePolicy.forceCache,
  hitCacheOnErrorExcept: [401, 403],
  maxStale: const Duration(days: 7),
  priority: CachePriority.normal,
);
dio.interceptors.add(DioCacheInterceptor(options: cacheOptions));

6. React Native / TypeScript

Use a fetch wrapper or library (react-query, SWR) that already implements stale-while-revalidate:

const { data } = useQuery({
  queryKey: ['feed', userId],
  queryFn: () => api.getFeed(userId),
  staleTime: 60_000,          // fresh for 60 s
  gcTime: 5 * 60_000,
  refetchOnWindowFocus: true, // refresh when returning to foreground
});

7. Batching Writes

Frequent small POSTs from background drain the radio. Collect and batch.

Kotlin pattern — buffer + periodic flush:

class EventBatcher(private val api: Api, scope: CoroutineScope) {
    private val queue = MutableSharedFlow<Event>(extraBufferCapacity = 1_000)

    init {
        queue
            .chunked(50, 10.seconds)    // 50 events or 10 s, whichever first
            .onEach { batch -> runCatching { api.send(batch) } }
            .launchIn(scope)
    }

    fun track(event: Event) { queue.tryEmit(event) }
}

Swift pattern:

actor EventBatcher {
    private var buffer: [Event] = []
    private var flushTask: Task<Void, Never>?

    func track(_ e: Event) {
        buffer.append(e)
        if buffer.count >= 50 { flushNow() } else { scheduleFlush() }
    }

    private func scheduleFlush() {
        flushTask?.cancel()
        flushTask = Task { [weak self] in
            try? await Task.sleep(for: .seconds(10))
            await self?.flushNow()
        }
    }

    private func flushNow() async {
        let batch = buffer; buffer.removeAll()
        guard !batch.isEmpty else { return }
        try? await api.send(batch)
    }
}

8. Request Coalescing / De-duplication

If multiple callers ask for the same resource simultaneously, de-dup into one in-flight request.

class RequestCoalescer<K, V>(private val load: suspend (K) -> V) {
    private val inFlight = mutableMapOf<K, Deferred<V>>()
    private val mutex = Mutex()

    suspend fun get(key: K): V = mutex.withLock {
        inFlight.getOrPut(key) {
            CoroutineScope(coroutineContext).async {
                try { load(key) } finally { mutex.withLock { inFlight.remove(key) } }
            }
        }
    }.await()
}

react-query and SWR do this automatically by key.

9. Cache Invalidation

• Always keyed by normalized URL + variant (Accept-Encoding, device class).
• On mutation, invalidate the exact queries instead of nuking everything. In react-query: queryClient.invalidateQueries({ queryKey: ['feed'] }).
• Persist the most-recently-viewed N resources across process death; evict older entries by LRU.

Checklist

• HTTP client has a disk cache (OkHttp Cache, URLCache, dio_cache).
• Server responses include Cache-Control, ETag, and stale-while-revalidate.
• Stale-while-revalidate actually triggers a background refresh on the client.
• Events / writes batched to ≤ 1 request per 10 s in steady state.
• Duplicate in-flight requests de-duped by key.
• Cache invalidation is targeted, not blanket.
• Offline reads of critical screens are possible from cache.