agentsclimarketplace

Kafka debounce event coalescing

Skill kjuhwa/skills-hub/skills/backend/kafka-debounce-event-coalescing

Self-correcting knowledge corpus for Claude Code — 9 stable shape clusters, bias-correction pipeline baked into contribution flow. 47 papers, 45 techniques, 1.1k skills.

Install
npx -y skills add kjuhwa/skills-hub --skill kafka-debounce-event-coalescing

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Coalesce bursts of change events into one Kafka event per tenant per debounce window using Redis state (first/last change time + changed-tenant SET), with a MAX_WAIT fallback so a never-quiet stream still fires.

SKILL.md

2.1 KB, as published. Nobody here has run it

Kafka Debounce Event Coalescing (Redis-backed)

Shape

  • Debouncer service keeps per-tenant timing state in Redis: first_change_time, last_change_time, SET changed_tenants.
  • Publisher fires one Kafka event per tenant when either (a) the debounce window elapses with no new event, or (b) the MAX_WAIT ceiling since the first change is reached.
  • Consumer acquires a per-tenant Redis lock; concurrent events set a recheck flag, forcing a second pass rather than being dropped.

Steps

  1. On each upstream change, capture tenantId. SET NX first_change_time=now; always update last_change_time=now.
  2. SADD changed_tenants tenantId.
  3. If now - first_change_time >= MAX_WAIT (e.g. 5 min) → fire immediately. Else schedule debounce timer (e.g. 1 min idle).
  4. On timer, SMEMBERS changed_tenants and publish one Kafka event per tenant (not a bulk event — keeps downstream sharding simple).
  5. Clear first_change_time, last_change_time, changed_tenants.
  6. Consumer: per-tenant Redis lock with TTL; if lock fails, SET recheck:<tenant> 1 and return.
  7. Lock holder loop: DEL recheck:<tenant> → run logic → if recheck re-set during logic, loop again.

Counter / Caveats

  • TTLs must exceed worst-case processing time or lock is stolen mid-run.
  • MAX_WAIT guards a continuously-noisy stream from starving downstream updates forever.
  • One-event-per-tenant shape only helps when downstream is tenant-keyed.
  • Re-check flag must be cleared before running logic, not after, or you'll miss events arriving during logic.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.