Changelog compaction tombstone retention
Skill kjuhwa/skills-hub/skills/backend/changelog-compaction-tombstone-retention
Self-correcting knowledge corpus for Claude Code — 9 stable shape clusters, bias-correction pipeline baked into contribution flow. 47 papers, 45 techniques, 1.1k skills.
npx -y skills add kjuhwa/skills-hub --skill changelog-compaction-tombstone-retentionAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
State-backend changelog topics need tombstone retention longer than compaction interval to survive consumer rebalance gaps.
SKILL.md
1.5 KB, as published. Nobody here has run it
changelog-compaction-tombstone-retention
When backing operator state with a compacted Kafka topic (changelog pattern), a delete is a null-value tombstone. Log compaction eventually removes tombstones after delete.retention.ms, but if a consumer/operator rebalances after the tombstone is compacted away, it will replay the old pre-delete value and never see the delete — resurrecting zombie state. The pattern: set delete.retention.ms to at least max(sessionTimeout, checkpointInterval) * 3 and never rely on default (24h) if your checkpoint cycle is longer.
# topic config for state changelog
cleanup.policy=compact
min.cleanable.dirty.ratio=0.1
delete.retention.ms=604800000 # 7d, >> checkpoint interval
segment.ms=3600000
The corollary: when you do want to prune state (e.g., TTL'd keys), emit the tombstone and then force-roll a segment before the operator can be considered safe to rebalance. Also, never compact on a topic where you replay from earliest as part of bootstrap — compaction can remove intermediate values your bootstrap logic depends on for audit reconstruction. Use a separate non-compacted audit topic for that, and keep the compacted topic as materialized current-state only.