Deploying kafka k8s

Skill bg-szy/TOP-SKILLS/skills/marketplace/deploying-kafka-k8s

全球最大的 Claude Code 技能聚合库 · 收录 3900+ 来自 12+ 来源的技能，提供在线搜索与趋势分析看板 / The world's largest Claude Code skill aggregation hub — 3900+ skills from 12+ sources with online search and trend dashboard

Install

npx -y skills add bg-szy/TOP-SKILLS --skill deploying-kafka-k8s

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Deploys Apache Kafka on Kubernetes using the Strimzi operator with KRaft mode. Use when setting up Kafka for event-driven microservices, message queuing, or pub/sub patterns. Covers operator installation, cluster creation, topic management, and producer/consumer testing. NOT when using managed Kafka (Confluent Cloud, MSK) or local development without K8s.

SKILL.md

7.5 KB, as published. Nobody here has run it

Deploying Kafka on Kubernetes

Deploy production-ready Apache Kafka clusters using Strimzi operator (v0.49.1+) with KRaft mode.

Quick Start

# 1. Create namespace
kubectl create namespace kafka

# 2. Install Strimzi operator
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka

# 3. Wait for operator
kubectl wait deployment/strimzi-cluster-operator --for=condition=Available -n kafka --timeout=300s

# 4. Deploy Kafka cluster
kubectl apply -f https://strimzi.io/examples/latest/kafka/kraft/kafka-single-node.yaml -n kafka

# 5. Wait for ready
kubectl wait kafka/my-cluster --for=condition=Ready --timeout=300s -n kafka

Strimzi Operator Installation

Standard Install (Cluster-wide)

kubectl create namespace kafka
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka
kubectl get pods -n kafka -w

Namespace-scoped Install

# Download and modify for single namespace
curl -L https://strimzi.io/install/latest?namespace=kafka > strimzi-install.yaml
# Edit RoleBindings and ClusterRoles as needed
kubectl apply -f strimzi-install.yaml -n kafka

Kafka Cluster Configurations

Single Node (Development)

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: kafka
spec:
  kafka:
    version: 3.9.0
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

Production Cluster (3 Nodes + KRaft)

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-production
  namespace: kafka
spec:
  kafka:
    version: 3.9.0
    replicas: 3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: nodeport
        tls: false
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.9"
    storage:
      type: jbod
      volumes:
        - id: 0
          type: persistent-claim
          size: 100Gi
          deleteClaim: false
    resources:
      requests:
        memory: 2Gi
        cpu: "500m"
      limits:
        memory: 4Gi
        cpu: "2"
  entityOperator:
    topicOperator: {}
    userOperator: {}

Topic Management

Create Topic via CRD

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: task-events
  namespace: kafka
  labels:
    strimzi.io/cluster: my-cluster
spec:
  partitions: 3
  replicas: 1
  config:
    retention.ms: 604800000    # 7 days
    segment.bytes: 1073741824  # 1GB

List and Describe Topics

# List topics
kubectl -n kafka run kafka-topics -ti --rm --restart=Never \
  --image=quay.io/strimzi/kafka:0.49.1-kafka-3.9.0 -- \
  bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list

# Describe topic
kubectl -n kafka run kafka-topics -ti --rm --restart=Never \
  --image=quay.io/strimzi/kafka:0.49.1-kafka-3.9.0 -- \
  bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 \
  --describe --topic task-events

Producer/Consumer Testing

Console Producer

kubectl -n kafka run kafka-producer -ti --rm --restart=Never \
  --image=quay.io/strimzi/kafka:0.49.1-kafka-3.9.0 -- \
  bin/kafka-console-producer.sh \
  --bootstrap-server my-cluster-kafka-bootstrap:9092 \
  --topic my-topic

Console Consumer

kubectl -n kafka run kafka-consumer -ti --rm --restart=Never \
  --image=quay.io/strimzi/kafka:0.49.1-kafka-3.9.0 -- \
  bin/kafka-console-consumer.sh \
  --bootstrap-server my-cluster-kafka-bootstrap:9092 \
  --topic my-topic --from-beginning

Service Discovery

Kafka bootstrap services for client connections:

Service	Port	Use
`my-cluster-kafka-bootstrap:9092`	Plain	Internal cluster apps
`my-cluster-kafka-bootstrap:9093`	TLS	Secure internal apps
`my-cluster-kafka-0.my-cluster-kafka-brokers:9092`	Plain	Direct broker access

Connect from Another Namespace

# In your app deployment
env:
  - name: KAFKA_BOOTSTRAP_SERVERS
    value: "my-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092"

Monitoring

Enable Prometheus Metrics

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml

Check Cluster Status

kubectl get kafka -n kafka
kubectl describe kafka my-cluster -n kafka
kubectl get pods -n kafka -l strimzi.io/cluster=my-cluster

Troubleshooting

Operator Not Starting

kubectl logs deployment/strimzi-cluster-operator -n kafka
kubectl describe pod -l name=strimzi-cluster-operator -n kafka

Kafka Pods Not Ready

kubectl describe pod my-cluster-kafka-0 -n kafka
kubectl logs my-cluster-kafka-0 -n kafka
kubectl get events -n kafka --sort-by='.lastTimestamp'

Common Issues

Error	Cause	Fix
PVC pending	No storage class	Add `storageClassName` or use ephemeral
Pods OOMKilled	Insufficient memory	Increase resource limits
Connection refused	Wrong bootstrap URL	Use `cluster-kafka-bootstrap:9092`

Cleanup

# Delete cluster
kubectl -n kafka delete kafka my-cluster

# Delete PVCs (data)
kubectl delete pvc -l strimzi.io/name=my-cluster-kafka -n kafka

# Remove operator
kubectl -n kafka delete -f 'https://strimzi.io/install/latest?namespace=kafka'

# Delete namespace
kubectl delete namespace kafka

Integration with Dapr

For Dapr pub/sub integration, see configuring-dapr-pubsub skill:

# Dapr component pointing to Strimzi Kafka
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: kafka-pubsub
spec:
  type: pubsub.kafka
  metadata:
    - name: brokers
      value: "my-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092"
    - name: authType
      value: "none"

Verification

Run: python scripts/verify.py

Related Skills

operating-k8s-local - Local Minikube cluster setup
configuring-dapr-pubsub - Dapr Kafka pub/sub integration
scaffolding-fastapi-dapr - FastAPI services with Kafka events