How to clear all messages in Kafka 0.10.1.0 topic without deleting the topic?

Sometime, the messages in Kafka topic would be overwhelming and we need a quick way to clear these messages without deleting the topic.

Note deleting the topic is an option that should be used with caution in Production.

Kafka topics by default have a concept of retention, i.e how long a message in a topic needs to be persisted. By default this setting is 7 days. We are going to make use of this to clear the messages in the topic.

We are going to set the retention to 1 second and then bring it back to 7 days. Login to the Kafka directory and go to the folder location where Kafka is installed.


cd /usr/local/kafka/kafka_2.11-0.10.1.0/bin

./kafka-topics.sh --zookeeper zookeeper1:2181 --alter --topic yourtopicname --config retention.ms=1000

Wait for some time so that the messages are cleared. Check if there are any messages by reading from the topic from the beginning

 bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic yourtopicname --from-beginning 

Make sure that you reset the timing to the original 7 days

Important Please do not forget to do this step, else all your messages in Kafka would be deleted after 1 second delay.

cd /usr/local/kafka/kafka_2.11-0.10.1.0/bin

./kafka-topics.sh --zookeeper zookeeper1:2181 --alter --topic yourtopicname --config retention.ms=604800000