Monthly Archives: August 2018

Kafka Streams – Resetting Application State

Kafka Streams – Resetting Application State

I the previous post, we discussed that we might need to reprocess data during development during application development. Since Kafka Streams ensures the application state, it doesn’t pull and reprocess data. In this case, you might find yourself keep waiting for the join operations to get triggered, but to your disappointment, you might not see the breakpoints being hit.

There are two alternate ways which can be used during application development and debugging ensuring the reprocessing of data.

Here is the streams configuration for our Kafka Streams application:

Use New Application Id

One possible solution is to use the new application state each time you need to re-process in your development environment. You can just use a random number for the configuration. It is very common to destroy the whole confluent environment using confluent destroy command. So if it looks non-manageable anytime, you can always destroy it. I wouldn’t recommend this as there is an alternate solution available which is also very easy to use.

Application Reset Tool

Alternatively you can use Kafka Application reset tool. Here we are resetting the state of application with myApplicationId id. After executing the command, the offset should be set to the beginning of input topics. It would also delete all intermediate topics.

https://docs.confluent.io/current/streams/developer-guide/app-reset-tool.html