Apache has provided an amazing documentation for Kafka. It also includes step-by-step guide to run download and run kafka. Unfortunately, the documentation misses any info for running it on windows platform. I really had to struggle running it on my windows box, so i thought i should add it here for anyone else going through the same issues.

Step I [Download Kafka Binaries]

First of all, we can download kafka from the following:

The download is a tar gzip compressed file with tgz extension. So first we need to un-compress it before using any files. There are a number of tools available for this. I have 7-zip installed on my machine, which also supports extracting contents from this type of compressed file. Here I have downloaded 0.8.11 with the scala version 2.10.

Just extract the contents of the archive into a folder.

Step II – Install support for running shell (*.sh) scripts

Now the second step of the kafka documentation suggests running Zookeeper directly. For a windows machine there is no built-in support for this, so you need to add tools for running the shell script. If you already have Git client installed, there is already a support, otherwise, install Cygwin making sure that you are selecting utils for installation.

Step III – Updating the compatible batch files

But if you just run it now, it just throws the following error message. Here I am using Cygwin terminal.

Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain

I have seen a lot of people have this issue. Apparently, it is because of the batch files provided with the download. It is in Windows sub-directory of kafka folder.

I am using 64-bit Windows 8.1 machine. The solution that worked for me is to replace the win batch files with the ones provided here.

Step – IV Handling spaces in JAVA_HOME Environment Variable

I have JRE installed on this windows machine. The environment variable, JAVA_HOME, is set to the installation directory which has spaces in its path.

Running with this path set would introduce another problem.

If you clearly look at the error message, it is apparent that, somehow, kafka-run-class.sh script is being executed, which is throwing this error on line 153. Let’s do the universal solution for handling spaces in file paths, enclose it in quotes.

Step V – Missing Server JVM components

Now If you just have JRE installed, as I had on this machine, you should see the error message for missing server JVM.

In order to just get over this problem, you can just create a new folder, and name it server. Now copy the contents of client folder to this new folder.

Step VI – Unrecognized VM Option UseCompressedOops

Now as we move forward and run it, we are faced with another error. Only this time, it doesn’t tell where is the actual problem. This is the error message.

Unrecognized VM option ‘UseCompressedOops’

Just removing this JVM option, would let you get rid of this error message.

Step VII – QuorumPeerMain – Once Again

After going through all the above steps, reality hits you and you end up with the same error message that you started with. And that makes it really funny. Has all this effort been fruitless. Actually, No!!!

Since we are running it in windows, we are having class path problem. Since we already have cygwin installed, we can use cygpath. Just update line 153 in kafka-run-class again and use cygpath with class path.

Finally!!!

Finally, after going through all this hassle, you can have zookeeper running on your machine. It looks so pretty 🙂

Now you can continue with the next instructions for running kafka server, creating topics, sending message from producer and picking them up from consumer.

Zindabad!!!