Confluent Hub Client

Confluent Hub is an online repository for extensions and components for Kafka. Kafka is based on extensible model for many of its services. It allows plug-ins and extensions which makes it generic enough to be suitable for many real world streaming based applications. They include both Confluent and 3rd party components. Generally, you would go there to search for components including:

• Connectors
• SMT (Single Message Transforms)
• Converters

Since you wouldn’t be changing the serialization a lot, you might find yourself search connectors and SMTs on this repository most of the time.

Confluent Hub obviously have a web page where you can go and search for the component you might be looking for.

Confluent Hub Web

Confluent Hub CLI

There is also a CLI available with the latest versions of Confluent Enterprise. You can find it in bin folder of the installation.

Confluent Hub CLI

But if you have a community edition, the CLI can also be downloaded as a tar ball. Just make sure that your confluent installation and confluent hub packages are added to PATH environment variable.

PATH

Here is the command we can use to download the components. Please note that there are switches to specify the component’s download directory. You can also note that there is an additional switch to specify the worker’s properties file. This is to ensure that the plugins details are updated so connector service can use the component from the specified directory.

The complete list of command’s switches can be found here:

Command reference

And the component does get downloaded to the specified directory.

Installed component

You can see that the worker’s properties file specified has also been updated with the details where the component has been dowloaded and can be used from:

Possible Configs Issue

Although the confluent tutorials suggest to run the installation command directly providing just the component’s details but there are other switches too, some of them seem mandatory.

Configs issue

Posting components to Confluent Hub

You can certainly post your connectors and other components to Confluent Hub and make it available to community once approved. Further details about such postings can be found here:

Contributions to Confluent Hub

Of course, you need to build the component before you can upload it. There are some specifications for building such components. The details can be found here:

Component Archive