KAFKA REST Proxy – Publishing Avro Messages to Kafka Apache Kafka supports message publishing through REST too. We can use Kafka REST proxy for this purpose. It supports http verbs including GET, POST and DELETE. Her is an example POST using curl which we will be trying to dissect throughout this post: Required Schema Apache… Continue reading »
Building Serde for Kafka Streams Application While developing Kafka Streams applications, I found myself requiring some utility code over and over. One such code is to build Serde for custom types. It is better if it is refactored into separate types and used when needed. It has two methods specificAvroSerde and genericAvroSerde. It must be… Continue reading »
avro-tools in Scala Projects – Don’t… In this post we are going to discuss how Avro-tools dependency can mess up your logging in a Scala project. Let’s create a sample Scala SBT project. We update the build.sbt as follows: Here are we adding dependencies for logback in addition to avro-tools dependency. Let’s create a simple… Continue reading »
Using Avro Schema in JVM based applications Like Protobuf, Apache Avro is specially suitable for organizations with polyglot development stack. Data serialized through one language can easily be deserialized in other language on the other end of your messaging platform. This is one of the supported serialization for Confluent platform. The benefits of Avro over… Continue reading »
Overriding Default Serialization for Kafka Connect – Be Careful Serialization / Deserialization for Confluent platform can be sometimes tricky. There are certain defaults to the platform. As developers, it is our responsibility to understand the defaults and tweak the parts which are necessary. Even just a little mismatch can be sometimes very cumbersome and might… Continue reading »