Confluent Components & Their HTTP Ports It is very important to remember the ports of individual components of Confluent Platform. The following is the list of default ports for the respective components. It must be remembered that these ports can be overridden during deployment. More details can be found on this page on confluent.io where… Continue reading »
Kafka and Confluent logs in Log File During development and debugging, it is very useful to see traces for Kafka and confluent in log file to determine the issues. The traces are specially useful when we use DSL for Kafka Streams. Here is the logback configuration to use logback appenders for kafka and confluent logs.
Log4J Dependency with Kafka Many of the Kafka artifacts have Log4J dependencies. If you are using logback then your logging stops all of a sudden. Here is an example for one of our sample app. Here we have added such dependency and the App seems to have issues with multiple SL4J bindings. It has finally… Continue reading »
avro-tools in Scala Projects – Don’t… In this post we are going to discuss how Avro-tools dependency can mess up your logging in a Scala project. Let’s create a sample Scala SBT project. We update the build.sbt as follows: Here are we adding dependencies for logback in addition to avro-tools dependency. Let’s create a simple… Continue reading »
Kafka Message Size Issue – Record Too Long to Send As soon as the message size increases, Kafka starts throwing errors when sending the message. Here are the error message we received when sending a message with size a few mege bytes. The message clearly suggests that the size needs to updated in some configuration… Continue reading »
Using Avro Schema in JVM based applications Like Protobuf, Apache Avro is specially suitable for organizations with polyglot development stack. Data serialized through one language can easily be deserialized in other language on the other end of your messaging platform. This is one of the supported serialization for Confluent platform. The benefits of Avro over… Continue reading »
Overriding Default Serialization for Kafka Connect – Be Careful Serialization / Deserialization for Confluent platform can be sometimes tricky. There are certain defaults to the platform. As developers, it is our responsibility to understand the defaults and tweak the parts which are necessary. Even just a little mismatch can be sometimes very cumbersome and might… Continue reading »
CORS with Kafka REST Proxy Landoop’s Kafka Topics UI is an excellent tools to view the topics in a Kafka Cluster in Confluent platform. It is based on AngularJS. If you just follow the instruction and run it using http_server, you would get the following error: The issue is with the default CORS settings on… Continue reading »
Create In-Memory file & Downloading Data In this post we are going to add ability to create an in-memory file and add ability to download it using a link and a button. Here is an HTML file. Here we are adding a Link with download attribute set as Download2.csv. This would download the file as… Continue reading »
ng-content with Angular2 (Transclusion) – AngularCli In this post we are going to look at a special feature in Angular2. The feature allows to create templates such that parent components can inject parts into the view of the child component. The child components use the injected part using ng-content. When should we use ng-content? ng-content… Continue reading »