avro-tools in Scala Projects – Don’t… In this post we are going to discuss how Avro-tools dependency can mess up your logging in a Scala project. Let’s create a sample Scala SBT project. We update the build.sbt as follows: Here are we adding dependencies for logback in addition to avro-tools dependency. Let’s create a simple… Continue reading »
Kafka Message Size Issue – Record Too Long to Send As soon as the message size increases, Kafka starts throwing errors when sending the message. Here are the error message we received when sending a message with size a few mege bytes. The message clearly suggests that the size needs to updated in some configuration… Continue reading »
Using Avro Schema in JVM based applications Like Protobuf, Apache Avro is specially suitable for organizations with polyglot development stack. Data serialized through one language can easily be deserialized in other language on the other end of your messaging platform. This is one of the supported serialization for Confluent platform. The benefits of Avro over… Continue reading »
Overriding Default Serialization for Kafka Connect – Be Careful Serialization / Deserialization for Confluent platform can be sometimes tricky. There are certain defaults to the platform. As developers, it is our responsibility to understand the defaults and tweak the parts which are necessary. Even just a little mismatch can be sometimes very cumbersome and might… Continue reading »
CORS with Kafka REST Proxy Landoop’s Kafka Topics UI is an excellent tools to view the topics in a Kafka Cluster in Confluent platform. It is based on AngularJS. If you just follow the instruction and run it using http_server, you would get the following error: The issue is with the default CORS settings on… Continue reading »
System Level Environment Variables in RHEL 7Environment variables are created using export statement. In order to create a system wide environment variable, the export must be executed when any user is logged-in. You can copy scripts that need to be executed when a user is logged-in in /etc/profile.d directory. Just create a file with extension… Continue reading »
Angular2 – Attribute Directives Directives give us the ability to create tags which can be directly used in the template. Since they are not HTML5 templates, we need to define how Angular can render these components In the end, they would be converted to a representation that a browser understands. Angular can convert these directives… Continue reading »
Postman – Interceptor Chrome Extension Postman has an alternate to fiddler to capture and replay the HTTP requests sent by browser. It is called Postman Interceptor. There is a number of steps you need to follow in order to get request interception. You can start with Postman and enable interceptor from the header. If Interceptor… Continue reading »
Curator for Managing Elastic Search Indices Curator is one of the officially supported plugins of Elastic Search. It supports a number of features including the following: Add or remove indices (or both!) from an alias Change shard routing allocation Close indices Create index Delete indices Delete snapshots Open closed indices forceMerge indices reindex indices, including… Continue reading »
Play Framework – Input line is too long… I have recently come across this problem where Play framework service wouldn’t run on some windows servers and result in the following error: The problem is APP_CLASS. Just update it to the following it should work fine: set “APP_CLASSPATH=%APP_LIB_DIR%\..\conf\;%APP_LIB_DIR%\*” Note: This would load all the libraries from… Continue reading »