Tag Archives: big data

Kafka Tools – kafka-delete-records

Kafka Tools – kafka-delete-records While working with kafka we, sometimes, need to purge records from a topic. This is specially needed in a development environment where we just want to get rid of some records and want to keep the other ones. Previously we have discussed how we can purge all records from a topic… Continue reading »

Kafka Connect – Single Message Transforms (SMTs)

Kafka Connect – Single Message Transforms (SMTs) Single Message Transforms were released with 0.10.2 release [ Release notes ]. It provides us the ability to transform a message before they get in or out of a connector using Kafka Connect. Source Transforms Source connector can be configured with a list of transforms. These transforms are… Continue reading »

KAFKA REST Proxy – Publishing Avro Messages to Kafka

KAFKA REST Proxy – Publishing Avro Messages to Kafka Apache Kafka supports message publishing through REST too. We can use Kafka REST proxy for this purpose. It supports http verbs including GET, POST and DELETE. Her is an example POST using curl which we will be trying to dissect throughout this post: Required Schema Apache… Continue reading »

Neo4j – An Introduction

Neo4j – An Introduction A graph database allows data to be saved in terms of nodes and edges and their properties. This type of database suits the business cases where entities and their relationships defines the whole concept. They are faster to query too. In addition to storing data as nodes and relationships, they provide… Continue reading »