Kafka Message Size Issue – Record Too Long to Send

As soon as the message size increases, Kafka starts throwing errors when sending the message. Here are the error message we received when sending a message with size a few mege bytes.

RecordTooLarge

The message clearly suggests that the size needs to updated in some configuration file before we could send it. Since we are specifying the sending details in the properties configuration, let’s add the configuration for the message size in the properties configuration too.

But the error message doesn’t go away but it changes to a different form. We are still getting a similar message:

Record Too Large for Server

You also need to update it on the Confluent side. Here are the updates I made to server.properties and producer.properties as specified at this github post: