Introducing Keyed Messages

How does a producer know which partition to send a message to once we have more than one partition in a topic? By default (the message doesn’t have a key), the producer will publish messages in a round-robin fashion sending messages to different partitions. Additional routing modes are available. As we test Exclusive and Failover Subscriptions, we will notice message ordering is only guaranteed on a per-partition basis. We can reintroduce some additional ordering guarantees by adding a key to our messages. Messages with the same key will be routed to the same partition based on a hashing mechanism. This guarantees that messages of the same key will be consumed in order since they were sent to the same partition.

To add a key to your message, use the key() method when creating your message:

String myKey  = "cat";
MessageId msgID = producer.newMessage().key(myKey).value(message.getBytes()).send();

To increase efficiency, Pulsar clients use a batching mechanism when sending messages to a Pulsar cluster asynchronously. While we are currently using send() and not sendAsync(), its important to mention that when using sendAsync() you should use a KEY_BASED batcherBuilder in your Producer to ensure messages with the same key are batched together.

Producer<byte[]> producer = client.newProducer()
.topic("persistent://public/default/bytetopic")
.batcherBuilder(BatcherBuilder.KEY_BASED)
.create();