Bootstrap

(场景)kafka的topic多分区的情况,如何保证跨区的消息消费的顺序性

这个问题严格来说是肯定有的,kafka只能保证分区内的有序性


下面是kafka作者Jay Kreps的blog中介绍kafka设计思想的一段话。

Each partition is a totally ordered log, but there is no global ordering between partitions (other than perhaps some wall-clock time you might include in your messages). The assignment of the messages to a particular partition is controllable by the writer, with most users choosing to partition by some kind of key (e.g. user id). Partitioning allows log appends to occur without co-ordination between shards and allows the throughput of the system to scale linearly with the Kafka cluster size.


针对部分消息有序(message.key相同的message要保证消费顺序)场景,可以在producer往kafka插入数据时控制,同一key分发到同一partition上面。


kafka源码如下,支持该方式

private [kafka] class DefaultPartitioner[T] extends Partitioner[T] {
   private
;