MCPcopy Create free account
hub / github.com/IBM/sarama / TestJavaMurmur2PartitionerInterop

Function TestJavaMurmur2PartitionerInterop

functional_java_interop_test.go:402–480  ·  view source on GitHub ↗

TestJavaMurmur2PartitionerInterop verifies that Sarama's NewMurmur2Partitioner routes keyed messages to the same partitions as the Apache Kafka Java client's DefaultPartitioner (which uses murmur2 internally).

(t *testing.T)

Source from the content-addressed store, hash-verified

400// routes keyed messages to the same partitions as the Apache Kafka Java client's
401// DefaultPartitioner (which uses murmur2 internally).
402func TestJavaMurmur2PartitionerInterop(t *testing.T) {
403 skipIfExistingEnvironment(t)
404 setupFunctionalTest(t)
405 defer teardownFunctionalTest(t)
406
407 checkKafkaVersion(t, "0.10.0")
408
409 const topic = "test.64"
410 const numPartitions = int32(64)
411
412 keyedMessages := []struct{ key, value string }{
413 {"foo", "value-foo"},
414 {"bar", "value-bar"},
415 {"baz", "value-baz"},
416 {"kafka", "value-kafka"},
417 {"sarama", "value-sarama"},
418 {"hello", "value-hello"},
419 {"world", "value-world"},
420 }
421
422 // record the end offsets per partition before we produce anything
423 startOffsets := make([]int64, numPartitions)
424 for p := int32(0); p < numPartitions; p++ {
425 startOffsets[p] = endOffsetForPartition(t, topic, p)
426 }
427
428 // produce with Java's DefaultPartitioner (murmur2)
429 produceKeyedWithJava(t, topic, keyedMessages)
430
431 // determine which partition each key landed on via Java
432 javaPartition := make(map[string]int32, len(keyedMessages))
433 for p := int32(0); p < numPartitions; p++ {
434 endOff := endOffsetForPartition(t, topic, p)
435 if endOff <= startOffsets[p] {
436 continue
437 }
438 msgs := consumeKeyedFromPartition(t, topic, p, startOffsets[p], int(endOff-startOffsets[p]))
439 for key := range msgs {
440 javaPartition[key] = p
441 }
442 }
443 require.Len(t, javaPartition, len(keyedMessages), "Java producer did not produce all messages")
444
445 // record new end offsets before Sarama production
446 for p := int32(0); p < numPartitions; p++ {
447 startOffsets[p] = endOffsetForPartition(t, topic, p)
448 }
449
450 // produce the same keyed messages with Sarama's murmur2 partitioner
451 config := NewFunctionalTestConfig()
452 config.Producer.Partitioner = NewMurmur2Partitioner
453 config.Producer.Return.Successes = true
454 config.Producer.RequiredAcks = WaitForAll
455
456 producer, err := NewSyncProducer(FunctionalTestEnv.KafkaBrokerAddrs, config)
457 require.NoError(t, err)
458 defer producer.Close()
459

Callers

nothing calls this directly

Calls 13

CloseMethod · 0.95
SendMessageMethod · 0.95
setupFunctionalTestFunction · 0.85
teardownFunctionalTestFunction · 0.85
checkKafkaVersionFunction · 0.85
endOffsetForPartitionFunction · 0.85
produceKeyedWithJavaFunction · 0.85
NewFunctionalTestConfigFunction · 0.85
StringEncoderTypeAlias · 0.85
NewSyncProducerFunction · 0.70

Tested by

no test coverage detected