spring-cloud-stream kafka how to produce message with round robin strategy?
up vote
2
down vote
favorite
We have upgrade spring boot to 2.0.3 Release (previously 1.5.10).
Due to that, we have changed the release of kafka & spring-cloud-stream (2.0.0-Release)
In our property YML file we dont always configure partitionKeyExpression and most of our partitionCount is greater than 1.
In previous release, these settings along with not passing any key would lead to produce messages in round robin strategy.
In new release, same settings leads to method isPartitioned to return true, which adds Interceptor with new PartitionHandler where extractKey method fail due to missing key (assert with message "Partition key cannot be null").
The question is, How can we maintain the previous behavior with the same settings or with less changes. How we can keep not using a key and ensure messages goes to partitions in round robin strategy?
spring-boot apache-kafka spring-cloud-stream
asked Nov 18 at 11:44
Shay Hatab
113
add a comment |
up vote
2
down vote
favorite
We have upgrade spring boot to 2.0.3 Release (previously 1.5.10).
Due to that, we have changed the release of kafka & spring-cloud-stream (2.0.0-Release)
In our property YML file we dont always configure partitionKeyExpression and most of our partitionCount is greater than 1.
In previous release, these settings along with not passing any key would lead to produce messages in round robin strategy.
In new release, same settings leads to method isPartitioned to return true, which adds Interceptor with new PartitionHandler where extractKey method fail due to missing key (assert with message "Partition key cannot be null").
The question is, How can we maintain the previous behavior with the same settings or with less changes. How we can keep not using a key and ensure messages goes to partitions in round robin strategy?
spring-boot apache-kafka spring-cloud-stream
asked Nov 18 at 11:44
Shay Hatab
113
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
We have upgrade spring boot to 2.0.3 Release (previously 1.5.10).
Due to that, we have changed the release of kafka & spring-cloud-stream (2.0.0-Release)
In our property YML file we dont always configure partitionKeyExpression and most of our partitionCount is greater than 1.
In previous release, these settings along with not passing any key would lead to produce messages in round robin strategy.
In new release, same settings leads to method isPartitioned to return true, which adds Interceptor with new PartitionHandler where extractKey method fail due to missing key (assert with message "Partition key cannot be null").
The question is, How can we maintain the previous behavior with the same settings or with less changes. How we can keep not using a key and ensure messages goes to partitions in round robin strategy?
spring-boot apache-kafka spring-cloud-stream
asked Nov 18 at 11:44
Shay Hatab
113
We have upgrade spring boot to 2.0.3 Release (previously 1.5.10).
Due to that, we have changed the release of kafka & spring-cloud-stream (2.0.0-Release)
In our property YML file we dont always configure partitionKeyExpression and most of our partitionCount is greater than 1.
In previous release, these settings along with not passing any key would lead to produce messages in round robin strategy.
In new release, same settings leads to method isPartitioned to return true, which adds Interceptor with new PartitionHandler where extractKey method fail due to missing key (assert with message "Partition key cannot be null").
The question is, How can we maintain the previous behavior with the same settings or with less changes. How we can keep not using a key and ensure messages goes to partitions in round robin strategy?
spring-boot apache-kafka spring-cloud-stream
spring-boot apache-kafka spring-cloud-stream
asked Nov 18 at 11:44
Shay Hatab
113
asked Nov 18 at 11:44
Shay Hatab
113
edited Nov 18 at 12:22
asked Nov 18 at 11:44
Shay Hatab
113
asked Nov 18 at 11:44
Shay Hatab
113
asked Nov 18 at 11:44
Shay Hatab
113
113
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
The count is now used in the decision.
public boolean isPartitioned() {
return this.partitionCount > 1 || this.partitionKeyExpression != null
|| this.partitionKeyExtractorName != null || this.partitionKeyExtractorClass != null;
}
I think this is wrong #1531.
As a work around, don't set the stream producer.partitionCount if you want to use native kafka partitioning (and the topic already exists).
answered Nov 18 at 15:53
Gary Russell
76.8k64166
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
The count is now used in the decision.
public boolean isPartitioned() {
return this.partitionCount > 1 || this.partitionKeyExpression != null
|| this.partitionKeyExtractorName != null || this.partitionKeyExtractorClass != null;
}
I think this is wrong #1531.
As a work around, don't set the stream producer.partitionCount if you want to use native kafka partitioning (and the topic already exists).
answered Nov 18 at 15:53
Gary Russell
76.8k64166
add a comment |
up vote
1
down vote
The count is now used in the decision.
public boolean isPartitioned() {
return this.partitionCount > 1 || this.partitionKeyExpression != null
|| this.partitionKeyExtractorName != null || this.partitionKeyExtractorClass != null;
}
I think this is wrong #1531.
As a work around, don't set the stream producer.partitionCount if you want to use native kafka partitioning (and the topic already exists).
answered Nov 18 at 15:53
Gary Russell
76.8k64166
add a comment |
up vote
1
down vote
up vote
1
down vote
The count is now used in the decision.
public boolean isPartitioned() {
return this.partitionCount > 1 || this.partitionKeyExpression != null
|| this.partitionKeyExtractorName != null || this.partitionKeyExtractorClass != null;
}
I think this is wrong #1531.
As a work around, don't set the stream producer.partitionCount if you want to use native kafka partitioning (and the topic already exists).
answered Nov 18 at 15:53
Gary Russell
76.8k64166
The count is now used in the decision.
public boolean isPartitioned() {
return this.partitionCount > 1 || this.partitionKeyExpression != null
|| this.partitionKeyExtractorName != null || this.partitionKeyExtractorClass != null;
}
I think this is wrong #1531.
As a work around, don't set the stream producer.partitionCount if you want to use native kafka partitioning (and the topic already exists).
answered Nov 18 at 15:53
Gary Russell
76.8k64166
edited Nov 18 at 17:46
answered Nov 18 at 15:53
Gary Russell
76.8k64166
answered Nov 18 at 15:53
Gary Russell
76.8k64166
answered Nov 18 at 15:53
Gary Russell
76.8k64166
76.8k64166
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53360492%2fspring-cloud-stream-kafka-how-to-produce-message-with-round-robin-strategy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
