Skip to content

Conversation

emasab
Copy link
Contributor

@emasab emasab commented Oct 16, 2025

Closes #286.

batch size is now configurable through the 'js.max.batch.size` property (-1 for unlimited batch size). Even when unlimited batch sizes are actually limited by librdkafka consumer buffer.

Checklist

  • Contains customer facing changes? The new js property.
  • Did you add sufficient unit test and/or integration test coverage for this PR? TODO

Test & Review

automatic tests and running the performance example

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from e881b8f to 6900c8c Compare October 16, 2025 11:37
@emasab emasab changed the base branch from master to dev_performance_test_improvements October 16, 2025 11:37
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from 9e73424 to ed0fb03 Compare October 16, 2025 11:38
@emasab emasab changed the base branch from dev_performance_test_improvements to master October 16, 2025 11:41
@emasab emasab changed the base branch from master to dev_performance_test_improvements October 16, 2025 11:42
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 6900c8c to 9b3c43a Compare October 16, 2025 12:07
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch 2 times, most recently from 15928f1 to 4209743 Compare October 17, 2025 10:29
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 9b3c43a to 25ec968 Compare October 17, 2025 10:32
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_performance_test_improvements branch from 4209743 to 9f4461f Compare October 17, 2025 11:02
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 25ec968 to 7d20ea4 Compare October 17, 2025 11:04
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 7d20ea4 to e8bd34f Compare October 17, 2025 12:26
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 8bb5221 to 4ad7f02 Compare October 19, 2025 12:39
rebalances in case of long processing time
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch 3 times, most recently from f776f0c to 0a91e33 Compare October 19, 2025 17:34
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 0a91e33 to e62aee1 Compare October 19, 2025 17:48
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 92368f1 to cfa7321 Compare October 19, 2025 18:27
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_fix_configurable_batch_size branch from 94e66ba to 4f22357 Compare October 19, 2025 19:30
@sonarqube-confluent

This comment has been minimized.

@sonarqube-confluent
Copy link

Failed

  • 72.50% Coverage on New Code (is less than 80.00%)

Analysis Details

11 Issues

  • Bug 0 Bugs
  • Vulnerability 4 Vulnerabilities
  • Code Smell 7 Code Smells

Coverage and Duplications

  • Coverage 72.50% Coverage (48.40% Estimated after merge)
  • Duplications No duplication information (2.00% Estimated after merge)

Project ID: confluent-kafka-javascript

View in SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

batch size limit is severely limiting compared to KafkaJS

1 participant