PIP-189: No batching if only one message in batch

[AnonHxy]

discuss mail-thread: https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0

Motivation

The original discussion mail :
https://lists.apache.org/thread/5svpl5qp3bfoztf5fvtojh51zbklcht2

linked issue: #16547

Introduce the ability for producers to publish a non-batched message if there is only one message in the batch.

It is useful to save the SingleMessageMetadata space in entry  and reduce workload of consumers to deserialize the SingleMessageMetadata,  especially  when sometimes there is an amount of batched messages with only one real message.

API Changes

N/A

Implementation

For BatchMessageContainerImpl :

public OpSendMsg createOpSendMsg() throws IOException {
        if (!producer.conf.isBatchingSingleMessage() && messages.size() == 1) {
             // If only one message,  create OpSendMsg as non-batched publish.
        }
        
       // ....
}

For BatchMessageKeyBasedContainer,  there is no need to change, because it uses BatchMessageContainerImpl to create OpSendMsg

Reject Alternatives

• Always return BatchMessageIdImpl when enableBatching is set as true, even if publish single message  with a non-batched message.
  Rejection reason: Consumer have to deserialize to check if there is SingleMessageMetadata from the payload

• Add a configuration for the producer to enable or disable this feature
  Rejection reason: It is always a good idea to not create a batch message. So there is no reason to turn off this feature

[eolivelli]

I think that we should not add a configuration flag.
the fact that we are building batch messages or single messages is an implementation detail that is opaque to the Producer and to the Consumers.
Applications should not use Impl classes or count on a specific Impl class.

There is no reason to turn off this feature

[eolivelli]

@AnonHxy did you see my comment ?

[AnonHxy]

@AnonHxy did you see my comment ?

Yes. I saw the comment. And I had replied from the email. Maybe I should attatch the link to this issue. :)
https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0

I agree with your point. And I have updated the PR to remove the configuration flag. But there is still some flaky-test failure and I am working on it. Thanks @eolivelli