Fix Fetch Request Bug - Discard Remaining Response Data on Error

[aratz-lasa]

Description

This PR addresses a bug in the Fetch request handling logic. Currently, when an error occurs while reading the header (e.g., OffsetOutOfRange error), the buffer is not returned with the batch, and neither is the remaining response data discarded (specifically the records field in the response that is set to null, as it is a NULLABLE type). The non-discarded bytes remain as a prefix in the buffer. This leads to failures in the correlation ID verification for the next message.

To resolve this issue, we have implemented a fix that discards the remaining bytes in case of an error during Fetch request processing. This ensures that leftover data from previous erroneous responses does not affect subsequent requests.

Changes

• Updated the Fetch request handling logic to discard any remaining response data when an error occurs during header reading.

Testing

Due to the nature of this bug and the absence of utility to generate responses with errors in the conn_test file, it was challenging to create specific tests for this scenario. However, we have thoroughly reviewed and manually tested the changes to ensure the correctness and stability of the implementation.

[petedannemann]

@aratz-lasa this seems like a plausible fix but given that we've broken kafka-go before trying to fix this issue I would feel a lot better if we first found some way to:

1. Reproduce this issue in a test
2. Add test coverage to prevent us from breaking this in a similar way as we did in

#788