Comprehensive and Detailed Explanation From Exact Extract:
* #C. Consumers can consume both committed and uncommitted transactions.By default,Kafka consumers only read committed messagesif they are configured with isolation.level=read_committed.
However, if configured as read_uncommitted, theycan also consume uncommitted (potentially aborted) transactional messages.
From Kafka Documentation:
"The isolation.level setting controls whether the consumer will read only committed messages or all messages, including uncommitted messages from ongoing or aborted transactions."
* #D. Information about producers and their transactions is stored in the _transaction_state topic.
Kafka uses an internal topic named__transaction_stateto maintain metadata about producer transactions. This topic is essential for tracking thetransaction lifecycle, fencing, and recovery.
From Kafka Internals:
"Kafka stores the state of active and completed transactions in an internal topic called __transaction_state."

This metric shows the current lag (number of messages behind the broker)

The onCompletion() method of Kafka'sCallback interfaceis executedonce the broker acknowledgesthe message. This includes success or failure, and it isinvoked asynchronouslyby the producer.
FromKafka Java Client API Documentation:
"The onCompletion method will be called when the record sent to the server has been acknowledged, or when an error occurs." This ensures thatthe record was sent and acknowledged, not just added to a batch or local buffer.
Reference:Kafka Java Client API > org.apache.kafka.clients.producer.Callback

The REST Proxy requires to receive data over REST that is already base64 encoded, hence it is the responsibility of the producer