To ensure that duplicate messages are not processed and no messages are skipped, ksqlDB’s persistent queries must be configured to provide exactly-once processing guarantees. This is achieved by setting the processing.guarantee property to exactly_once. This configuration ensures that messages are neither lost nor processed more than once in the event of failures or restarts.

As per the official Confluent documentation:

“ksqlDB persistent queries can be configured for either at-least-once or exactly-once processing semantics by setting the processing.guarantee configuration property.”

“When processing.guarantee is set to exactly_once, ksqlDB ensures that messages are processed only once, even in failure scenarios. This eliminates duplicates and guarantees strong message delivery semantics.”

Other options:

Option B and C (ksql.streams.auto.offset.reset) only control the starting offset for new consumers, and do not enforce exactly-once semantics.

Option D (ksql.fail.on.production.error) controls whether errors during message production should fail the query, and does not affect exactly-once processing.

Page Reference:

Confluent Kafka: The Definitive Guide, 1st Edition, Chapter 11 (Stream Processing with ksqlDB), p. 365.

Confluent Documentation, “ksqlDB Configuration Properties” (processing.guarantee).

??????????????????????????????????????????????

Kafka performs an authorization check every time a client attempts to access a resource (topics, consumer groups, etc.), ensuring real-time enforcement of security policies.

From Kafka documentation:

“Kafka checks authorization for every operation, every time the resource is accessed.”

This behavior ensures that any changes to ACLs are applied immediately and consistently.

Page Reference:

Kafka: The Definitive Guide, 1st Edition, Chapter 9, p. 284

Apache Kafka Documentation: “Kafka Security - Authorization”

??????????????????????????????????????????

In Kafka, the Controller is responsible for managing partition leadership. When a broker that was leader for certain partitions goes down, the Controller immediately detects the failure and:

Triggers a new leader election among the in-sync replicas (ISRs) of those partitions.

Updates the partition metadata to reflect the new leaders and notifies the cluster.

Kafka documentation states:

“When a broker fails, the controller triggers a leader election for all partitions previously led by the failed broker.”

Page Reference:

Kafka: The Definitive Guide, 1st Edition, Chapter 6, p. 190–191

Apache Kafka Documentation: “Kafka Controller and Leader Election”

??????????????????????????????????????????