Skip to content

Kafka Source in combination with Kafka Sink leads to infinite loop #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from
Open

Kafka Source in combination with Kafka Sink leads to infinite loop #65

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions flume-ng-doc/sphinx/FlumeUserGuide.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1286,6 +1286,10 @@ Other Kafka Consumer Properties -- These properties are used to co
The Kafka Source also provides defaults for the key.deserializer(org.apache.kafka.common.serialization.StringSerializer)
and value.deserializer(org.apache.kafka.common.serialization.ByteArraySerializer). Modification of these parameters is not recommended.

.. note:: The Kafka Source sets several event headers if they don't exist already: "timestamp", "topic", "partition" and "key" (if it's a keyed kafka message).
Especially the header field "topic" can lead to unexpected behaviour when used in combination with the Kafka Sink.
This header field overwrites the destination topic of the sinks configuration and leads to an indefinite loop producing the messages to into the topic where they has been read from.

Deprecated Properties

=============================== =================== =============================================================================================
Expand Down