Skip to content

Add a simple split interceptor #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: trunk
Choose a base branch
from
Open

Add a simple split interceptor #14

wants to merge 6 commits into from

Conversation

seekingua
Copy link

a simple split interceptor , aims to deal with the situation as follows:

the source data like this:

1,tom,boy,13
2,lili,girl,14
3,jack,boy,10
...

and i hope to sink source data into two different hdfs directories named by boy and girl like this:

hdfs://xxxx/sink/boy/....
hdfs://xxxx/sink/girl/....

we can use this interceptor to accomplish this goal.

@harishreedharan
Copy link
Contributor

We don't accept pull requests. Could you please attach the patch to a jira here: https://issues.apache.org/jira/browse/FLUME ? Thanks!

@seekingua
Copy link
Author

@harishreedharan
hi harishreedharan, i have created FLUME-2781 and attached the patch, is that ok? Thanks!

@harishreedharan
Copy link
Contributor

Yep, a committer will review he patch. Thanks @SeekerResource!

@lew181818
Copy link

Hello - we are interested in this enhancement. Do you have a readme available? Has this been merged into a master version? The JIRA ticket FLUME-2781 does not refer to this issue. Thanks!

@bessbd
Copy link
Member

bessbd commented Nov 4, 2016

@lew181818 : what kind of readme are you looking for?

I don't think this has been merged yet.

Also, the corresponding JIRA ticket is https://issues.apache.org/jira/browse/FLUME-2807 , I believe.

@lew181818
Copy link

@bessbd : For a readme, something simple like should we be using multiplexing source to direct the output into different sinks? Any other configuration to know to get this working in flume?

Thanks for the jira ticket.

@bessbd
Copy link
Member

bessbd commented Nov 7, 2016

@lew181818 : if you are only looking for a documentation on the configuration, maybe https://github.com/apache/flume/pull/14/files#diff-4d9ec6998a1233dda992589304f831d4R67 should help.

Please, let me know, if I misunderstood your request.

Btw, if there is demand for this PR to be merged, we can reboot the review process.

hejiang2000 added a commit to hejiang2000/flume that referenced this pull request Apr 13, 2018
…r-join-interceptor

修改 joinKeys 默认值为 key,简化代码。
@asfgit
Copy link

asfgit commented Aug 17, 2018

Can one of the admins verify this patch?

waidr pushed a commit to waidr/flume that referenced this pull request Jul 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants