CSV Splitter or Filter with Talend Java

One Star

CSV Splitter or Filter with Talend Java

We had a requirement today about splitting CSV which as you can see from http://thinkinginsoftware.blogspot.com/2012/10/the-data-team-was-in-need-of.html we resolved with a tJavaFlex component but ideally I think it should be either a new component in Talend or added as a future in existing ones.
The need is to parse an $inputFile using a $delimiter, and output only the lines where the $lookupColumn has a specific $lookupValue (four parameters).
Will this make sense to be added as a feature request? Or the recommended way to go would be to build an in house component? Probably Talend should open a project to host contributions as "talend custom components library"? A google search reveals tutorials on how to build custom components but I believe an open project to host the additional components will only help.
Thanks!
-Nestor
One Star

Re: CSV Splitter or Filter with Talend Java

tMap can certainly return matching rows only as that is one of the basic functionality of the application. On the fly schema is a different matter. Did you do a search at all and/or read the documentation?
One Star

Re: CSV Splitter or Filter with Talend Java

Yes. I did the research. We have been using talend for a while. The only reason you see me as a guest is that I cannot longer login in http://www.talendforge.org
When I login I land in talend.com and when I try to get back to http://www.talendforge.org I look like logged off and I can only post as guest.
I know what a tmap can do but that cannot provide the solution we need.
I searched on the web and this forum, I looked into the manual, I tool a look at all possibly related components before going the tJavaFlex route.
One Star

Re: CSV Splitter or Filter with Talend Java

I saw you kindly pointed out to the contextualized Help in Talend GUI in http://www.talendforge.org/forum/viewtopic.php?pid=96420#p96420 so I am researching on that now as "Dynamic Schema". I will get back with my findings. Thanks!
One Star

Re: CSV Splitter or Filter with Talend Java

Please correct me if I am wrong but all I see about "Dynamic Schema" refers to TalendIntegartionSuite_Studio_UG.pdf which is only available with a paid subscription meaning is not open source. So you must be a user of Talend Integration Suite to read the chapter from that book called "dynamic schema" .
So given the fact that as I posted in my tutorial it is fairly simple to build a CSV Splitter or Filter do you think it makes sense to request the addition of this component (or similar) to the open source version?
Thanks!
-Nestor
Community Manager

Re: CSV Splitter or Filter with Talend Java

Hi Nestor
Thanks for your posts!
Probably Talend should open a project to host contributions as "talend custom components library"?

Talend Exchange is the best place for all community users to share custom components, you can always find some useful components developed by community users.
A google search reveals tutorials on how to build custom components but I believe an open project to host the additional components will only help.

A good tutorial about how to create custom component is written by saburo, see
http://community.talend.com:80/t5/Design-and-Development/Writing-a-component-creation-tutorial/m-p/5...
Please correct me if I am wrong but all I see about "Dynamic Schema" refers to TalendIntegartionSuite_Studio_UG.pdf which is only available with a paid subscription meaning is not open source.

Yes, the dynamic schema feature is only available in commercial subscription version.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: CSV Splitter or Filter with Talend Java

Thanks Shong! I will check Talend Exchange next.
Great community here at Talend. Way to go, keep it up guys!
Best,
-Nestor
One Star

Re: CSV Splitter or Filter with Talend Java

I ended up creating a tFileInputCSVFilter available in github together with a tutorial to build components https://github.com/nestoru/talend-custom-components
Community Manager

Re: CSV Splitter or Filter with Talend Java

I ended up creating a tFileInputCSVFilter available in github together with a tutorial to build components https://github.com/nestoru/talend-custom-components

Hi nestoru
Great job! Thanks for your sharing to Talend community!
Why don't you upload an example job to github? It help the users who want to use this component to learn the component quickly.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: CSV Splitter or Filter with Talend Java

Hi Shong,
I have added a sample on how to use it in my blog http://thinkinginsoftware.blogspot.com/search?q=csvfilter#talendComponent
This component is really simple. There are 4 params to be used and the picture should be self explanatory.

Cheers,
-Nestor
Community Manager

Re: CSV Splitter or Filter with Talend Java

Hi Shong,
I have added a sample on how to use it in my blog http://thinkinginsoftware.blogspot.com/search?q=csvfilter#talendComponent
This component is really simple. There are 4 params to be used and the picture should be self explanatory.

Cheers,
-Nestor

Cool, Smiley Wink Thank you!
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: CSV Splitter or Filter with Talend Java

I have added two new features to the component BTW, now it is possible to have the header somewhere in the file other than on the top row. This is handy for some files containing garbage at the beginning. I am also pushing the headers to the output as this is a component to just filter the raw CSV file.
Cheers,
-Nestor