Input Row As Data File

One Star

Input Row As Data File

Hi all,

I've had my fair share of wierd and wonderful stuff within Talend, but now I'm doing something a little more simple. Unfortunately I seem to be struggling to put this into words even so I cannot seem to find anything about it online.

Basically, I have a component which only accepts a file (csv or excel) as the data input. This is very sad news for me because I originally thought that I would be able to extract my data from a database and send this directly to the output of mentioned component without the use of a file.

I've used Talend a lot, but I cannot think of a way to do this. I'm sure there must be one? Is there a simple Talend/Java command that can be put into the 'Source File' line to trick the component into grabbing the data from the main input row?

Any help on this is appreciated. Also any other ideas are more than welcome as this is currently a major issue for myself.

Regards,
V Pem.
Community Manager

Re: Input Row As Data File

I'm a bit confused by this question. What is it that this file component does that is so special that you have to use it? Is there not an alternative that will take the data as a flow? Anyway, if you must use this component, can you not use a DB component to take the data from the database and put it into a file so that it can be used by this component?
One Star

Re: Input Row As Data File

Hi Rhall,

The component is actually a beta test component currently not implemented otherwise. It basically updates a file in their cloud, however to update their file, you need to source the (new) data file. No I cannot use a different component as this is specific for their cloud.

Could I input the data from the database into a file first? Yes - We've tested it this way already and it works. However for business purposes this is not acceptable or viable for us at all, so I'm trying to move away from the middle guy (the file) and do a straight "extract data from database --> update file with data" process.

I hope this helps you understand my question better.

Regards,
Community Manager

Re: Input Row As Data File

I see. It does seem strange that you can only update a file from the file itself. Very strange. I would get back to the team running the beta test and point that out :-) 
As it is a beta test component and I have not used it, I don't think I can help. Good luck
One Star

Re: Input Row As Data File

Okay, well thank you for your replies at least Rhall.

Though, one question which you might be able to help me with then perhaps (as this is my only current thought on the matter): Is there a java/talend command that picks up data from the input row(s)? Something like the following perhaps:

globalMap.put("code_1", input_row.code);
globalMap.get("code_1", input_row.code);

I'm really going out there on this one, it is something I'm not sure can be done. But like you say, I will mention this to the team and point this out, as personally they are using Talend but putting their component behind a massive screen blocking them from half the stuff Talend is very good and useful for.

Thanks again.
Community Manager

Re: Input Row As Data File

You can use the globalMap hashmap to hold values. You can also use context variables. I have just written a job where I needed to pass a complete recordset to a child job. To do this I created a Java class to hold my data and then added the data to instances of that class in a tJavaFlex. I then added them to a HashMap. Once the HashMap held all of the data, I then set a context variable of type Object to be the value of the HashMap. I then passed the context variable (as an Object) to the child job. Inside the child job I then cast the Object context variable back to the correct type (HashMap containing instances of my class) and then used the data within the child job. 
That is a relatively complicated example, but it shows that practically anything is possible in Talend if you can write Java.
One Star

Re: Input Row As Data File

Would it be possible you could give me a simple example of the java for 2 things:

"I then added them to a HashMap." How to do this?
&
"..set a context variable of type Object to be the value of the HashMap.." How to do this?

I am not very knowledgeable with java, I have only learned and used what I needed within Talend here and there, so it is limited. I am confident how to save values as context variables in general however, and I assume the above is similar. It does sound like what I am looking for! Smiley Happy

Thanks again.
Community Manager

Re: Input Row As Data File

OK, this is a REALLY complicated example (based on what you said your Java knowledge is), but you might be able to work with it. You can download the job and related material and if you understand the Oracle "Connect-by" functionality, it will make more sense to you. Sorry I can't be of more help, but the question you have asked would take a bit more time to answer specifically for your requirement. I hope this helps....

http://www.rilhia.com/node/32
One Star

Re: Input Row As Data File

Thank you very much once more Rhall. This looks interest and I will see what I can do with it. I will leave the topic 'Un-resolved' for now just incase someone else comes along with a different idea, but I will update this in future accordingly.

Regards.
One Star

Re: Input Row As Data File

Hi,
I am designing a job for sync operation at Intercom server such as -
I have two DB, One is have live data and second is have intermediate data, so if both DB of user table have any differences then i have to track three conditions (Add,Remove,Update) as per first DB. I designed this job and maintain the flags in a different table.
Secondly, I have to fetch these three users which would be sync at intercom as per these tags.
I have to call Intercom bulk api for sending users information by using tRest component and client told us that we should not use java code for this. So, how can we transfer our users which is available in tmap component in the job.
Please suggest me its urgent.

Thanks in advance.
Highlighted
One Star

Re: Input Row As Data File

Hey,

I am wondering that where i will post my question but i found this bug so anyone is able to see my question so please suggest me.

Thanks.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog