I've had my fair share of wierd and wonderful stuff within Talend, but now I'm doing something a little more simple. Unfortunately I seem to be struggling to put this into words even so I cannot seem to find anything about it online.
Basically, I have a component which only accepts a file (csv or excel) as the data input. This is very sad news for me because I originally thought that I would be able to extract my data from a database and send this directly to the output of mentioned component without the use of a file.
I've used Talend a lot, but I cannot think of a way to do this. I'm sure there must be one? Is there a simple Talend/Java command that can be put into the 'Source File' line to trick the component into grabbing the data from the main input row?
Any help on this is appreciated. Also any other ideas are more than welcome as this is currently a major issue for myself.
I'm a bit confused by this question. What is it that this file component does that is so special that you have to use it? Is there not an alternative that will take the data as a flow? Anyway, if you must use this component, can you not use a DB component to take the data from the database and put it into a file so that it can be used by this component?
The component is actually a beta test component currently not implemented otherwise. It basically updates a file in their cloud, however to update their file, you need to source the (new) data file. No I cannot use a different component as this is specific for their cloud.
Could I input the data from the database into a file first? Yes - We've tested it this way already and it works. However for business purposes this is not acceptable or viable for us at all, so I'm trying to move away from the middle guy (the file) and do a straight "extract data from database --> update file with data" process.
I hope this helps you understand my question better.
I see. It does seem strange that you can only update a file from the file itself. Very strange. I would get back to the team running the beta test and point that out :-) As it is a beta test component and I have not used it, I don't think I can help. Good luck
Okay, well thank you for your replies at least Rhall.
Though, one question which you might be able to help me with then perhaps (as this is my only current thought on the matter): Is there a java/talend command that picks up data from the input row(s)? Something like the following perhaps:
I'm really going out there on this one, it is something I'm not sure can be done. But like you say, I will mention this to the team and point this out, as personally they are using Talend but putting their component behind a massive screen blocking them from half the stuff Talend is very good and useful for.
You can use the globalMap hashmap to hold values. You can also use context variables. I have just written a job where I needed to pass a complete recordset to a child job. To do this I created a Java class to hold my data and then added the data to instances of that class in a tJavaFlex. I then added them to a HashMap. Once the HashMap held all of the data, I then set a context variable of type Object to be the value of the HashMap. I then passed the context variable (as an Object) to the child job. Inside the child job I then cast the Object context variable back to the correct type (HashMap containing instances of my class) and then used the data within the child job. That is a relatively complicated example, but it shows that practically anything is possible in Talend if you can write Java.
Would it be possible you could give me a simple example of the java for 2 things:
"I then added them to a HashMap." How to do this? & "..set a context variable of type Object to be the value of the HashMap.." How to do this?
I am not very knowledgeable with java, I have only learned and used what I needed within Talend here and there, so it is limited. I am confident how to save values as context variables in general however, and I assume the above is similar. It does sound like what I am looking for!
OK, this is a REALLY complicated example (based on what you said your Java knowledge is), but you might be able to work with it. You can download the job and related material and if you understand the Oracle "Connect-by" functionality, it will make more sense to you. Sorry I can't be of more help, but the question you have asked would take a bit more time to answer specifically for your requirement. I hope this helps....
Thank you very much once more Rhall. This looks interest and I will see what I can do with it. I will leave the topic 'Un-resolved' for now just incase someone else comes along with a different idea, but I will update this in future accordingly.
Hi, I am designing a job for sync operation at Intercom server such as - I have two DB, One is have live data and second is have intermediate data, so if both DB of user table have any differences then i have to track three conditions (Add,Remove,Update) as per first DB. I designed this job and maintain the flags in a different table. Secondly, I have to fetch these three users which would be sync at intercom as per these tags. I have to call Intercom bulk api for sending users information by using tRest component and client told us that we should not use java code for this. So, how can we transfer our users which is available in tmap component in the job. Please suggest me its urgent.