My team is planning to move on to Talend from Datastage. We are in initial stage right now, were just trying to figure out if we can migrate Datastage code directly to Talend.
Can anyone help me with any thoughts.
It is not directly Possible. You have to code in Talend. Though Talend will add lots of features which can not be achived directly in DataStage like FileWatcher,archiving in s3.
Like checking if any File comes then only trigger Staging Process--FileWatcher
To Send your File after your end to end Process to a Location(s3/Linux) after your Job completition-Archival
Good to know that you are also doing ETL Jobs migration from DataStage to Talend. I'm sure you and your team might be half way on your journey for full migration.
I'm working for a company which hired me as a Business Analyst for ETL Jobs migration from DataStage to Talend. As this is my first BA job in IT, i want to know what are the best practices and where to kick start the project? Any help is greatly appreciated.
Below are my questions
1) Where should i start my analysis?
2) What all the documents i need to create?
3) Do you have any existing documentation. You can delete your client names and send it please.. i can give you my person email id, if you would like to.
Thanks a ton in advance.
Im not migrating from Datastage... to ... talend... however, you're the business analyst and should know where to start to be honest.
However I do get your question on where to start regarding Talend and hope.
Many times I end up in discussions ... why using talend... or ... why not this or why not that.
My advice, for what its worth, start with 3 things.
> Support, monitoring, performance, scheduling.
>> This is highly underestimated. Where to store or write your log-files to.
>> Go for log4j and make sure its proper implemented in your etl-jobs
>> Make sure the teams fully understand tLogCatcher. tStatcatcher tFlowmeter. Use is wisely
>> Configuration of your jobs, think how to maintain configuration of your processes and the way its implemented...
>> When and how to process your jobs. Atomic approach,
> (Technical) Datasources Integration
>> If you have a lot of api / web / ftp go for cUrl dont even bother using Talend components, components work but eventual these REST and SOAP components wont work as you wanted they're limited (search it).
>> Files, Databases, no problems at all.
> Seperation of concerns.
>> Keep your config manageable outside of talend. in a custom json, xml (you have to design this yourself)
>> Extraction layer, processing layer, storage and re-running whenever your jobs failed.
>> Consider implementing middleware as an interface to your etl-processes. Often etl-teams try and create middleware processes to control this type of process. Take a look at node-js and its roll/function in your data (etl) landscape
>> Keep it simple.
No docs sorry.
Cheers and good luck
Watch the recorded webinar!
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.