Migration of datastage code to Talend

Five Stars

Migration of datastage code to Talend

Hi,

My team is planning to move on to Talend from Datastage. We are in initial stage right now, were just trying to figure out if we can migrate Datastage code directly to Talend.

Can anyone help me with any thoughts.

Ten Stars

Re: Migration of datastage code to Talend

Not possible. Im not sure if I understand your question, what do you mean with "directly" ?
Side question, Im curious, why and for what reason?
Eight Stars

Re: Migration of datastage code to Talend

Hi @pushpsourav,

It is not directly Possible. You have to code in Talend. Though Talend will add lots of features which can not be achived directly in DataStage like FileWatcher,archiving in s3.

Five Stars

Re: Migration of datastage code to Talend

what FileWatcher,archiving do ?
Eight Stars

Re: Migration of datastage code to Talend

Like checking if any File comes then only trigger Staging Process--FileWatcher

To Send your File after your end to end Process to a Location(s3/Linux) after your Job completition-Archival

Five Stars

Re: Migration of datastage code to Talend

ohh that's indeed a good thing. For such things we were using shell scripts to do such tasks in Datastage Smiley Happy
Four Stars

Re: Migration of datastage code to Talend

Hi pushpsourav,

Good to know that you are also doing ETL Jobs migration from DataStage to Talend. I'm sure you and your team might be half way on your journey for full migration.

I'm working for a company which hired me as a Business Analyst for ETL Jobs migration from DataStage to Talend. As this is my first BA job in IT, i want to know what are the best practices and where to kick start the project? Any help is greatly appreciated.

 

Below are my questions 

1) Where should i start my analysis?

2) What all the documents i need to create?

3) Do you have any existing documentation. You can delete your client names and send it please.. i can give you my person email id, if you would like to.

 

Thanks a ton in advance.

 

Ramesh

 

Ten Stars

Re: Migration of datastage code to Talend

@rameshbongu

 

Im not migrating from Datastage... to ... talend... however, you're the business analyst and should know where to start to be honest.

However I do get your question on where to start regarding Talend and hope.

 

Many times I end up in discussions ... why using talend... or ... why not this or why not that. 

My advice, for what its worth, start with 3 things.
> Support, monitoring, performance, scheduling.

>> This is highly underestimated. Where to store or write your log-files to. 

>> Go for log4j and make sure its proper implemented in your etl-jobs

>> Make sure the teams fully understand tLogCatcher. tStatcatcher tFlowmeter. Use is wisely

>> Configuration of your jobs, think how to maintain configuration of your processes and the way its implemented...

>> When and how to process your jobs. Atomic approach, 

 

> (Technical) Datasources Integration

>> If you have a lot of api / web / ftp go for cUrl dont even bother using Talend components, components work but eventual these REST and SOAP components wont work as you wanted they're limited (search it).

>> Files, Databases, no problems at all.

 

> Seperation of concerns.

>> Keep your config manageable outside of talend. in a custom json, xml  (you have to design this yourself)

>> Extraction layer, processing layer, storage and re-running whenever your jobs failed.

>> Consider implementing middleware as an interface to your etl-processes. Often etl-teams try and create middleware processes to control this type of process.  Take a look at node-js and its roll/function in your data (etl) landscape

>> Keep it simple. 

 

No docs sorry.

Cheers and good luck

 

Jesper

Four Stars

Re: Migration of datastage code to Talend

Thank you very much for your reply. I really appreciate that.