Code migration from DEV to QA

Highlighted
Seven Stars

Code migration from DEV to QA

Hello All,

Can you please help me with this simple scenario....

I have a simple job that reads table DEV_DB.src_T1 and loads DEV_DB.Tgt_T1. I have developed and tested this on my Dev DB and it works well.

Now I want to do the following, where i need your help:

1) Assuming QA is nothing but a different folder on a different machine (weird but looks like that's what it is in Talend) how do i move the code to QA, is it just a export and import? When i import, how do i ensure that the code in QA folder points to QA DB and not point to DEV DB

Meaning job reads table QA_DB.src_T1 and loads QA_DB.Tgt_T1

 

This is a very very basic need of any project and i don't find a direct answer to this simple problem from the documentation. I have read through most of the related threads on similar topics where people are talking about context variables and all those threads are left half way through...and some are pointing to Repository manager, don't know where that Repository Manager is.

 

Can you share some thoughts on this please.

 

Tags (1)
Eight Stars

Re: Code migration from DEV to QA

I agree that this is not as slick as I would like.

 

We have two approaches to this. One is to store context variables in a MySQL database, and we have a Joblet that reads the values from the database. Pretty much every job has this joblet at the start, and reads the values based on a single context key (context.Interface_Key) and populates the rest of the context variables that are in the database. Others that are not in the database are as default or specified by the user or TAC entry.

 

The other is to use a metadata object, for example the MySQL connection. The conneciton in Dev points to one database server, in QA it points to another, and in production to a third. The down-side of this is that when promoting code from one environment to another, we need to open and save every job in the Studio so as to pick up that environment's copy of the MySQL connection. I have just created a thread in this forum to explore better ways of doing this.

Seven Stars

Re: Code migration from DEV to QA

Thank You PhilHibbs..I pretty much ended doing the second approach with a very slight difference...looks like it is working well, will respond to my own post once i am personally confident that it is working.

 

Interestingly, this is a very very basic phenomena and i am surprised to see people who claim to be using Talend for some time, didn't pay attention to the question..either because it is too basic or because they haven't done anything on those lines, which i think is really slim.

 

To add to all this, even Talend documentation/deployment section doesn't talk about this...may be a request to Talend to please incorporate this useful basic information in their Deployment documentation.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog