look for an optimized solution

Highlighted
Four Stars

look for an optimized solution

Hello, 

 

I am looking for an optimized solution,
the goal is to migrate data from one mysql database to another, my concern is to define a text file that contains the parameters that defines the source database (host, port, databasename ..) and also the file parameters of the output database.
Do you have any ideas how to do this work ?

 

Thank you

Highlighted
Four Stars

Re: look for an optimized solution

Please check for how to setup contexts in the Talend flow. This way all dB information will be picked up as the job starts.
Highlighted
Four Stars

Re: look for an optimized solution

actually, what I use is the following schema:
tContextLoad_1 -> tFileInputDelimited_1-> tMysqlInput_1 (db_source) -> tMap->
tMysqlOutput_2 (db_destination)

depending on the problem that has occurred is:
1- the tContextLoad_1 text file content change is not supported and the pointing is always on the first database "source database".
2- When I create a second tContextLoad_2 can not be connected by a (but) or (trigger-On subject Ok) with tFileOutputDelimited_1 that linked with tMysqlInput_2 (db_destination), how to define a file that contains the definition parameters of a base output data (host, databasename, identifier, password ...) ??

Highlighted
Four Stars

Re: look for an optimized solution

I have no idea until now, i'm searching how to do it.

Highlighted
Four Stars

Re: look for an optimized solution

I created a job that does the job using its own context, I use export the database parameters as conxtext, the problem is after depolement talend create his own file that contains the settings against my work is to retrieve data from a database that can change its environment, so finally it uses its own default.properties parameter file.


Do you have an idea how on how to represent a database that can change its environment?

Highlighted
Sixteen Stars
Sixteen Stars

Re: look for an optimized solution

Hi,

Define all the context variables you need for database connection and other as usually but don't set any value for these variables (they will come from an external source).

Define a new context variable (let say ContextFilepath) to define the full pathname of the file in which you will put the values for your context variables.

Go to the Job tab then select the Extra tab.

Untick the option "Use Project Settings".

Tick the options "Implicit tContetxLoad" and "From File".

Enter "context.ContextFilepath" into the "From File" field".

Enter "==" into the "Field Separator" field (or choose an othe separator).

It shoukd look like this:

Capture.PNG

From outside the Studio, create a file (name and path according to the ContextFilepath context variable.

Enter variable names and values for all your context variables (except ContextFilepath) like this:

variable1 == value1
variable2 == value2
...

Now, each time you'll run your job, context variables will be loaded from your properties file (except ContextFilepath) so you can forget the stardard context files.

If you need different contexts, just create a properties file per context and set the variable ContextFilepath with the correct full pathname, and that's all.

Hope this help.

 


TRF
Highlighted
Four Stars

Re: look for an optimized solution

very well explained Can you give the steps of creating the job exactly, because you mentioned components like tContextFilPath that I never used.

 

sincerely yours.

Highlighted
Sixteen Stars
Sixteen Stars

Re: look for an optimized solution

You don't need any additional component for this solution.
Just follow the instructions from my previous answer and that's all.
Don't forget to mark your case as solved as soon as you get the expected result for other guys facing a similar question (Kudos also accepted).

TRF

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog