I am looking for an optimized solution,
the goal is to migrate data from one mysql database to another, my concern is to define a text file that contains the parameters that defines the source database (host, port, databasename ..) and also the file parameters of the output database.
Do you have any ideas how to do this work ?
actually, what I use is the following schema:
tContextLoad_1 -> tFileInputDelimited_1-> tMysqlInput_1 (db_source) -> tMap->
depending on the problem that has occurred is:
1- the tContextLoad_1 text file content change is not supported and the pointing is always on the first database "source database".
2- When I create a second tContextLoad_2 can not be connected by a (but) or (trigger-On subject Ok) with tFileOutputDelimited_1 that linked with tMysqlInput_2 (db_destination), how to define a file that contains the definition parameters of a base output data (host, databasename, identifier, password ...) ??
I created a job that does the job using its own context, I use export the database parameters as conxtext, the problem is after depolement talend create his own file that contains the settings against my work is to retrieve data from a database that can change its environment, so finally it uses its own default.properties parameter file.
Do you have an idea how on how to represent a database that can change its environment?
Define all the context variables you need for database connection and other as usually but don't set any value for these variables (they will come from an external source).
Define a new context variable (let say ContextFilepath) to define the full pathname of the file in which you will put the values for your context variables.
Go to the Job tab then select the Extra tab.
Untick the option "Use Project Settings".
Tick the options "Implicit tContetxLoad" and "From File".
Enter "context.ContextFilepath" into the "From File" field".
Enter "==" into the "Field Separator" field (or choose an othe separator).
It shoukd look like this:
From outside the Studio, create a file (name and path according to the ContextFilepath context variable.
Enter variable names and values for all your context variables (except ContextFilepath) like this:
variable1 == value1
variable2 == value2
Now, each time you'll run your job, context variables will be loaded from your properties file (except ContextFilepath) so you can forget the stardard context files.
If you need different contexts, just create a properties file per context and set the variable ContextFilepath with the correct full pathname, and that's all.
Hope this help.
very well explained Can you give the steps of creating the job exactly, because you mentioned components like tContextFilPath that I never used.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.