I am using Talend Open Studio Free version 6.3.0.
I have a requirement to add new columns to the schema dynamically. Example: I am reading a table from a db and one day a client adds one new column in the db-table then I should be able to accommodate the newly added column in the Talend schema dynamically, without changing the code. This schema is propagated to the output file I am generating. How can we achieve this. Please suggest solutions. Thanks!!
This can't be done. If the input or output schema are changed you will need to modify your job to accommodate this. You *could* use a query stored outside of your job to concatenate all columns of your input database into a comma separated String, read this query in during the job, then use it in your input component. If you had a single column for your input component, this would allow you to kind of achieve this output to a file. I suggest this as if you are outputting to a file, you will have to convert all data types to String anyway. Therefore you can create a single job and only have to modify a query stored externally to the job. This will limit what you can do with the data though.
You can still do this with the method I suggested. It would just take some String manipulation inside the job. However, it seems your transformation rules are not consistent. This leads me to ask, how you expect to implement your rules if you do not know of the columns you will receive? I think you need to tell your client that if they change the schema, they will have to change the job.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.