How to Remove Columns with null values

One Star

How to Remove Columns with null values

Hi All,
Currently I am working on a job in Which some columns may contain all null values .
I want to remove those columns with all null values . In this case which component i should use.
please Help me out. Thanks in advance.
Four Stars

Re: How to Remove Columns with null values

Hi Bhanu,
tFilterRow component could be used for filtering the records. Please refer to documentation available at
https://help.talend.com/search/all?query=tFilterRow&content-lang=en
In your case the filter condition would be based on null value.
Vaibhav
Seventeen Stars

Re: How to Remove Columns with null values

I guess you mean columns containing null should not be part of the insert/update statement, e.g. to get the column default value instead of null?
If it is always the same column - the easy - simply use a build-in schema for the output component. If it changing from record to record, you need a bit Java "magic" and the t<DB>Row component. For every record you have to check which column has null value and create the statement new. The Row component should use a separate database connection to avoid connecting every record.
It would be a nice task for a custom component ;-)
One Star

Re: How to Remove Columns with null values

Hi Vaibhav,
I want to remove entire column from output schema if it is having null values in all rows(all null values).
In this job my input schema contains 6 columns but i am having values for first three columns so i am giving null values to last three columns and taking it as input. 
here, I need to delete last three columns which are having null values using my ETL job. 
tFilterRow can  filter row values but it is not deleting entire column from output schema.
Is there any other solution for this problem.
Note: i am not using Enterprise version so i don't have dynamic schema concept in my talend.
Four Stars

Re: How to Remove Columns with null values

Hi Bhanu,
If you are sure that last three columns are to be replaced, then this is quite possible using tFilterColumn component. But in case if you are not sure whether all three columns need to be filtered and may contain some non null values as well, then you will have to apply different logic.
In second scenario, first you will have to read complete data to verify whether particular column contains null values for all the rows. During the flow, mark that respective column and using tjavarow you can remove that column based on if-then-else condition.
Thanks
Vaibhav