Suggestion for set components like UNION, UNION ALL, INTERSECT, MINUS

One Star

Suggestion for set components like UNION, UNION ALL, INTERSECT, MINUS

I would like to have components for set operations they can be called like SET FUNCTION and all set operation supported components should be placed there. Like UNION, UNION ALL, INTERSECT, MINUS which are analogues to arithmetic set operations.
Please share your views for my suggestion.
Community Manager

Re: Suggestion for set components like UNION, UNION ALL, INTERSECT, MINUS

Hi Vinod
Thanks for your suggestion! For these kind of operations, all can be implemented by doing a join on tMap. From our perspective, we do not expect a component too much processing logic, we have to consider this component can be used with other components used together, but not alone. At the same time, we consider the components is not repeated with the existing components or similar, tMap is sufficient for these kind of operations.
Talend | Data Agility for Modern Business
One Star

Re: Suggestion for set components like UNION, UNION ALL, INTERSECT, MINUS

Hi Shong,
I understood your point of view.
Now let us consider some scenarios
1) Date processing parsing and processing - Can be done with tJavaRow, tJavaFlex, tMap, tExtractRegexFields in combination of some other components.
2) Join - Can be performed with tMap, tJoin (at least Inner and Outer joins).
Like above we can have some other problems which can be solved using different approaches, still we are not dropping these components. As tExtractRegexFields could be used for many things but logically it has specifically designed to take care of the problems which deals with regular expression.
I hope due to that we have categorized our component and have named them accordingly.
Same way there is no component in specific that deals with the SET OPERATIONS, however though I have not tested anything but I believe those operations can still be performed with the combination of existing components after some thinking.
In addition to that SET OPERATIONS are quite common in real world, so I believe it is better to have a component instead of generating certain code with combination of some components, all the time whenever we require that logic.
Due to its nature solution to the problem would always look similar to all the user however they all have to spend time and re invent the wheel.
Thanks for listening.

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables


How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration


Downloads and Trials

Test drive Talend's enterprise products.