Cannot bind 2 schemas to a tFileOutputMSDelimited

Six Stars

Cannot bind 2 schemas to a tFileOutputMSDelimited

Hi,

 

I have a specific problem with a Multi Schema Delimited File.

I receive a Multi Schema file, and i need to make some transformations to the data using tmaps (1 for each schema).

This works fine but the problem is that i cannot bind them to their initial shape in a tFileOutputMSDelimited

 

Here is a picture to illustrate my problem : 

multischema_job.PNG

(in red the bind i can't make)

 

My data is set like following :

 

E;X;Y;Z

P;X;Y;M;T;Z

P;X;Y;M;T;Z

P;X;Y;M;T;Z

E;X;Y;Z

P;X;Y;M;T;Z

P;X;Y;M;T;Z

...

And i need to keep the same order for my output file.

That restriction forbids me to do the following thing :

not_multischema.PNG

Here are my settings for my tFileInputMSDelimited :

settingmsfileinput.PNG

 

If you have any suggestions, it's welcome.

Best Regards,

 

Pierre


Accepted Solutions
Eleven Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

You have to use tHashinput/output component . because 

https://help.talend.com/reader/~V1UYTyXpjBKjP5PCMy4EA/khJeOaSeNMnFt0o~DMx6qQ

 

Attaching an example job how the order can be maintained. Change path accordingly.

 

Hope this is useful.

Regards
Abhishek KUMAR
Six Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

Ok,

 

Thank you for your support, i found an alternative solution to my problem.

 

Multischema_module.PNG

 

I set my sequencer in tJava

Input each lines of the Multi Schéma file

tMap1 : - I build my schemas sorting lines with substring function.

             - I set the sequencer to each outputs to obtain at the end the result : 

1|E|X|Y|Z

1|P|A|B|C|D|E|F

2|E|X|Y|Z

2|P|A|B|C|D|E|F

...

 

I make my transformations to the data at tMap2 tMap3 tMap4

Fetching correctly the datawith tFileOutputMSDelimited with sequencer "Id" as Key Column

 

 

If this helps someone in future.

 

Pierre

 


All Replies
Eleven Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

You have to use tHashinput/output component . because 

https://help.talend.com/reader/~V1UYTyXpjBKjP5PCMy4EA/khJeOaSeNMnFt0o~DMx6qQ

 

Attaching an example job how the order can be maintained. Change path accordingly.

 

Hope this is useful.

Regards
Abhishek KUMAR
Six Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

it does not work, 

i only have the following outputs :

 

E;X1;Y1;Z
E;X1;Y1;Z

 

P schema lines are missing

 

Also, i am not allowed to use temporary memory for performance reasons i guess.

Eleven Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

aah it should work .

It might be because Schema for P type might have been removed .

 

Can you please redefine it and check. TalendImage.JPG

 

I am getting correct output.

 

Instead of Hash Component , you could write these in csv files and read it.

Regards
Abhishek KUMAR
Six Stars

Re: Cannot bind 2 schemas to a tFileOutputMSDelimited

Ok,

 

Thank you for your support, i found an alternative solution to my problem.

 

Multischema_module.PNG

 

I set my sequencer in tJava

Input each lines of the Multi Schéma file

tMap1 : - I build my schemas sorting lines with substring function.

             - I set the sequencer to each outputs to obtain at the end the result : 

1|E|X|Y|Z

1|P|A|B|C|D|E|F

2|E|X|Y|Z

2|P|A|B|C|D|E|F

...

 

I make my transformations to the data at tMap2 tMap3 tMap4

Fetching correctly the datawith tFileOutputMSDelimited with sequencer "Id" as Key Column

 

 

If this helps someone in future.

 

Pierre

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download