[resolved] tJavaFlex output issue

One Star

[resolved] tJavaFlex output issue

Hi Everyone,

I'm having problem with the component tJavaFlex. I'm using it to create a row for each day between two date as you can see in the printscreen bellow.



It's working well, except when I want to insert the result in my DB, because I want to drop and create the table before the new insert. I did put this Action on table parameter, on my tMSSqlOutput, but because of the tJavaFlex loop, it drop and create a table each time a new loop is finish, ans I only get the last result in my DB.

Does anyone know how change the output flow of tJKavaFlex in only one flow ?


Do not hesitate to tell me if my explanation isn't clear enough,

Thank's in advance,

Julien 


Accepted Solutions
Ten Stars

Re: [resolved] tJavaFlex output issue

OK. It is the tFlowToIterate that is causing the issue.

For every row in the flow that hits the tFlowToIterate, it iterates the rest of the subjob per row. For every iteration, the tMSSqlOutput is restarted which is causing your issue. This is actually quite useful functionality in some situations, but not in this one. What you can do to get round this is to output your data from the tJavaFlex to a tHashOutput and use a tHashInput in a new subjob to read the data in and pass it to the DB. 

An example of how to do this is here: https://help.talend.com/pages/viewpage.action?pageId=8123271#technical-thashinput-scenario_speed

All Replies
Ten Stars

Re: [resolved] tJavaFlex output issue

Can you show us your job structure please? My immediate thought is that if you are connecting using a flow, this shouldn't happen. If you show us your job structure, it will be easier to see what might be happening.
One Star

Re: [resolved] tJavaFlex output issue

Hi Rhall,



Here is my job structrure,

Thank's for your help,

Julien
Ten Stars

Re: [resolved] tJavaFlex output issue

OK. It is the tFlowToIterate that is causing the issue.

For every row in the flow that hits the tFlowToIterate, it iterates the rest of the subjob per row. For every iteration, the tMSSqlOutput is restarted which is causing your issue. This is actually quite useful functionality in some situations, but not in this one. What you can do to get round this is to output your data from the tJavaFlex to a tHashOutput and use a tHashInput in a new subjob to read the data in and pass it to the DB. 

An example of how to do this is here: https://help.talend.com/pages/viewpage.action?pageId=8123271#technical-thashinput-scenario_speed
One Star

Re: [resolved] tJavaFlex output issue

Thank's a lot Rhall, this solution is great !!

Could you please explain me, for my general knowledge, why it's so faster than updating the table each time a loop is finish ?
Update takes more than 300s and tHash solution takes less than 15s.

Julien
Ten Stars

Re: [resolved] tJavaFlex output issue

I suspect you are seeing a difference in performance because previously you were dropping and creating a table for every row from your D_SEJOUR_MVT query. Now you are creating your dataset and inserting it after dropping and creating your table once.
One Star

Re: [resolved] tJavaFlex output issue

That makes sense.

Thank's again for your help !!

Have a good day,

Julien

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

APIs for Dummies

View this on-demand webinar about APIs....

Watch Now