Ten Stars

Talend ESB swapped request/responce

Intersting - anybody have similar problem, and if - how to handle them?

Talend OpenStudio ESB (6.4)
all work fine, till today morning:

 

Screen Shot 2017-07-05 at 10.20.43 AM.png

 

looking like normal situation - 1 request pass, 1 fail ... but !!! really to target system transferred data from message 105, and not from 104
yes, both message happens at the same time, from same ip (automation process) - but it must be often situation with real loading
logs from other side confirm same wrong information - 104 successes, 105 - failed , 

Job is standard:

  • receive parameter
  • Iterate - add to target database
  • return response 
  • no any other errors or warnings in log files

 

Screen Shot 2017-07-05 at 10.57.51 AM.png

 

Would be thankful for any suggestions and advices 

 

 

 

regards, Vlad

 

-----------
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: Talend ESB swapped request/responce

Thanks.  Still it is not the full end to end job in the screenshots.  

 

You have quite a few tDie components in your rest service.   You should not use tDie in a service.  tDie are for Batch DI jobs, not for services.  Hence, don't use tDie please.  You need to track the occurrence of an error and send either a tRestResponse or tRestFault.  No tDie at all.

 

Generally I recommend to build the rest response and hold it in memory till the last thing you do in your service and then you send it out through the tRestResponse.   If you send the tRestResponse first, and your service still has more "processing" or "work" to do, it can be difficult to debug or trace.  

 

Also, the only way to get to tRestResponse_2 is if you go down the path of OnSubjobOk only.   What if you get a error and you go through OnSubjobError, then you do not send a response because you never reach the tRestResponse_2.  This is bad for a service.  For every single call, there must be a complete path to a response and a valid response either saying an error occured or with successful result.  Hence, you need to think it this way.

 

Generally, I tend to use the logic shown in the example screenshot attached.  Think differently from DI Batch ETL for services.

 

Of course the screenshot is an example I have whipped up in 5 minutes to illustrate the idea.  You need to add more logic to it to check your status for error or continue processing etc.  But if you do it right, you can avoid these edge cases you are having which are hard to debug and reproduce.  Better do defensive logic coding when it comes to web services.

 

 

 

 

 

 

 

9 REPLIES
Employee

Re: Talend ESB swapped request/responce

Can you upload the full picture of the job?  It is truncated below.

Ten Stars

Re: Talend ESB swapped request/responce

I can of course, just it not help nobody

Screen Shot 2017-07-05 at 12.46.56 PM.png

 

insert, than commit and return status


Important this:

--------------------------------------
2017-07-05T08:56:59,686 | INFO  | qtp1959734321-97 | eptor.AbstractLoggingInterceptor  273 | 32 - org.apache.cxf.cxf-core - 3.1.10 | Inbound Message
----------------------------
ID: 104
Address: http://1xx.xx.xx.xx:8040/services/v1/TalendFlowClientImport?AccountID=xxxxxxxxx78
Http-Method: GET
Content-Type: 
Headers: {accept-encoding=[gzip, deflate], Content-Type=[null], Host=[1xx.xx.xx.xx:8040], User-Agent=[HTTP_Request2/2.3.0 (http://pear.php.net/package/http_request2) PHP/5.5.9-1ubuntu4.14]}
--------------------------------------
2017-07-05T08:57:00,929 | INFO  | qtp1959734321-97 | eptor.AbstractLoggingInterceptor  273 | 32 - org.apache.cxf.cxf-core - 3.1.10 | Outbound Message
---------------------------
ID: 104
Response-Code: 200
Content-Type: application/json
Headers: {Content-Type=[application/json], Date=[Tue, 04 Jul 2017 20:57:00 GMT]}
Payload: {"message":"Account - xxxxxxxxx78 updated","status":"OK"}


and

--------------------------------------
2017-07-05T08:56:59,687 | INFO  | qtp1959734321-4942 | eptor.AbstractLoggingInterceptor  273 | 32 - org.apache.cxf.cxf-core - 3.1.10 | Inbound Message
----------------------------
ID: 105
Address: http://1xx.xx.xx.xx:8040/services/v1/TalendFlowClientImport?AccountID=xxxxxxxx77
Http-Method: GET
Content-Type: 
Headers: {accept-encoding=[gzip, deflate], Content-Type=[null], Host=[1xx.xx.xx.xx:8040], User-Agent=[HTTP_Request2/2.3.0 (http://pear.php.net/package/http_request2) PHP/5.5.9-1ubuntu4.14]}


--------------------------------------
2017-07-05T08:57:01,416 | INFO  | qtp1959734321-4942 | eptor.AbstractLoggingInterceptor  273 | 32 - org.apache.cxf.cxf-core - 3.1.10 | Outbound Message
---------------------------
ID: 105
Response-Code: 500
Content-Type: application/json
Headers: {Date=[Tue, 04 Jul 2017 20:57:01 GMT], Content-Length=[0]}


as You can see - AccountID same in INBOUND and OUTBOUND message with same ID, but

xxxxxxxx77 for ID 105 - in database
xxxxxxxx78 for ID 104 - not

 

-----------
Ten Stars

Re: Talend ESB swapped request/responce

jut for understanding - I create loading test with SOAP UI for total more than 5 000 iterations in 10 parallel threads ... and no one errors!!!

so, I can not reproduce this for understand - what it was?!!!
and for this reason - I ask may be somebody other meet with similar problem?

-----------
Employee

Re: Talend ESB swapped request/responce

Thanks.  Still it is not the full end to end job in the screenshots.  

 

You have quite a few tDie components in your rest service.   You should not use tDie in a service.  tDie are for Batch DI jobs, not for services.  Hence, don't use tDie please.  You need to track the occurrence of an error and send either a tRestResponse or tRestFault.  No tDie at all.

 

Generally I recommend to build the rest response and hold it in memory till the last thing you do in your service and then you send it out through the tRestResponse.   If you send the tRestResponse first, and your service still has more "processing" or "work" to do, it can be difficult to debug or trace.  

 

Also, the only way to get to tRestResponse_2 is if you go down the path of OnSubjobOk only.   What if you get a error and you go through OnSubjobError, then you do not send a response because you never reach the tRestResponse_2.  This is bad for a service.  For every single call, there must be a complete path to a response and a valid response either saying an error occured or with successful result.  Hence, you need to think it this way.

 

Generally, I tend to use the logic shown in the example screenshot attached.  Think differently from DI Batch ETL for services.

 

Of course the screenshot is an example I have whipped up in 5 minutes to illustrate the idea.  You need to add more logic to it to check your status for error or continue processing etc.  But if you do it right, you can avoid these edge cases you are having which are hard to debug and reproduce.  Better do defensive logic coding when it comes to web services.

 

 

 

 

 

 

 

Ten Stars

Re: Talend ESB swapped request/responce

Look,

 

think tDie - not have relation to this case at all

 

The question is - "Why confirmed record not in database, but record with error 500 in?"

 

of course I use prepared variable - based on parameter from tRESTRequest - it used for select data and for response text

 

but why with internal error it take variables from other thread? 


I make cross check - database logs show both thread was connected as separate connections (unfortunately on SQL Server no more information at this moment)

-----------
Highlighted
Ten Stars

Re: Talend ESB swapped request/responce

Screen Shot 2017-07-05 at 1.41.24 PM.png

as You can see - tDie was not called, because before tDie - it must:

 

  • rollback transactions
  • send response with error code and Clean message
  • no one of this message not registered 

 

and again !!! - when it confirmed:

  • it first commit
  • and on success commit - send message Smiley Happy

Screen Shot 2017-07-05 at 1.45.03 PM.png

 

and this committed message - not in database!!!

 

ideas with single response - is interesting, I will change Job, but it not answer for question - "where the successful records?" Smiley Happy

-----------
Ten Stars

Re: Talend ESB swapped request/responce

Thank You for discussion and Your ideas! will test more

at leat one error I saw - again not answer about confirmed records, but at least it wrong and could prevent proper message

 

Screen Shot 2017-07-05 at 1.58.46 PM.png

 

after rollback must be on SubjobOk!!! - it could be reason for wrong Commit, will redesign it to single response 

-----------
Employee

Re: Talend ESB swapped request/responce

I can't say unless we debug the service in detail. 

 

The tRestRequest and tRestResponse are special components because they change the way the code is generated for threadpool for services as well.  Hence sometimes we get these edge cases, where it is hard to reproduce and debug because maybe there is more than 1 tRestResponse in the job.  Your responses are getting mixed basically.  It is sending the response for the wrong request because it is not a first in first out queue. It is happening in parallel.  And sometimes it can get confusing if 1 call did not get a response because of some path that did not reach a response.  Then it appears inverted or mixed up.  It is a hard problem to debug.

 

I like to keep the path to a tRestResponse to 1, or at most 2, so that I can easily debug it.

Employee

Re: Talend ESB swapped request/responce

Yep, that could be the case or not.  Need to debug it to see.