[resolved] tXMLMap - catch errors/exceptions

One Star

[resolved] tXMLMap - catch errors/exceptions

Hi guys,
I have a small Talend job that reads CSVs and makes an XML from its rows, then moves the CSVs in an archive folder afterwards.
Im having a problem trying to catch CSVs with errors, the job stops and doesnt continue on the other CSVs afterwards.
How can I catch the errors in the XML mapping and be able to move that certain CSV, then continue with the other CSVs in processing them?
Community Manager

Re: [resolved] tXMLMap - catch errors/exceptions

Hi  
You need to design the job as below:
main job:
tFileList--iterate--trunjob
trunJob: call the child job, pass the current file path to the child job, uncheck the 'die on error' option so that the main job can continue to process the next file even though the child job has an error. 
child job:
tFileInputDelimited--main--tXMLMap--tLogRow
Fefer to this KB article to know how to pass a value from main job to child job. 
Let me know if you have any problems.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Does this mean I will have 2 Talend Jobs now?
Is there another workaround that only involved 1 job? Because this will be built as a standalone job that will be schedule using a script.
Community Manager

Re: [resolved] tXMLMap - catch errors/exceptions

Hi
Strictly speaking, they are not entirely separate jobs, they are not entirely separate jobs, the child job is called by the main job, you just need to export the job script of main job as a standalone job and schedule it. 
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Hello,
Unfortunately my manager/in-charge does not want the job to change that much anymore, he just wants to have a way to continue processing other CSV rows and files after encountering an error in a certain CSV. Would this be possible without altering the job that much?
Can I catch/bypass a row/file error and continue with the other rows and files in the folder?
Community Manager

Re: [resolved] tXMLMap - catch errors/exceptions

Hi
If the error occurs on tFileInputDelimited component, unchecking the 'die on error' option on tFileInputDelimited will allow the job to continue to read other rows and files, but the error messages can't be captured by tLogCatcher component with this way.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

The error is in the tXMLMap, when it attempts to substring invalid/null fields. How can I catch those?
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

There is a way you can trap the errors in the tXMLMap and continue...well, actually you will need to add a tMap before the tXMLMap and put the logic from the tXMLMap into the tMap. This is because for some reason the functionality I demonstrate in this link is not available in the tXMLMap. 
Why is that Shong?
Community Manager

Re: [resolved] tXMLMap - catch errors/exceptions

Hi rhall_2.0
I agree with you that tMap can be used to trace the error and continue to process the next rows if the error is not an I/O error occurred on input/output component. 
----------------------------------------------------------
Talend | Data Agility for Modern Business
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

Why is that functionality not available in the tXMLMap? 
Community Manager

Re: [resolved] tXMLMap - catch errors/exceptions

A new feature has been reported in bugtracker, see 
https://jira.talendforge.org/browse/TDI-18968
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

I see, so my job layout should now be . . .
Before:
tFileList --(iterate)-- iFileInputDelimited --(row)-- tXMLMap
After:
tFileList --(iterate)-- iFileInputDelimited --(row)-- tMap --(row)-- tXMLMap
Where tMap now has the formatting codes and tXMLMap is merely assigning the fields to their respective XML elements, is that correct?
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

Yes, that is the way that I would attempt this to make use of the tMap error logging functionality.
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Okay thanks for the suggestion, will try this out and update this topic accordingly.
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Hello,
I am now able to continue after encountering a row error. Thanks rhall_2.0 and shong.
Unfortunately if consecutive errors are shown as 1 when I try to print something in the ErrorReject row.
Also, it ignores the first okay row after encountering an error row/group of error rows.
For example, this is my csv file:
- headers
- ok row
- ok row
- ok row
- error row
- error row
- ok row
- ok row
- ok row
- error row
- error row
- error row
- ok row
- ok row
It prints the following:
- ok
- ok
- ok
- error (should be 2 times)
- ok
- ok (should be 3 times together with previous print)
- error (should be 3 times)
- ok (should be 2 times)
How can I make it print "error" for each instance of the error row, and not as a group of consecutive error rows?
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

This is very odd. Can you take a screenshot of your job, your tMap configuration and the following tXMLMap config?
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

I find it very weird also, I was expecting it to correctly print according to the ok/error row.
Anyways here is my job layout:

My tMap component has the parsing and formatting conditions, with the die on error unticked:

And my tXMLMap is a straight auto-map into the XML structure:

Is there some hidden or inconspicuous setting that I might have missed or is this some Talend bug?
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

Where is the code written in the tJavaFlex component logging the errors? 
In your tXMLMap output table settings (click the spanner symbol), do you have "All in one" selected? This allows one XML file to be written containing several loops as opposed to a separate XML for every row.
Maybe try connecting you tMap outputs to tLogRow components (instead of the tJavaFlex and tXMLMap) and and see what comes out from there. You should be seeing something for every row.
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Where is the code written in the tJavaFlex component logging the errors? 
It is in the tJavaFlex_7, the one connected with ErrorReject row from the tMap
In your tXMLMap output table settings (click the spanner symbol), do you have "All in one" selected? This allows one XML file to be written containing several loops as opposed to a separate XML for every row.
I have set "All in one" into false because I have to generate 1 XML per row read from the CSV.
Maybe try connecting you tMap outputs to tLogRow components (instead of the tJavaFlex and tXMLMap) and and see what comes out from there. You should be seeing something for every row.
Only 1 instance of tLogRow output gets printed instead of the 2 or 3 consecutive for the error rows. Same goes for the ok rows.
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

ADDITIONAL:
My job also shows that it processed 8 rows, with 6 ok and 2 error when instead it should be 13 rows with 8 ok and 5 errors. No limit has been set.
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

Thanks for your answers. The question about the tJavaFlex was actually meant to be which section in the component (Start, Main, End). But your problem doesn't look like it is there now. You are getting 8 rows from your input file and that matches the output you are getting. We need to check the config of the tFileInputDelimited. The rest of the job looks like it is behaving correctly.
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

It might also be useful to see the contents of the file (if you can post that).
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

We need to check the config of the tFileInputDelimited.
Here it is:

It might also be useful to see the contents of the file (if you can post that).
Unfortunately this is a confidential XML from out client, but I can say that all the error rows are the same row copied over (for test purposes), I guess thats whats causing it to group them as one. Ill try to simulate unique error rows by tomorrow, the day is about to end in our time zone, glad you mentioned about it, hopefully this will get it working right.
Sixteen Stars

Re: [resolved] tXMLMap - catch errors/exceptions

My thoughts on possible causes are....
1) The CSV input file does not always use a line feed to separate rows. Could it be "\r\n"
2) The CSV does not have a header maybe?
I suspect  the rows with the wrong row separator are likely the ones that you have added as errors. The tFileInputDelimited will not filter these because they are the same. It will just return rows according to the settings and data in the file.
One Star

Re: [resolved] tXMLMap - catch errors/exceptions

Client is currently testing the application, hopefully things go well considering possible errors would be within the data and unique from each other.
Thanks so much for the help guys. Much appreciated.