One Star

reject output of tschemacompliance not working - TOS 6

Hi
Here is my job
http://prntscr.com/9k7q67
Reading a file having records 26586, one record is broken i.e date format is wrong as required.
26585 processed properly and 1 rejected but couldn't show up in output like tlogrow.
Any Help ?
14 REPLIES
Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi,
From your screenshot, there are only 26585 rows loaded from tfileinputdelimited.
Could you please try to drao "Rejects" from tfileinputdelimited to see if there is any rejected row?(check out the "Trim" option to trim the leading and trailing white space in tfileinputdelimited).
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: reject output of tschemacompliance not working - TOS 6

Hi xdshi
Yes I get the rejected row when I link tlogrow to reject of tfileinputdelimited.
But I'm asking, Why am not able to get rejected records from reject output of tschemacompliance ?
Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi,
How did you set your tschemacompliance component? What does the expected result look like?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: reject output of tschemacompliance not working - TOS 6

I'm having the kind of same issue !!
One Star

Re: reject output of tschemacompliance not working - TOS 6

Hi
In tschemacompliancecheck, I have select custom defined option. All columns are in string type except the date. The date format I have mentioned there is "dd/MM/yyyy" and its a required date format.
In tfileinputdelimited, I'm reading all columns as string again except the date format and I have given the date pattern "dd/MM/yyyy" with date data type.
I know I shouldn't give date type to date column in tfileinputdelimited according talend doc, but if I put date as a string as well and put tschema same as I have mentioned above, its rejecting not even a single record.
https://help.talend.com/search/all?query=tSchemaComplianceCheck
And yes I'm able to get get rejected records from tfileinputdelimited from reject out of that. But My concern is why not from tscehmacompliance if it has reject option in output ?
See here is my job
http://prntscr.com/9kogft
In result you can, there is one wrong record I was talking about having wrong date format. But that didn't flow from tschemcompliancecheck's reject output.
Help !!
Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi,
There is unparseable date issue in your console. The one wrong record was rejected directly by tfileinputdelimited component because you set a wrong date pattern "dd/MM/yyyy"(2015-11-09, yyyy-MM-dd) in schema setting.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi ashish123,
What does your job design look like? More information will be preferred.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: reject output of tschemacompliance not working - TOS 6

Hi
Yes exactly, there is one wrong date pattern present in file. But instead of error, that record should flow from the reject output of tschemacompliancecheck component.
Isn't it ?
Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi,
Yes exactly, there is one wrong date pattern present in file. But instead of error, that record should flow from the reject output of tschemacompliancecheck component.

This record has been rejected by tfileinputdelimited component which means it can't be loaded into tschemacompliancecheck component.
You should make the record(2015-11-09) pass tfileinputdelimited component firstly without any error.
Have you tried to set it as "String" data type in tfileinputdelimited to see if tschemacompliancecheck component works?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: reject output of tschemacompliance not working - TOS 6

Hi xdshi
I have tried with string as well. But When I use string for date column, tschemacompliancecheck rejects all 26586 records.
Can you see what's the matter ?
One Star

Re: reject output of tschemacompliance not working - TOS 6

Moderator

Re: reject output of tschemacompliance not working - TOS 6

Hi,
Could you please also show us your tSchemaCompliance component setting?
We made a simple testing on this component, everything is going well on our side.
For example,
My input:
sabrina;L;05/11/2016
dan;M;2016/11/04
I want to get the rejected input row which is against date pattern "MM/dd/yyyy".
Please see my screenshots for details
Best regards
Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: reject output of tschemacompliance not working - TOS 6

Hi  xdshi,
The tSchemaCompliance component setting u showed even i tried it is working fine .It works with ouput as tlogrow .But it fails in case where output is DB .my scenario is i want to upload a vaild data into table from CSV while rejecting invalid data parking separtely in file or table for error.
Attaching my component settings .With this the records with invalid DATE  also gets uploaded which actually should be rejected.
Seven Stars

Re: reject output of tschemacompliance not working - TOS 6

Hi @xdshi,

I am facing an issue with the Incorrect input error which is not flown to LogRow or an output file.

Error is displayed in console.

My aim is to find out the specific column name which has this issue.

Pls help

 

Thanks

Vidya