Data Validation with Hierarchical Mapper

Eight Stars

Data Validation with Hierarchical Mapper

I have several million EDI records that I'm mapping to flat files, and I'm having trouble getting the data validation in the Hierarchical Mapper to work. We are only interested in records that have the RecordType segment = P (Production), so I created a rule in the Validate tab of the Mapper saying that RecordType must equal 30. When I run the job, it flags any invalid records, but it still processes them and includes them in the output, which isn't what I want. I've tried using the ValidateGroup and ValidationReport functions, but the documentation on them isn't clear, and my job either didn't compile, or it didn't make any difference. How can I exclude invalid records so that the Mapper doesn't include them in its output? Thanks.


Accepted Solutions
Nine Stars

Re: Data Validation with Hierarchical Mapper

Hi,

 

Why don't you just filter at loop level ?

 

Could you upload your map and a test input file ?

 

Eric

View solution in original post


All Replies
Nine Stars

Re: Data Validation with Hierarchical Mapper

Hi,

 

Why don't you just filter at loop level ?

 

Could you upload your map and a test input file ?

 

Eric

View solution in original post

Eight Stars

Re: Data Validation with Hierarchical Mapper

I didn't think about using a filter, but that worked. Thanks!

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog