How to stop processing file or switch mid job

Seven Stars

How to stop processing file or switch mid job

Using open studio 6.4 data integration is it possible to stop processing a file midway through a job if certain conditions are met? I need to re-design a talend job which attempts in ingest excel files but where the structure of the data can change between files.

 

At the moment my job attempts to recover from these changes by re-reading the file in an attempt to match the ingested file against another schema but this way seems to end with me repeatedly executing components x times where x is the number of rows incorrectly matching the schema.

 

I am ingesting my data as strings but I'm finding it difficult to understand the best way of handling working with data where the data within a file is consistent when compared to a schema but not necessarily consistent across files

Community Manager

Re: How to stop processing file or switch mid job

This might help you figure out a way to identify/handle the switching schema

https://www.rilhia.com/tutorials/dynamic-column-order

Seven Stars

Re: How to stop processing file or switch mid job

The problem is I can't be sure the column names would stay the same, some reports have more columns than others, a column may disappear for a few months then return back 

 

Is there a way to break out of/away from processing a particular flow and start another?

Seven Stars

Re: How to stop processing file or switch mid job

Hi,

In tjavarow component, you can check the condition and if condition occurs, you can exit there only using System.exit statement. your job will fail.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Downloads and Trials

Test drive Talend's enterprise products.

Downloads