error in exported job(convert csv to tde) run in loop

One Star

error in exported job(convert csv to tde) run in loop

Hi,
I have created and exported a job a while back and it does convert csv files to tde. I have 900 csv files and they were separated into folders. I have a main folder and inside it is folder1 having the first 100 csv files then folder2 having the second 100 files..and so on. I have bat file that will call this exported job looping through all folders in Main folder. It was like below

echo off
set PATH=E:\TALEND_64Bit\TOS_DI-r99271-V5.2.2\;%PATH%

echo "START TIME: %time%"

REM Loop through all stores and Run TDE Conversion
echo "%errorlevel%"
cd /D %~dp0

REM 373 Days Daily CSV to TDE
for /D /r E:\Main\Store %%a in (*) do ^
start E:\Main\ScheduleJobs\TDEConversion_373DaysDaily_0.1\TDEConversion_373DaysDaily\TDEConversion_373DaysDaily_run.bat %%a%
cd /D %~dp0
wait
type nul > E:\Main\Store\TDEStore_Final373D.tde

echo "FINISH TIME: %time%"


My job consists of tfilelist > iterate > tfiledelimited >tmap>tTableauExtractOutput
In tfilelist I have checked global expression as filemask and filemask ="*.csv"
Now I needed to re-export the job...however it is not producing tde files as expected. What should i do..I need help here really badly
Thanks everyone!

Ten Stars

Re: error in exported job(convert csv to tde) run in loop

Why are doing the looping over folders in the bat file? You can do this within the Talend job using a tFileList component.  

What is the purpose of this line in your bat file...

type nul > E:\Main\Store\TDEStore_Final373D.tde

...it looks like it is creating a 0 byte file called TDEStore_Final373D.tde. Why is this? All files should be created, written to and deleted within the Talend job.
With that in mind, maybe you can take a screenshot of your job and post it. So far all we can really do is comment on the bat file.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog