dealing with csv delimiter like ,, and ""

Highlighted
Six Stars

dealing with csv delimiter like ,, and ""

Hi Team, 

I have csv file like 

col_1, col_2, col_3, col_4, col_5, col_6

abc,,,,,xyz

mno," "," "," ","0","0",abc

 

In Talend TOS, while reading this csv file, HOW CAN DO FOLLOWING 

1. convert the cols where there is no value like ,,, to -> NULL and insert to DB

2. convert the cols where there is only qoutes like "","","" to -> NULL and insert to DB

 

Please suggest 


Accepted Solutions
Sixteen Stars TRF
Sixteen Stars

Re: dealing with csv delimiter like ,, and ""

"". equals(row1.myField) || "\" \"".equals(row1.myField) ? null : row1.myField

TRF

View solution in original post


All Replies
Nine Stars

Re: dealing with csv delimiter like ,, and ""

Prepare your job flow like this - 

tFileInputDelimited- > main - > tMap - > main -> tDbOutput (the respective db output component as per your use-case)

 

In tMap you need to search for replace the values for each of your input columns, like this - 

row42.col_2.isEmpty()? null : row42.col_2 
(row42.col_3.startsWith("\"") && row42.col_3.endsWith("\"")) ? null : row42.col_3 

the above code is simply looking for any blank string (no value) and making it null. 
The next one is searching for quotes ("") and making it null - to be double sure you could also check for the length (length = 2) in case if any input column has double quotes to start and end with.

Also in your tMap output do remember to make your output columns as nullable.

Sixteen Stars TRF
Sixteen Stars

Re: dealing with csv delimiter like ,, and ""

"". equals(row1.myField) || "\" \"".equals(row1.myField) ? null : row1.myField

TRF

View solution in original post

Six Stars

Re: dealing with csv delimiter like ,, and ""

thanks will check ... 

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog