Avoid multiple header rows?

Four Stars

Avoid multiple header rows?

I have a couple of CSV files that I load into Data Prep. All at once (I only specify a directory in "Add Dataset", no individual files). So far, so good.

 

All files have the same structure, the first line is the header.

 

 

Is there a way to globally set the first row as header for all files? I know there is this "Row" -> "Make as header..." feature, but what happens in my case is:

 

file1.csv:

Firstname;Lastname;Age

Felix;Kjellberg;23

Julian;Ilett;43

 

file2.csv:

Firstname;Lastname;Age

Ben;Heck;58

Dave;Jones;48

 

The result in Data Prep is:

Firstname|Lastname|Age

Ben Heck 58

Dave Jones 48

Firstname Lastname Age

Felix Kjellberg 23

Julian Ilett 43

 

So even if I set the blue line as header, the green line will stay. Is there a way to avoid this?

 

Moderator

Re: Avoid multiple header rows?

Hi,

 

Out of curiosity, can you confirm the following?

  • You are using Data Prep 2.0.
  • The CSV files are on HDFS.

 

To answer your question: there is no dedicated data set parameter or function to remove subsequent occurrences of the header but you can do it in a single preparation step: set a filter on the first column with the column header as filter value (so filter on "Firstname" in your example below) and use the function "delete filtered rows".

 

Regards,

 

Gwendal

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

APIs for Dummies

View this on-demand webinar about APIs....

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now