Compare two schemas and data formats

One Star

Compare two schemas and data formats

Hey!
I need to compare schemas of various .csv files (to make sure they have the same no. of columns and rows) and also the data formats (e.g. integer, string or date) in each cell. I want to do this by comparing each schema with a refererence file. Can anyone suggest how to do this?
Thanks!
Community Manager

Re: Compare two schemas and data formats

Hello
Does tFileCompare comonent fit your need?
Best regards
shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Compare two schemas and data formats

No, tFileCompare compares two files with identical data. In my case, the data is different in the files though the schema has to be the same. I mean to say that each file has to have lets say 9 columns, the one with 10 columns has to be skipped and the data type in any cell under the same column should be same.
If I have 10 files in a folder, I want to skip those with errors and continue with the rest so that my job does not crash.
One Star

Re: Compare two schemas and data formats

Hi,
If you define the metadata of CSV file in a CSV file or some where else(outside) then it is possible .
As CSV file does not provide all the metadata, it is impossible to compire the datatype, size of one CSV file to another.
In case you try to get the metadata from data then you may not get correct metadata always from the data available in the CSV file.
Thanks and Regards,
Pravu Mishra.
One Star

Re: Compare two schemas and data formats

Hi!
I want to skip the files having 9 columns or 11 columns and pass the files with 10 columns for further processing.. I am trying to do it by tSchemaComplianceCheck.
I created the test schema in repository and kept the check file schema as built-in. Also , I have kept the base schema fields nullable but test schema fields not nullable. But still it doesnt not filter out the corrupt files.
So can you tell me how to do it?
One Star

Re: Compare two schemas and data formats

You Have Database tool just for that, its called Columbo. (http://www.nobhillsoft.com/Columbo.aspx)
One Star

Re: Compare two schemas and data formats

Did you ever sort this? I would have assumed Talend could do this, but if you want to compare different data formats in text files or otherwise there are tools out there, e.g.: http://www.citrustechnology.com/solutions/data-comparison/data-transform
One Star

Re: Compare two schemas and data formats

Hi guys, is there any tool which is free to do this?

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download