[resolved] auto detect column names in delimted file, based on a header row?

One Star

[resolved] auto detect column names in delimted file, based on a header row?

Is there a way to auto fill the schema based on a header row?
and, if so, can the data type be auto detected as well? or could a default data type be set?
first row is the example header row
1|2|3|4|5|6|7|8|9
A|B|C|D|E|F|G|H|I
J|K|L|M|N|O|P|Q|R
S|T|U|V|W|X|Y|Z|AA
Tags (1)

Accepted Solutions
Employee

Re: [resolved] auto detect column names in delimted file, based on a header row?

With Perl, use "Custom ANSI" '\|' as delimiter. This problem comes back from time to time (see 1436) but it should be better now (TOS 2.3.x) because the properties are transmitted "as is" from the metadata to the component.
What if I want the columns to be set up dynamically every time the job is run?

In most cases, you can't. Job design is strongly related to the schema so the schema is declared during design, not during generation or execution time.
Can you give an example of two files with distinct schemas you would like to manage with the same job? (in some cases, it can work)
tips:
- if your topic is related to perl, use the "perl" tag (same for java)
- if you want to give a link to another topic, write "topic: 12345" (without the space) it will be automatically converted to a beautiful link with the topic title . It also works for "bug:" "feature:"

All Replies
One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

you can use the metadata manager, and check set heading row as column name Smiley Happy
One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

What if I want the columns to be set up dynamically every time the job is run? (There is the distinct possibility that the columns in this file will change.)

Ok, my delimiter (in the file I'm currently looking at) is |, but apparently I'm doing something wrong in specifying my metadata settings, because its currently making each individual character a separate column (see 1st pic).
As per this 2402 I tried changing "Field Seperator" to Custom ANSI and the Custom ANSI field to "|", but had the same issue.

Is there specific documentation on individual components that I could be directed to? I've tried looking at the "Talend Open Studio User's Guide Version 2.3_b" but haven't found it be in depth enough to be helpful.
Employee

Re: [resolved] auto detect column names in delimted file, based on a header row?

You can have a look at the Component Reference Guide (RG) in the documentation section
HTH,

One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

"\\|" generates an interesting, different, result (see attached pic)
One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

ok, all the screen shots I've posted so far were in perl.
I realized this and switched to a java project and tried to enter the same information... However, in java:
1. I can't get it to show me a preview
2. Clicking the "next" button doesn't do anything
3. The "stop" button doesn't switch to a "Refresh Preview" button, it just goes inactive

This is a <100 row file, by the way.
Employee

Re: [resolved] auto detect column names in delimted file, based on a header row?

With Perl, use "Custom ANSI" '\|' as delimiter. This problem comes back from time to time (see 1436) but it should be better now (TOS 2.3.x) because the properties are transmitted "as is" from the metadata to the component.
What if I want the columns to be set up dynamically every time the job is run?

In most cases, you can't. Job design is strongly related to the schema so the schema is declared during design, not during generation or execution time.
Can you give an example of two files with distinct schemas you would like to manage with the same job? (in some cases, it can work)
tips:
- if your topic is related to perl, use the "perl" tag (same for java)
- if you want to give a link to another topic, write "topic: 12345" (without the space) it will be automatically converted to a beautiful link with the topic title . It also works for "bug:" "feature:"
One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

what about for Java? my metadata is working for Java now, so thats what I'm trying to work in.
"|" seems to work for Java, except that the preview is only showing the first column (there are like 50 columns)... it only seems to be processing to the first instance of the delimiter.

sorry for switching back and forth on yall, I didn't realize I was in a perl job when I first started trying to work with metadata today.
One Star

Re: [resolved] auto detect column names in delimted file, based on a header row?

what about for Java? my metadata is working for Java now, so thats what I'm trying to work in.
"|" seems to work for Java, except that the preview is only showing the first column (there are like 50 columns)... it only seems to be processing to the first instance of the delimiter.

I'm still encountering this problem