One Star

Pivot Values to Columns

i have a structure
id | name | value
1 | name1 | value1
1 | name2 | value2
2 | name1 | value1
2 | name2 | value2
and I need it in the form
id | name1 | name2
1 | value1 | value2
2 | value1 | value2
the number of name columns to pivot is dynamic.
i have tried using the tPivotToColumnsDelimited component. it is way too slow on a large set, approx 300-800 rec / sec. processing 500k rows is rediculous. anyone have an alternative? using multiple splits/denormalizing would be great if possible. I am still trying to figure out the dynamic column routine somehow.
One Star

Re: Pivot Values to Columns

The slow performance is because tPivotToColumnsDelimited needs to load 500K first and calculate the number of columns then.
Is it possible to split 500k files into several files in terms of your logic?
Or I don't think a workaround exists.