Using dynamic schema throws error in tSnowflakeInput component

Five Stars

Using dynamic schema throws error in tSnowflakeInput component

Hi I have a very simple workflow to source data from snowflake and store it out as flat file.  When I use dynamic as the schema type, the job fails with following error code if I explicitly specify the column names in my query.  However, when i just specify 'Select *' it just runs fine even having 'dynamic' as the datatype.  

 

INFO: sql command to get primary key metadata: show /* JDBCSmiley Very HappyatabaseMetaData.getPrimaryKeys() */ primary keys in table "OPENFIT"."RECURLY_STITCH_PROD"."RECURLY_SUBSCRIPTIONS"

 

[FATAL]: bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow - tSnowflakeInput_2 2

java.lang.ArrayIndexOutOfBoundsException: 2

at org.talend.components.common.avro.JDBCResultSetIndexedRecordConverter$ResultSetIndexedRecord.get(JDBCResultSetIndexedRecordConverter.java:111)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.getDynamicValues(DiOutgoingDynamicSchemaEnforcer.java:146)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.get(DiOutgoingDynamicSchemaEnforcer.java:105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeInput_2Process(Openfit_Snowflake_Simple_Flow.java:1105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeConnection_2Process(Openfit_Snowflake_Simple_Flow.java:489)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.runJobInTOS(Openfit_Snowflake_Simple_Flow.java:1644)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.main(Openfit_Snowflake_Simple_Flow.java:1475)

[statistics] disconnected

Exception in component tSnowflakeInput_2 (Openfit_Snowflake_Simple_Flow)

java.lang.ArrayIndexOutOfBoundsException: 2

at org.talend.components.common.avro.JDBCResultSetIndexedRecordConverter$ResultSetIndexedRecord.get(JDBCResultSetIndexedRecordConverter.java:111)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.getDynamicValues(DiOutgoingDynamicSchemaEnforcer.java:146)

at org.talend.daikon.di.DiOutgoingDynamicSchemaEnforcer.get(DiOutgoingDynamicSchemaEnforcer.java:105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeInput_2Process(Openfit_Snowflake_Simple_Flow.java:1105)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.tSnowflakeConnection_2Process(Openfit_Snowflake_Simple_Flow.java:489)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.runJobInTOS(Openfit_Snowflake_Simple_Flow.java:1644)

at bi_data_lake.openfit_snowflake_simple_flow_0_1.Openfit_Snowflake_Simple_Flow.main(Openfit_Snowflake_Simple_Flow.java:1475)

Job Openfit_Snowflake_Simple_Flow ended at 11:24 06/02/2019. [exit code=1]

 

SnowflakeDynamicSchema.png

Moderator

Re: Using dynamic schema throws error in tSnowflakeInput component

Hello,

Could you please clarify in which talend version/edition you are?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Using dynamic schema throws error in tSnowflakeInput component

We are usin Talend Studio 6.4. Please note that this behavior is not consistent for all tables. For a different table in the same Schema, it works fine with dynamic Schema and select clause with explicit column names.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog