Four Stars

tAzureSqlDWBulkExec Not Working with gzip files

tAzureSqlDWBulkExec has been incredibly helpful with my Azure DW migration, but I cannot get the component to work with a gzip file. I am using tFileArchive to create the file in a gzip format. My file is around 35 gb so the compression would be really helpful. This is the error that I am seeing.

 

Exception in component tAzureSqlDWBulkExec_1 
com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format. 
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:845) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:752) 
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151) 
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:729) 

 

I am not seeing any issues with the compression turned off.

 

Here is the link to the component in the Talend Exchange:  https://exchange.talend.com/#marketplaceproductoverview:marketplace=marketplace%252F1&p=marketplace%...

  • Data Integration
  • Exchange components
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: tAzureSqlDWBulkExec Not Working with gzip files

@davidmcelroy21

 

I have updated the connector.  There was an issue with the compression option.  Please check the exchange for version 1.1.

 

I plan to add ADLS support in the coming days with version 2.0.

 

Usman

8 REPLIES
Employee

Re: tAzureSqlDWBulkExec Not Working with gzip files

@davidmcelroy21

 

have you tried with a smaller size gzip file?

 

Usman

Four Stars

Re: tAzureSqlDWBulkExec Not Working with gzip files

Yeah, I added (top 100) to my sql query.  It was not working even with that very small file.

Employee

Re: tAzureSqlDWBulkExec Not Working with gzip files

what is the error you are getting when using the GZIP option?

Four Stars

Re: tAzureSqlDWBulkExec Not Working with gzip files

The error I am getting is:

 

com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format. 

 

I have seen this error before when the file does not exist, but the gzipped file is being placed in azure storage and the file name matches the name of the file in the  tAzureSqlDWBulkExec component.  

@uszubair

Employee

Re: tAzureSqlDWBulkExec Not Working with gzip files

@davidmcelroy21

 

I have updated the connector.  There was an issue with the compression option.  Please check the exchange for version 1.1.

 

I plan to add ADLS support in the coming days with version 2.0.

 

Usman

Four Stars

Re: tAzureSqlDWBulkExec Not Working with gzip files

Fantastic! Thank you!
Four Stars

Re: tAzureSqlDWBulkExec Not Working with gzip files

Do you know when this change will be live?  I have been checking once an hour or so and it is still version 1.0.   I am not sure if there is an approval process with the talend exchange.

Employee

Re: tAzureSqlDWBulkExec Not Working with gzip files

@davidmcelroy21

 

i have submitted it for approval...

 

attaching it for reference as well