Query s3 data using Redshift Spectrum

Highlighted
Six Stars

Query s3 data using Redshift Spectrum

Hello Talendians,

 

I have a use case where i have to query directly from S3 using Redshift Spectrum. The data stored in s3 is in parquet format. We already have external schema and tables created using Glue Catalog. What should be the design approach? I see Talend does not have any Spectrum Components. Only RedshiftBulkexec and redshiftoutput components. Can someone throw somelight on how we can achieve this in Talend? 


Accepted Solutions
Employee

Re: Query s3 data using Redshift Spectrum

Hi,

 

    I have not personally tried connecting Redshift Spectrum using Talend Redshift components. Are you getting any error while connecting using them? As a fallback strategy you can check generic JDBC DB components also to connect the Spectrum.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

View solution in original post


All Replies
Employee

Re: Query s3 data using Redshift Spectrum

Hi,

 

    I have not personally tried connecting Redshift Spectrum using Talend Redshift components. Are you getting any error while connecting using them? As a fallback strategy you can check generic JDBC DB components also to connect the Spectrum.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

View solution in original post

Six Stars

Re: Query s3 data using Redshift Spectrum

Hi Nikhil,

Thanks for the reply

I don't think there are any Spectrum Components in Talend. Like you said i can use generic Database connection to connect to spectrum. I was trying to do a proof of concept to query data from s3 using Redshift spectrum and was wondering how do it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

How to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud reg...

Learn how to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud registries

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog