How to fetch a file from Box and load it on the DB which is in the Local System????

Nine Stars

How to fetch a file from Box and load it on the DB which is in the Local System????

Hi,

I am bit confused about a scenario - 

i have uploaded a file on BOX and now i would like to download the file and do some operations like updating the file etc.. and then i would like to insert into DB. 

I do have a rough idea like - 

tBOXLIST(Details to be given here) -------->>>>LOCAL_DIRECTORY(FILE TO BE DOWNLOADED)---------->>>>>Transformations Logics(tmap or other components)--------->>>>>>>To be loaded to DB(DB_OUTPUT_Component).

Can anyone help me on this ???

Are there any videos/docs such that i can refer to -> for loading data from cloud to DB..

Pls reach out,if necessary.

Thanks,

Ankit.

Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi Ankit,

 

     Your thought process is right.

   

      You can bring multiple files from Box to local as first stage. Then use tFilelIst and other usual components to process each file and load them to DB.

 

image.png

 

 

The best place to look for examples is always help.talend.com

 

https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/hj2FnNv1btiSRqPm2wWjrQ?section=sect-components...

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

 

 

Nine Stars

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi @nikhilthampi,

Thanks for your reply...

But.. i came across another workaround like using HTTP request.. if i were to use HTTP request then how should i design and are there any precautions i must be aware of.

Thanks,

Ankit

Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi @ankit7359,

 

     Could you please provide more details about the workaround using http mode?

 

Warm Regards,

 

Nikhil Thampi

Nine Stars

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

hi @nikhilthampi,

Thanks for the reply.. 

well i was thinking what if we use tHttprequest component and loading the component with box details like access token and etc..  and then using get method to pull files from the box and then placing those files in remote FTP server and then taking the file and loading to DB.

tHttprequest(BOX-link)------>>>COMPONENT(GET METHOD)*-------->>>>PLACING THE FILES IN THE REMOTE FTP SERVER(TFTPCONNECTION)------->>>>TFTPGET(PULLING THE CERTAIN FILES AS PER FILEMASK)*------>>>>TFILELIST(ITERATING THE FILES AS PER FILEMASK)------>>>>TFILEINPUT(AS PER FILEMASK)------>>>LOADING TO DB...

I am not sure of the above workaround.. but i am tryin to explore what are the other ways it could be achieved and keeping performance parameters aside.

Pls do suggest and give your opinions on the above.

Thanks,

Ankit.

Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi Ankit,

 

      End of day, the Box components of the Talend are also making API calls to move the data between Box and your local server. The method you are using above is also doing the same stuff but you are calling the Box APIs through generic HTTP request components instead of Box native calls shared by Talend.

 

     If you notice, you are making multiple hops for same files in above methodology. So my recommendation would be to use the native Box components of Talend and complete the processing.

 

      Simple, easy and widely used by many Talend customers :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Nine Stars

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

hi @nikhilthampi,

Thanks again for the reply.

I do have a query for this.. for example.. what if we use box component but my admin user says that we cant give the access tokens and other credentials as they hold sensitive data...???

What are your suggestions on this Hypothetical scenario.... the only reason i am asking these many questions is i want to cover all the gaps.. like a workaround solution for the every scenario...

Pls do give your suggestions on this..

Thanks,

Ankit

 

Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi Ankit,

 

    If we have to connect to a service like Box, we need to definitely give the connection parameters shown below.

 

image.png

 

We cannot escape from that option. You need to safely keep the credentials and pass them upon request by Talend job.

 

Seems we are deviating from the original topic :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

 

 

Nine Stars

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

hi @nikhilthampi,

the reason why i asked about using httprequest is because.. not always we get access from admin team and even if we get it then it is limited.. keepin that in mind.. if we were to build this scenario without box componnets how would we do it.. ??

this is the reason, i was asking question which arent straight but around the loop..

Thanks,

Ankit

Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

Hi Ankit,

 

     I believe it is a Box support related query. Even if you are using http request my understanding is that you will still come across the point where you will have to pass the user credentials. In that case also I believe your admin team has to provide the credentials to connect to Box.

 

     But if you still want to create a component of your own, you will have to build them using the Box reference guide shown below. I would suggest to contact Box team to provide the best practices to you in this area.

 

https://developer.box.com/reference

 

      The aim of Talend is to remove the underlying complexities of connecting a component. But developers who are interested in coding part are welcome to create there own components using custom components or by creating routines from their end. In this case, they will have to design and develop the components according to their specific need as Talend do not have any control on the flow.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Nine Stars

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

@Nikhil,
Thanks Again.
I shall get back to you on this.. i have been thinkin on the same... so once i m able to think of something and try it out.. i shall close the topic.. as resolved.. i hope it is okay...
Congrats on retaining the top spot on Top Solutions Author.
Thanks,
Ankit
Employee

Re: How to fetch a file from Box and load it on the DB which is in the Local System????

@ankit7359

 

Hi Ankit,

 

     Could you please mark the topic as resolved since I have answered your original query? This will help other members when they are searching for a similar solution in Talend community.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

Modern Data Engineering in the Cloud

Learn about modern data engineering in the Cloud

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now