Hi , Thanks for your interest in talend. Here is a reference about Talend Data Integration Studio functional architecture. Hope it will helpful for you. In addition, we have promoted Talend Help Center to community guests which provides a structured source of enriched information and knowledge, certified by Talend, where you can quickly and easily find out how to make best use of the full range of Talend products. Best regards Sabrina
-- Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Hi Sabrina, Thanks for your reply! I went through above links, but it gives me the information about different blocks while a user in working through the web, how he use to send the request to server and the role of Execution server,Admin & metadata,etc. But actually am looking for some other information. I am concerned about few points as in how Talend use to control the data flow at architectural level. Like other ETL tools use TCP protocols, ODBC, etc for connecting with Source DB, pulling the data and load in to DWH by using same protocols. In the similar way, what protocols/services does Talend use to pull the data from source, transform it and load in DWH. What are the services, protocols used by Talend to connect to DB, different nodes in Repository like Job design, metadata etc.. What is the means of communication use by Talend? Any help will be appreciated!
Hi, Talend is a code generator ETL which use JAVA as the underline technology generated to perform the Data Extraction, Transformation and Loading. For Database connectivity; we use ODBC and JDBC drivers delivered and certified by the Database vendors themselves (Oracle, DB2 IBM, Teradata, SQLServer, MySQL, etc... ); we implement the BULK LOADER of those Database vendors following their API or Executables. For Files connectivity : regarding the format we leverage different library udnerneath; file delimited is straight forward; but for XML or JSON we use the proper lib for that XPathQuery, etc... For CRM, ERP and other business application we integrate through Webservices API provided by the Business application vendors; for SAP we reuse the JCO connector/lib which is certified and provided by SAP himself to reach out RFC, BAPI functions; we also provide a iDoc connector for SAP. etc.... We provide also connector and protocol support for : FTP (FTPS or SFTP), SCP, SOAP, REST, WEBSERVICE, RSS, LDAP etc.... For DWH implementation; you'll have for sure connection to your SOURCE and TARGET and you'll specifiy the metadata into your METADATA REPOSITORY. Then you'll have a bunch of JobDesigns; the one which load your DIMENSION TABLES. For Dimension table we offer a connector called "SCD" to load the data in your Database providing SLOWLY CHANGING DIMENSION (type 0, type 1, type 2, type 3) to manage your Dimension tables. Finally you'll have to reuse these Dimension tables and map them using tMAP to bring the data into your FACT_TABLE. Pretty straight forward design and development. Talend is offering TRAININGS and PROFESSIONAL SERVICES if you're interested to learn more about how to use our solution.
Hi All, Thanks for sharing valuable information! Actually I often use Talend as ETL tool. I am acquainted with all basic transformations & there significance. I also know that the base language used in Talend in Java & Perl. While working with Talend, I am curious to know what exactly happens on the back side when we use different components in Talend for ETL. How it use to communicate and perform things so easily... Based on my understanding i have created a block diagram which is attached within. Please have a look in to it... and let me know i have missed some thing. Also if you can share any of such block diagram which would help me to understand the architectural representation.. it would be a great help!
As Cantoine says above from the jobs you design it builds java code, which is uses JDBC/ODBC, etc. to "use to communicate and perform things so easily" you can even see the code it produces if you select the code tab at the bottom left of the designer view in TOS if you wanted to go through it line by line.
Talend components in fact are code generators. All components have a begin + main + end part of code and all sub jobs (everything within a blue rectangle) is a Java method. Talend provides for all components a schema (if declared as needed) and input/output flows, iterates (actually also a flow) and the possibility of return values. This all together is one of the simplest and the cleverest code generator I have ever seen.
Hi All, Appreciate your reply! But actually am looking for architectural information in Talend. Any idea on that.. For reference I have attached one snapshot with my previous to previous post. Please have a look in to it. I have created that block diagram based on my understanding. Please suggest our view/comments on that block diagram. Best Regards, Mayur
Hi All, Does any one has graphical representation/block diagram/ image related to TOS Architecture, similar to the one which i have attached above. Please share, it will be helpful for to me to understand its architecture properly. Best Regards, Mayur
Sorry we don't have any architectural diagram that would be better than the one you propose. As suggested previously already, I advise you to check out the code tab of your Studio to see the generated java code and you'll then have the internal mechanics of your job. There is nothing much more to say. Elisa