Talend Integration with Solr Component Suggestion

One Star

Talend Integration with Solr Component Suggestion

Is there an already available component that does http posts on a url with multipart form like what curl does? I'd like to read from a SQL Server table do some transformation and post the row to Solr in realtime. Is there any component that does that? If not, can this be developed.
Thanks
Yash
Community Manager

Re: Talend Integration with Solr Component Suggestion

Hi Yash
As I known, there is no a official component can fit your request now, if you want the component to be developed soon and personalize services, please contact us.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Talend Integration with Solr Component Suggestion

But Can't this be done with tjavarow or tjavaflex component? All you need is ability to import external jars to use the solrj jars. And someway to convert the row data in to xml and feed it to Solr via an http post?
Yash
Community Manager

Re: Talend Integration with Solr Component Suggestion

Hi
You have to hard code on tJava or routine to do http posts on a url with multipart form to Solr. We have a tLibraryLoad component which can be used to import external jars, the tWriteXMLField is used to convert rows to a xml string, the tAdvancedFileoutputXML is used to generated a xml file.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Talend Integration with Solr Component Suggestion

I was able to achieve this using SolrJ...so no XML mess to do in Talend. I wrote a blog post about this - check it out at http://technologyenablingbusiness.blogspot.com/2011/02/using-talend-to-do-etl-in-to-solr.html
Employee

Re: Talend Integration with Solr Component Suggestion

Hello yash,
Thank you for your interesting blog post!
One Star

Re: Talend Integration with Solr Component Suggestion

Hello,
I've created a solr output component using SolrJ that can be used for indexing data to Solr.
You can find it on github https://github.com/sourcesense/tSolrOutput
Cheers,
Bilgin Ibryam
One Star

Re: Talend Integration with Solr Component Suggestion

Im working on some solr compoents, tSolrInput, tSolrOutput and tSolrDelete. All components will supporting both http and embedded solr.
The tSolrOutput are finnish, supporting batch push of documents, batch commit, and optimize at the end. All controll by component settings.
Will put it on github when im finnish :-)
Community Manager

Re: Talend Integration with Solr Component Suggestion

Hi SleepyCrom
First, thanks very much for your sharing to all Talend users.
It will be good if you also submit the component to Talend exchange. Talend Exchange is the place where Talend community can share items related to Talend opensource products
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Talend Integration with Solr Component Suggestion

Hi again, I have 7 solr component, tSolrCommit, tSolrConnection, tSolrDelete, tSolrInput, tSolrOptimize, tSolrOutput and tSolrRollback. Supporting embedded and external solr index.
I have written the tSolrConnection to work like any other connection component. I want to use metadata, but I dont find example about define your own metadata.
Anyone know where to find metadata examples ?
rgd
Audun Wilhelmsen

Re: Talend Integration with Solr Component Suggestion

Hi SleepyCrom,
I came across this discussion which talks about Solr Components. I was doing a quick research on exchange.talend.com for solr and found Talend Solr High Speed Query plugin on first result. I am using this plugin and found very useful to update my existing solr documents.
Regards,
Dean
One Star

Re: Talend Integration with Solr Component Suggestion

Hi,
We're working on evaluation Talend with Solr. With respect to this implementation we need certain pointers to start with.
Can you please suggest on the following:
1) Is Solr components available with Talned Data Integration/Big Data Enterprise version(License edition) ?
2) Which Talend product suite provides out of box support for Solr Integration ?
3) There are custom components available on exchange "Talend Solr High Speed Query plugin". Is this the only integration route we currently hold? Do we have steps to include these custom components in Talned Open Studio?
Thanks