[resolved] Merge xml files in one with multiple loops

Five Stars

[resolved] Merge xml files in one with multiple loops

Hi,
I have a job that generate a complex XML file using tAdvancedFileOutputXML in append mode, the result is correct, but this component is extremely slow (for 15MB file size is like 4hrs !) this is due to their use of DOM4J instead a SAX parser.
So i tried this :
step 1 : To write different xml files, one for each loop in sax mode, very fast Smiley Happy
step 2 : Merge the xml files with the tXMLMap, but the tXMLMap doesn't allow more than one loop.

Here is the example on my output_xml file :  
<root>
 <genericData>
   <name>test</name>
   <service>Service Name</service>
   <serviceDesc>serviceDesciption</serviceDesc>  
   <Resp1>
     <codeResp>aaa</codeResp>
     <nomResp>bbb</nomResp>
   </Resp1>
 </genericData>
 <survey> ---> Loop Element
<A>blabla</A>
<B>blingbling</B>
<surveyNodeInfo>  ---> Loop Element
<surveyNodeInfoX>xxxx</surveyNodeInfoX>
<surveyNodeInfoY>yyyyy</surveyNodeInfoY>
</surveyNodeInfo>
<surveyNodeOtherInfo> ---> loop element
<surveyNodeZ>zzzz</surveyNodeZ>
<surveyNodeM>mmmmm</surveyNodeM>
</surveyNodeOtherInfo>
<sampling> ---> Loop Element
<C>blabla</C>
<D>blingbling</D>
<samplingNodeInfo> ---> Loop Element
<samplingNodeInfoX>INFO</samplingNodeInfoX>
<samplingNodeInfoY>OTHER INFO</samplingNodeInfoY>
</samplingNodeInfo>
<samplingNodeOtherInfo> ---> Loop Element
<samplingNodeW></samplingNodeW>
<samplingNodeU></samplingNodeU>
</samplingNodeOtherInfo>
<E>indformation node</E>
<sample>---> loop element
<F>blabla</F>
<G>blingbling</G>
<sampleNodeInfo> ---> Loop Element
<sampleNodeInfoS>INFO</samplingNodeInfoS>
<sampleNodeInfoT>OTHER INFO</samplingNodeInfoT>
</sampleNodeInfo>
<sampleNodeOtherInfo> ---> Loop Element
<sampleNodeC>CCCC</sampleNodeC>
<sampleNodeD>DDDDD</sampleNodeD>
</sampleNodeOtherInfo>
<H>indformation node b</H>
<sampleNodeJ>  ---> Loop Element
<sampleNodeInfoJ>JJJJJJ</sampleNodeInfoJ>
</sampleNodeJ>
</sample>
<samplingNodeJ>  ---> Loop Element
<samplingNodeInfoJ>JJJJJJ</samplingNodeInfoJ>
</samplingNodeJ>
</sampling>
<I>IIIII</I>
<surveyNodeJ>  ---> Loop Element
<surveyNodeInfoJ>JJJJJJ</surveyNodeInfoJ>
<surveyNodeInfoJJ>KKKKKK</surveyNodeInfoJJ>
</surveyNodeJ>
 </survey>
</root>
 

Using TOS 5.5.1
How can a merge the xml files in a correct time ? or maybe Talend is not allow to do this ?
Thanks for your help.
Moderator

Re: [resolved] Merge xml files in one with multiple loops

Hi,
Have you checked "all in one" feature in tXmlMap Settings which supports generating multi rows into one xml?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Merge xml files in one with multiple loops

Hi,
Yes i already have this option Smiley Sad
regards
Employee

Re: [resolved] Merge xml files in one with multiple loops

Talend Data Mapper can easily handle this.
Five Stars

Re: [resolved] Merge xml files in one with multiple loops

Hi,
The Talend Data Mapper is under licence ? can i istall it in TOS 5.5.1 ?
Moderator

Re: [resolved] Merge xml files in one with multiple loops

Hi _mTs_,
Talend Data Mapper is only available in Talend Enterprise Subscription Version.
Please take a look at Talend Data Mapper User Guide:http://doc.talend.com/enterprise/talend_data_mapper/user_guide/5.5/.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Merge xml files in one with multiple loops

Hi,
Thanks for the information.
I will like to know how fast is data mapper to generate a complex xml file 30 MB ?  rows/s ?
Regards
Moderator

Re: [resolved] Merge xml files in one with multiple loops

Hi,
I will like to know how fast is data mapper to generate a complex xml file 30 MB ?  rows/s ?

Usually performance depends on job design.
Feel free to contact us for your requirment of Talend Data Mapper.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Merge xml files in one with multiple loops

Hi again, i solve my problem using the tGroovy component.
This link help me to solve my probleme with TOS :
merge xml files with groovy
Regards
_mTs_
Moderator

Re: [resolved] Merge xml files in one with multiple loops

Hi _mTs_,

What's your customized code in tGroovy component? Would you mind sharing it with us?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.