Dynamic row_separator issue with new line "\n"

Four Stars

Dynamic row_separator issue with new line "\n"

Hi all,

 

I'm having issues using a context variable in tFileInputDelimited populated by a mysql query with "\n".

 

The ingestion process is metadata driven, so the files might have different row_separator depending on the value I retrieve from a stored procedure. Here is an example:

 

job_example.jpegcontext_example.jpegfile_example.jpegSP_example.jpeg

 

If I manually populate the context variable output with "\n" it works perfectly. however bringing it from sql and then populating the context variable it doesn't. Once it reaches the tFileInputDelimited it doesn't separate anything, bringing the entire file as one row.

 

It seems like an issue with the escape logic in the java backbone (not identifying the "\n" as an actual line break), but I haven't been able to understand completely why it only happens when the string comes from mySQL and how to workaround it.

 

Appreciate any ideas. I'm using enterprise version 7.0

 

Thanks

Highlighted
Four Stars

Re: Dynamic row_separator issue with new line "\n"

I've doing some debugging and research, it seems like this issue has been previously logged as a bug and was supposably solved.

 

Issue: TDI-17208

 

However I'm facing the same issue, although I pass the context variable as \n it sees it as \\n (using an extra "\" to escape). I've passed one variable with only \n and the other with double quotes "\n". Both come up with an extra backslash that I assume is why the tFileInputDelimiter is not interpreting it as a line break, but rather as a string \\n.

linebreakquote_example.jpeglinebreak_example.jpeg

 

 

Still no luck figuring a workaround thou.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Downloads and Trials

Test drive Talend's enterprise products.

Downloads