Dynamic row_separator issue with new line "\n"

Four Stars

Dynamic row_separator issue with new line "\n"

Hi all,

 

I'm having issues using a context variable in tFileInputDelimited populated by a mysql query with "\n".

 

The ingestion process is metadata driven, so the files might have different row_separator depending on the value I retrieve from a stored procedure. Here is an example:

 

job_example.jpegcontext_example.jpegfile_example.jpegSP_example.jpeg

 

If I manually populate the context variable output with "\n" it works perfectly. however bringing it from sql and then populating the context variable it doesn't. Once it reaches the tFileInputDelimited it doesn't separate anything, bringing the entire file as one row.

 

It seems like an issue with the escape logic in the java backbone (not identifying the "\n" as an actual line break), but I haven't been able to understand completely why it only happens when the string comes from mySQL and how to workaround it.

 

Appreciate any ideas. I'm using enterprise version 7.0

 

Thanks

Four Stars

Re: Dynamic row_separator issue with new line "\n"

I've doing some debugging and research, it seems like this issue has been previously logged as a bug and was supposably solved.

 

Issue: TDI-17208

 

However I'm facing the same issue, although I pass the context variable as \n it sees it as \\n (using an extra "\" to escape). I've passed one variable with only \n and the other with double quotes "\n". Both come up with an extra backslash that I assume is why the tFileInputDelimiter is not interpreting it as a line break, but rather as a string \\n.

linebreakquote_example.jpeglinebreak_example.jpeg

 

 

Still no luck figuring a workaround thou.