I have a really small problem with loading my structured csv to a MongoDB and achieving the right format.
What I currently have:
My Talend process loops through a directory of csvs, loads them, uses a tMap component to add some information, gives it to a tMongoDBOutput.
The input data is structured like like this:
The columns "info", "sensorbasis" and "filename" are added within the tMap component and come either user input into a context variable, or (the filename) from the tFileList component.
What I want to achieve:
In the MongoDB there is a database called "mobidic" and a collection called "sensordata". I want to load all information from the CSVs to this exact collection, but, in a very specific structure that we need to work with at later stages. The format should look like this:
So basically, the result should be grouped by the fixed information, such as "sensorbasis", "info" and "filename", but the other columns should be in an array within the object "measurements". I understand, that I somehow have to set something as group, and something as a looping element.
This is what I have achieved so far:
Data is loaded to the MongoDB. I have configured the tMongoDBOut component, but not as intended:
And this is how the json tree within the component looks like:
What am I missing? I have searched through the board and did find some posts regarding the same challenge, but none of them brought me any further.
I hope somebody here can give me a hint on what I am missing.
Tanks in advance!
Solved! Go to Solution.
Could you please show me the tMap component. I am also trying to create a nested document, but loop element is not creating a array, it is simply creating a sub element.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.