I have a really small problem with loading my structured csv to a MongoDB and achieving the right format.
What I currently have:
My Talend process loops through a directory of csvs, loads them, uses a tMap component to add some information, gives it to a tMongoDBOutput.
The input data is structured like like this:
The columns "info", "sensorbasis" and "filename" are added within the tMap component and come either user input into a context variable, or (the filename) from the tFileList component.
What I want to achieve:
In the MongoDB there is a database called "mobidic" and a collection called "sensordata". I want to load all information from the CSVs to this exact collection, but, in a very specific structure that we need to work with at later stages. The format should look like this:
So basically, the result should be grouped by the fixed information, such as "sensorbasis", "info" and "filename", but the other columns should be in an array within the object "measurements". I understand, that I somehow have to set something as group, and something as a looping element.
This is what I have achieved so far:
Data is loaded to the MongoDB. I have configured the tMongoDBOut component, but not as intended:
And this is how the json tree within the component looks like:
What am I missing? I have searched through the board and did find some posts regarding the same challenge, but none of them brought me any further.
I hope somebody here can give me a hint on what I am missing.
Tanks in advance!
Solved! Go to Solution.
Could you please show me the tMap component. I am also trying to create a nested document, but loop element is not creating a array, it is simply creating a sub element.
Join us live for a sneak peek!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.