I thought I would make life easier and run a dataprep on a live dataset that is generated from a talend job so I am able to run a tdatasetinput and a tdatapreprun to clense the results in real time.
However, I see now way of how it is a "live dataset" when it has been chosen as this option in the drop down, how do I update it? can I update it? if I can update it how? I have made changes to the excel source how to I re-run the job on this dataset?
Moving on.. I spent a while doing data prep changes such as removing invalid fields, changing the date to ISO etc... great this all looked fine in the UI so now lets create a Talend Job.
Using tDataSetInput it grabs the "live" dataset fine (sometimes runs connection errors but if I keep re-running the job it will eventually work), Runs this into a tDataPrepRun Job where I have selected the data prep that "looks fine in the cloud UI" then when I run the job it completely ignores all the date changes I made and throws errors.
Why? Is this even supposed to work? seems way too buggy to be released for anything above basic usage.
I have decided to use a tReplace to remove all invalid rows on the dataset and not use data preparation at all because it has been way too much hassle for something that should be simple.