One Star

[resolved] GPLoad component not loading data

Hi all,
I've been working with Talend for some time now and I've got the basics down pretty well. Over the past months I've been working with Greenplum databases (with and without Talend). For the purpose of the 'Talend Team' I am running TOS 5.6 on Windows Server 2008.
Now I am trying to use the tGreenplumGPLoad component. Firstly, I have done gploads manually before, hundreds of times, using .yml control files etc etc. But now I want to use Talend to try and do this for me. But I've run into several problems along the way...
As I've checked out the forums (searching through posts over the past 3 years) I found similar problems over and over so for the purpose of everyone else I will start from the beginning - Apologies for the longer post this may cause.
First I ran into problems where I could not get gpload to work AT ALL. Talend returned error after error, but I eventually solved this by installing the GPLoad Tools for Windows from Pivotal - I wasn't aware external software was required for Talends version to work at the time but it seems it is.
So now I had those I tested they worked from the server with a few manual tests - .yml files created for the test data I had. Now I tried gpload from talend again, nothing. But I realised that when I do it manually my user sources the 'greenplum_loaders_path.bat', so I think this is what the component has to do too. I did this by going into 'Advanced Settings', clicking 'Specify gpload path' and then pointing it at the file.
However from here on I ran into a new problem.. When I run the component this is what I get:
connected
Setting up PATH for Greenplum Loaders
LOADERS environment variables configured successfully.
disconnected
.. but if I give the component a non-existent file, table or schema name it still says the same thing: Plus, no matter what I do it will not load data (I think it isn't even trying.. Not sure). I can confirm it is pinging the database successfully as when I change any details here it fails to connect to server - So that's something.
Any help on this matter would be highly appreciated. I have searched for near 2 months on this issue and found no help as of yet unfortunately. Importantly my last post about this was deleted by someone - Not sure why Smiley Frustrated I log onto it daily and today it is no more... RIP old post.. ^.^
Regards,
1 ACCEPTED SOLUTION

Accepted Solutions
One Star

Re: [resolved] GPLoad component not loading data

I have, alas, solved my own problem. After trying for over a month without any help...
So as above, pointing the GPLoad component at the loaders path seemed to get me only halfway. I have recently moved away from the Windows 2008 Server to a CentOS based system whereby I have re-installed the gploaders, java and, of course, tested them manually against the GPDB.
I then loaded up a talend job (I had to build the job and sent it over as a script, but it is the same thing in reality) and tried again. At first I got a permission error, since Talend was not allowed access to the loaders file. So instead I tried it without a path to see if I got the same old problem of no directory existing and it ran without a problem... Loaded data and all...
So I think the major problems that arose were: Not having gploaders, and then simply trying to point the gpload component at them directly. Instead I simply sourced the gploaders path as the user running the script (job containing gpload component) and this worked correctly.
I tested this by running it again as root -> Error: No directory gpload exists -> Sourcing the gploaders as root and re-running -> Loaded test data correctly from file to database.
I still am unaware why it didn't work on my Windows 2008 Server, perhaps an underlying windows error, as opposed to a talend one? I do not know. But it does work without problems on a linux based machine. Tried and tested several different ways.
I hope this is of help to anyone that might come looking. I had problems with this for months so I feel your pain!
Regards,
1 REPLY
One Star

Re: [resolved] GPLoad component not loading data

I have, alas, solved my own problem. After trying for over a month without any help...
So as above, pointing the GPLoad component at the loaders path seemed to get me only halfway. I have recently moved away from the Windows 2008 Server to a CentOS based system whereby I have re-installed the gploaders, java and, of course, tested them manually against the GPDB.
I then loaded up a talend job (I had to build the job and sent it over as a script, but it is the same thing in reality) and tried again. At first I got a permission error, since Talend was not allowed access to the loaders file. So instead I tried it without a path to see if I got the same old problem of no directory existing and it ran without a problem... Loaded data and all...
So I think the major problems that arose were: Not having gploaders, and then simply trying to point the gpload component at them directly. Instead I simply sourced the gploaders path as the user running the script (job containing gpload component) and this worked correctly.
I tested this by running it again as root -> Error: No directory gpload exists -> Sourcing the gploaders as root and re-running -> Loaded test data correctly from file to database.
I still am unaware why it didn't work on my Windows 2008 Server, perhaps an underlying windows error, as opposed to a talend one? I do not know. But it does work without problems on a linux based machine. Tried and tested several different ways.
I hope this is of help to anyone that might come looking. I had problems with this for months so I feel your pain!
Regards,