Job hangs when running a COPY in Redshift as part of a larger job.

One Star

Job hangs when running a COPY in Redshift as part of a larger job.

I've got a job that puts some data into Redshift and then puts some more data into Redshift in a later stage. For some reason, the second step hangs and I can't figure out why - I've checked that connections all have autocommit set to true and if I run the subjobs individually they all work fine.
The only other thing I can think of is that there is a custom library I developed with functionality that is being loaded multiple times (once in each subjob).
The issue is that the job gets to the point of the COPY (I've got a tjava ahead of it with system.out.println in it) and then freezes - I've left it for over an hour and nothing happens (it's insert a couple of rows and only takes a couple of seconds when I run the subjob separately). Is there any logging or anything I can turn on to track down the issue?
One Star

Re: Job hangs when running a COPY in Redshift as part of a larger job.

I've tried various things to fix it and differing things can resolve the issue, but when I try repeating the fix, it doesn't work again.
Seventeen Stars

Re: Job hangs when running a COPY in Redshift as part of a larger job.

Sounds silly. Because of you can reproduce this problem I would use the Java Debugger to find out where exactly the job hangs. If you know the called method you should post again and we can try help you.
One Star

Re: Job hangs when running a COPY in Redshift as part of a larger job.

I went through and purged all references to an old jar and set all Redshift connections to autocommit and the problem went away.