Six Stars

Talend job hanging "starting"

Have a redshift output component pointing at a table with 37 million rows in it, being fed by a tMap and several lookups. When I start the job it doesn't even try to run the input query or the tMap, it just loads the lookup tables and then says "starting" above the flow to the tRedshiftOutput component.

 

This issue has been occurring all over the place across all my jobs, they just refuse to run.

 

Anyone ever heard of this or know what my problem might be?

 

Image is "Live Statistics" from TAC.

 

tripLoaderScreenshot.png

1 ACCEPTED SOLUTION

Accepted Solutions
Moderator

Re: Talend job hanging "starting"

Hi,

It seems the DB you were connecting was unresponsive. Did you try to restart your DB to see if it work?

Is it OK with you that output your data into a file instead of DB?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
10 REPLIES
Six Stars

Re: Talend job hanging "starting"

And here is an image showing the current status:

 

tripLoaderScreenshot2.png

Employee

Re: Talend job hanging "starting"

Maybe check the checkbox "Die on error" on the tRedshiftOutput component and run it to see what is causing it to hang.

Six Stars

Re: Talend job hanging "starting"

Thank you for the suggestion. I gave it a go but no luck, it seems there is not an error because the component does not die and I do not get a message in the log, it still hangs.
Six Stars

Re: Talend job hanging "starting"

I put a system.out.println in the input query box and it won't even run that. No idea what it is doing.

Employee

Re: Talend job hanging "starting"

Are you sure you have access from wherever you are running the job  to the redshift table?

Moderator

Re: Talend job hanging "starting"

Hi,

It seems the DB you were connecting was unresponsive. Did you try to restart your DB to see if it work?

Is it OK with you that output your data into a file instead of DB?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Talend job hanging "starting"

Writing to a file works.

 

It seems to be a problem with the database.

 

Thank you for the help.

Four Stars

Re: Talend job hanging "starting"

I have exactly the same issue. Was this ever resolved?

Six Stars

Re: Talend job hanging "starting"

This was almost a year ago now!

 

I can't really say what exactly it was, all I've got is a few key things we've done since:

 

- Use tRedshiftOutputBulkExec instead of a standard tRedshiftOutput. This will write files to S3 and then use the COPY command to load the data into redshift.

- Make sure to use streaming in the mysqlInput advanced options (for large result sets).

- We moved away from tMap in favour of tJavaFlex and doing all the mapping programmatically where feasible.

 

Let me know if you are still stuck after trying some of this stuff and I'll see if I can think of anything else.

Moderator

Re: Talend job hanging "starting"

Hello,

Does it work when you write to a file?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.