7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

Six Stars

7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

Hello everyone,

 

I am facing an extremely annoying talend issue for months and months that happens sometimes ... randomly. Let me try to explain you :

 

1) I want to update my database schema, so I left click on the first table and then "edit schema" : 

 

edit schema.png

 

2) Then I update what I want by clicking "retrieve schema" when needed. Obviously, I select the right table inside the left panel "Schema" before clicking "retrieve schema". Note : For this example, I won't update anything. That's important ...

 

2.1) Let's see the first table of the list (left panel : "schema") :

 

abstract_liberality.png

 

abstract_liberality schema is based on abstract_liberality table. When I click on "retrieve schema" I get the right fields etc.

 

2.2) Now let's see the last table  of the same list :

 

user_platform.png

 

user_platform_to_ schema is based on user_platform_to_ table. When I click on "retrieve schema" I get the right fields etc.

 

2.3) Absolutely nothing has changed in both tables and schemas. Both tables are correctly set up, based on the right table. Then I click finish button.

Talend takes some seconds to check the whole project to detect updates. Then it shows them :

 

changename.png

 

TADAM, here is real magic ! It detects unwanted and completely unexisting table name changes ...

Moreover, these detections can't be unchecked... "Deselect all" button does nothing... Each checkbox does nothing.

 

All I can do is click"OK" and it will unfortunately apply these stupids changes...

If you want to cancel by clicking the top right cross... changes will be applied though...

 

The only solution I found to not apply these updates is to force talend shutdown ... that's quite brutal !

 

As you can see here, the update detected is to rename abstract_liberality into user_platform_to_ but that is completely random, sometimes it will be the exact opposite : rename user_platform_to_ into abstract_liberality. Such an update is extremely annoying because it breaks all my links in tMap components, etc.

 

If someone has already encountered this issue, or know how it can be avoided, I would be very grateful !

 

Thanks,

Guillaume

Community Manager

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

Thanks for illustrating that issue very accurately. I have seen this before (a long time before I worked for Talend) but ended up working through it. It was a real pain. I actually believed it had been solved in later versions. If this is the same issue, it will happen with only a couple of tables, but will happen every time you make a change to one of those tables? Does that ring a bell? I will point a jira ticket to this post and see if we can get this problem resolved. 

 

Can I ask if you would add any further information to this post and if you can recreate it so that it can be packaged up and sent to us that would be great. I have tried to recreate it here, but cannot. I do not have access to the project I experienced it with either. If you have a support contract it would be good if you could contact support and show them this.

 

Once I have raised the jira ticket I will link to it so that you can see it.

 

A way that I was able to get round this in the past was to rename the table which caused this issue, then re-import the table as a new table. This seemed to get round it. This leads me to suspect that it is down to an issue with the UID of the schema in the project file.

 

Thanks for this post.

Community Manager

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

I have raised this here: https://jira.talendforge.org/browse/TDI-42199

 

Another way of resolving this (working around this) was to create a generic schema and use that in your jobs which are affected. Not ideal, but it stops you from having to correct your job whenever you update your schema.

Six Stars

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

I didn't expect such a quick answer, thank you very much ! I am pleased to see that I am not the only one who faced this issue...

What kind of additionnal information do you need ?! Unfortunately, I don't know if I can recreate this issue in an other project... I think the whole context should be replicated which is impossible. This talend project is quite huge, as well as databases linked to it. Moreover this work is confidential atm.

I may try to create a new project with just a little part of the database and these two tables but unfortunately I don't think it will work :/

However, I can give you some little more information :

1) It doesn't happen every time I make a change to one of those two tables. No no, it happens every time ! Even if I do absolutely nothing. Left click a table, edit schema, finish => it will search for updates and detect those unexisting changes. That means I can't update any table anymore atm.

2) It always happens between first and last table. In the past, the first table was "abstract_person" which you can see in the second screenshot and the last one was "right_domain" which you can see in third screenshot. So I had faced this issue between those two tables. That was not so embarassing because "right_domain" table is currently unused so I could have luck and not face the issue. Today, both involved tables are used so whatever the direction of the change is it will affect my jobs.

Thanks a lot for your help.
I remain available for any information.
Community Manager

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

You are correct. It has been a while since I have seen this. It is when you make any "read schema" call, there will be a perceived change to your tables which are affected, then they will be changed in the job. This is refreshing my memory. My project was also a very large project. I suspect this is to do with UID for the schemas. 

 

If you have a support account, please do raise this. If you do, let me know when (via private message if necessary) and I will see if I can chase it up to add my experiences. I suspect that not many people have seen this as it is likely only happening in certain scenarios.

 

Thanks again for raising this. 

Six Stars

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

Hi rhall_2_0,

 

Unfortunately we do not have any support account here and as you said I don't think that many people have already faced this issue because there is absolutely no testimony anywhere...

 

This is really annoying for me at the moment to be honest and so unlucky. This issue is hardly reproducible and it may take months, or even years, before someone finds a patch.

 

Thanks for your help Smiley Happy

Community Manager

Re: 7.1 : Retrieve schema issue (unwanted and non-cancelable table renaming)

The way to stop this affecting your jobs is to manually create a generic schema for the schemas which this affects. It does mean that you will have to manually update the generic schemas, but it will also mean that your jobs using those schemas will not be broken whenever you carry out a schema update.

 

I will continue to try to push this internally and see if we can find a way of recreating this and solving it

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

An API-First Approach to Modernizing Applications

Learn how to use an API-First Approach to Modernize your Applications

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend API Designer – Technical Overview

Take a look at this technical overview video of Talend API Designer

Watch Now