Four Stars

Troubles building a large REST service

Hi there!

 

So I'm currently building a large REST service in Talend Open Studio for ESB (6.1.1.20151214_1327) using tRESTRequest and tRESTResponse components.

 

And I have succesfully build a pretty complex REST service now. And I've started running into some limitations, as the number of API Mappings is in the twenties by now.

 

I have been slowly extending my rest service with more and more functionality untill I got an error similar to the following:

 

"The code of method XXXX is exceeding the 65535 bytes limit"

 

So the autogenerated java-code for my rest service has a method with too much code in it.

 

So I have split my huge tRESTRequest up into 7 smaller jobs with each their own tRESTRequest. And launch all jobs with a parent job running multi threaded. This way I'm able to start up some of my REST endpoints. But some will throw the following error:

 

"Cannot start provider with uri: http://127.0.0.1:8090/fmd/lookupcolumns. Port 8090 already in use."

 

Apparently if the tRESTRequest components gets run at the exact same moment they can start up in parrallel. But with 7 subjobs it seems 3-4 of them will get started a bit later and have the error about the port already being in use. Also its a bit random which ones will run correctly and which onces will get the error.

 

I've been looking into how to either disable the port check, or change the port number on the tRESTRequest component, but haven't found any options to achieve this.

 

So I'm in a bit of dilema. If I put everything into one tRESTRequest component I get hit by an error about the java-code being too long, and if I split it up into multiple tRESTRequest components I'm not able to run all of them in the same time.

 

I'm hoping someone in here have experience in building larger REST services in Talend ESB, and perhaps provide some help and ideas how to work around these problems. As I'm stuck with the following questions:

 

1. Is there a way to control the autogenerated java-code to prevent methods exceeding the size limit in java?

2. Is there a way to change the port number for a tRESTRequest component?

3. Is there a way to prevent tRESTRequest components from checking if a port is already in use? (As it seems multiple tRESTRequest components are able to run if they are started at the exact same time)

4. Or rather is the a way to proberly run multiple tRESTRequest components without port conflicts?

 

Any help would be highly appriciated!

 

  • Data Integration
  • ESB
Tags (2)
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: Troubles building a large REST service

1) Java Code size is a java limitation and the way you have approached it is correct.

2) If you are running your Rest services on port 8090, does that mean you are running all in studio?  Also your endpoint url in tRestRequest is relative or fully defined?  You can use a different port number in the tRestRequest component with each service.

3) No, and you cannot run 2 listener on the same ports.  You will get the port conflict problem.  Basically each service is a listener.

4) Yep run multiple services with different ports to avoid the conflicts.

2 REPLIES
Employee

Re: Troubles building a large REST service

1) Java Code size is a java limitation and the way you have approached it is correct.

2) If you are running your Rest services on port 8090, does that mean you are running all in studio?  Also your endpoint url in tRestRequest is relative or fully defined?  You can use a different port number in the tRestRequest component with each service.

3) No, and you cannot run 2 listener on the same ports.  You will get the port conflict problem.  Basically each service is a listener.

4) Yep run multiple services with different ports to avoid the conflicts.

Four Stars

Re: Troubles building a large REST service

Thanks for clearing that up.

 

1. Guess it would require a major rewrite of the code generator in Talend to actually include the amount of functionality I had going, I don't mind splitting up my functionality anyway the diagram was getting a bit bloated.

 

2. Yes I was running it all in the studio at the moment. Will get around putting it into a karaf container at some point. And I only had relative paths defined. I did however run one of the services through the java debugger and noticed the REST endpoint gets checked for "://" and uses it as absolute path if so. For some reason I was of the belief that you could only define a relative path. I've just set each REST Endpoints to something similar to the following: "http://127.0.0.1:8093/fmd/users", must importantly with different ports for each service.

 

3. Yeah it was buggy at best when running multiple listeners on the same port. I'm kinda surprised I was able to start more than one service on the same port now that I think about it.

 

4. Thanks, once I figured out how to setup different ports on the services it solved my issue. And I now have 7 different tRESTRequest components running on different ports, so my problem has been solved.

 

Thanks again for the help!