• Blog
  • Talend DI
  • Talend connection component retry logic & design for databases, web services etc...
Friday, 22 July 2016 14:43

Talend connection component retry logic & design for databases, web services etc...

Written by
Rate this item
(9 votes)

For the instances where for whatever reason, a Talend job does not always connect to a backend service layer - database, web service, ftp, salesforce, dropbox - on the first try, the job can be modified to retry the connection before failing the job. This solution shows a simple design to handle intermittent connection failures.

The following screenshots visually describe a retry solution for Talend component connections. You can update the code to use context variables or global variables for the values that are hard-coded - wait time (2 seconds), number of retries (5).



Starting job - create a boolean context variable and set to true


// resetting loop flag
context.continueLooping = true;

log.info("Starting job...");


Set Condition on tLoop to variable on While loop



Now, connect to the resource - here MDM (could be FTP, web service, database etc...)



Using onComponentOK (for successful connection), set the boolean variable to false (to exit loop).


 // on success, exit loop

context.continueLooping = false;

log.info("Connection established on attempt # " + ((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")).toString());

Using onComponentError (for failed connection), retry for 5 times then proceed (which will fail the job); else retry the job.


if (((Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")) > 5)
log.error("Connection failed. Exiting retry loop.");
// exit loop
context.continueLooping = false;
log.info("Connection failed. Retrying...");


Read 18881 times Last modified on Sunday, 24 July 2016 16:41
Will Munji

Will Munji is a seasoned data integration, data warehousing and business intelligence (BI) architect & developer who has been working in the DW/BI space for a while. He got his start in BI working on Brio SQR (later Hyperion SQR) and the Crystal Decisions stack (Reports, Analysis & Enterprise) and SAP BusinessObjects / Microsoft BI stacks. He currently focuses on Talend Data Management Suite, Hadoop, SAP BusinessObjects BI stack as well as Jaspersoft and Tableau. He has consulted for many organizations across a variety of industries including healthcare, manufacturing, retail, insurance and banking. At Kindle Consulting, Will delivers DW/BI/Data Integration solutions that range from front-end BI development (dashboards, reports, cube development, T-SQL/ PL/SQL ...) to data services (ETL/ DI development), data warehouse architecture and development, data integration to BI Architecture design and deployment.

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.

Contact Us

Kindle Consulting, Inc

6595 Roswell Road 
Suite G2025 
Atlanta, GA 30328

Phone: 404.551.5494
Fax: 404.551.5452
Email: info@kindleconsulting.com

Talend Gold Partner