Talend DI (24)
Talend connection component retry logic & design for databases, web services etc...
For the instances where for whatever reason, a Talend job does not always connect to a backend service layer - database, web service, ftp, salesforce, dropbox - on the first try, the job can be modified to retry the connection before failing the job. This solution shows a simple design to handle intermittent connection failures.
Testing Hive or MySQL database connections directly using JDBC driver
Every once in a while, one runs into a situation that one can not connect to a database from a tool or application. When this happens, the best way to isolate the issue is to try to connect to the same database using a quick java application using standard JDBC.
Best practice for creating Talend metadata connections to NoSQL DBs and Hadoop
Starting with Talend 5.6.2, it is now possble to create metadata connections for NoSQL databases and Hadoop platforms using the metadata feature in the Studio. Even better, the Studio now allows automatic discovery of these properties using Hadoop properties site-*.xml files.
Connecting Talend to an MS SQL Server DB with 'applicationIntent=ReadOnly'
Out of the box, Talend uses the open source jTDS driver to connect to MS SQL Server databases. This driver however does not support connecting to an AlwaysOn enabled database. A generic jdbc driver would have to be used as a work-around.
Enabling Hive High Availability in Talend Studio
Starting with Talend 5.6.1, a patch was released by Talend to update Hive components to be able to connect to Hadoop Clusters configured for HA - High Availability. In HA, instead of configuring the Hive host in the component and a standard port, we now instead specify the Zookeeper quorum that in turn 'discovers' the active Hive Host.
'Not implemented by the DistributedFileSystem FileSystem implementation' error in Talend Big Data
'Not implemented by the DistributedFileSystem FileSystem implementation' error occassionally rears its head when debugging Talend Big Data jobs. This is a cryptic message that actually intends to convey that your job includes JARs from different versions of Hadoop in its classpath.
Dynamic Select SQL statement execution for moving data from DB2 to Hadoop (Design Pattern # 1)
Data warehousing and ETL processes usually repeat common patterns across different data domains (databases, tables, subject areas etc...). One such pattern is copying data from a transactional system to Hadoop or some other data platform (Teradata, Oracle DBs) to create 'images' of those systems for downstream processing. Because these processes are repeated many times over in the design & construction of data warehouses, it is best to create repeatable patterns that reduce future technical debt in terms of support, maintenance and updates costs.
Introducing Talend 6.0
Talend 6 was released in September 2015 and with it come a number of new and important features and updates, including product name changes.
Configuring Talend Hive Components: HiveServer1 vs. HiveServer2, Embedded vs. Standalone?
Talend Hive components have a number of somewhat confusing options that could be tricky to understand when making connections to a Hadoop cluster. Options include selecting between HiveServer1 and HiveServer2, Embedded vs. Standalone modes, and what ports to connect to. We explore the options in this post, pulling in information from the Hive Wiki, Talend Support and other sources.
More...
Connecting to HiveServer2 in a Hadoop 2.x HA Cluster using DBVisualizer
In a previous post, the steps for downloading and configuring DBVisualizer to connect to Hive were presented. The connection was made using a Hive Host Name in a Hadoop cluster with a single Namenode. In this post, we look at connecting to a Hive in a Hadoop cluster that's configured for HA (High Availability), meaning it has multiple Hive hosts (and namenodes & resource managers etc...) where one Hive host is active and while others are passive.
Talend Studio: Use (Java) JRE or JDK?
Talend Studio (essentially a customized Eclipse IDE) requires that Java be installed on the client in order for the Studio to function - run jobs etc... Most often when installing Talend, the decision of installing Java is a no-brainer - basically click-through on java.com and you're done. But that doesn't always work, depending what you're doing in your Talend job.
Talend 5.x and Cassandra CQL 2 & 3
For quite some time, Talend has included a family of components for Apache Cassandra NoSQL database. However, recent versions of Talend 5.x have not yet started generating code to connect to Cassandra that leverages the newer Cassandra Query Language (CQL) version 3. CQL v3 is not backward compatible with CQL v2 and differs from it in numerous ways.
Downloading and Processing LivePerson Chat Data - Part 2
This is part 2 in a series about downloading and processing LivePerson chat data. LivePerson is a leading business chat, online messaging, marketing, and analytics platform that's integrated into many online sales channels / websites. In part 1 of the series, we looked at how to test connectivity to LivePerson API. In this part, we move on to developing the solution using Talend Data Integration Studio.
