In the previous post we were talking about the situation when an Ignite cluster is deployed in Google Compute Engine network and we need to have nodes auto-discovery mechanism.

Well, Apache Ignite has Compute Engine specific solution to fill this gap but what's about many others cloud platforms? What if I want to use AWS, Rackspace, GoGrid or other cloud provider?

Do you need to switch back to annoying and fragile static IP configuration? Luckily, the answer - you don't!

Since 1.1.0-incubating release Ignite has a so called TcpDiscoveryCloudIpFinder that supports a variety of cloud providers out of the box. Actually, under the hood this IP finder is integrated with well-known Apache jclouds multi-cloud toolkit. Using jclouds the IP finder retrieves IP addresses of all virtual machines in a cloud, talks to them and forms a list of active Ignite nodes (the machines with running Ignite instances). Every node stores a copy of such an up-to-date list. All this lets the nodes to discover each other automatically.

Having this in mind, let's set up and use TcpDiscoveryCloudIpFinder. As an example I'll keep using Google Compute Engine platform but you can use any provider from this list.

TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryCloudIpFinder ipFinder = new TcpDiscoveryCloudIpFinder();

// Configuration for Google Compute Engine.
ipFinder.setProvider("google-compute-engine");
ipFinder.setIdentity(yourServiceAccountEmail);
ipFinder.setCredential(pathToYourPemFile);
ipFinder.setZones(Arrays.asList("us-central1-a", "asia-east1-a"));

spi.setIpFinder(ipFinder);

IgniteConfiguration cfg = new IgniteConfiguration();
 
// Override default discovery SPI.
cfg.setDiscoverySpi(spi);
 
// Start Ignite node.
Ignition.start(cfg);

First three parameters that are passed to the instance of the IP finder in the code above are set according to jclouds requirements:

• Provider name is equal to a value stored in "Maven Artifact ID" column of the following table. Compute Engine's value is "google-compute-engine", that's why it's passed to ipFinder.setProvider();

• Both identity and credential are provider specific. To get a provider specific instructions click on a provider name in "Provider" column in the same table. Google Compute authentication method is an exception, it's authentication method is covered on this page.

This post I want to dedicate to one useful feature of Apache Ignite that I had a chance to contribute to the project and that became a part of it since release 1.1.0-incubating.

Actually, this will be my first post about this Apache project and for those who are unfamiliar with it I would recommend to overview a good quick start documentation that can be found here.

The feature I'm going to cover is related to nodes discovery process located in a cluster in case when the cluster is a set of virtual machines deployed on Google Compute Engine. Historically Apache Ignite offered us two ways for the discovery process - leveraging multicast protocol or pre-configuring nodes' IP addresses. Unfortunately, neither way works well for Google VMs (virtual machines) instances: multicast is not supported in Compute Engine network at all and static IP configuration is not flexible (machine IP address may change from time to time, machines can be removed or added to a cluster, etc.).

However, there is a solution to this problem called TcpDiscoveryGoogleStorageIpFinder.

This IP finder does exactly what its name says. It lets virtual nodes located in Compute Engine network discover each other automatically using Google Cloud Storage where Apache Ignite's kernal will keep track of all cluster's nodes.

What you need to do to activate this feature is set TcpDiscoveryGoogleStorageIpFinder as an IP finder for a VM node.

TcpDiscoverySpi spi = new TcpDiscoverySpi();

TcpDiscoveryGoogleStorageIpFinder ipFinder = new TcpDiscoveryGoogleStorageIpFinder();

ipFinder.setServiceAccountId(yourServiceAccountId);
ipFinder.setServiceAccountP12FilePath(pathToYourP12Key);
ipFinder.setProjectName(yourGoogleCloudPlatformProjectName);

ipFinder.setBucketName("your_bucket_name");

spi.setIpFinder(ipFinder);

IgniteConfiguration cfg = new IgniteConfiguration();
 
// Override default discovery SPI.
cfg.setDiscoverySpi(spi);
 
// Start Ignite node.
Ignition.start(cfg);

That's all on the code side. Easy enough, right?
Now let's go through the parameters that have to be set in order to make the IP finder workable.

Considering that you're setting up everything from scratch perform the following steps:

• First you need to sign in Google Developer Console;

• Create a new project, open its "Overview" section, find "Project Number" and pass it to ipFinder.setProjectName() method call;

• Open "APIs and auth"->"Credentials" section and create a service account by pressing on "Create new Client ID" button. Set account's email address to ipFinder.setServiceAccountId(). Generate and download new account's P12 key and pass a full path to its location using ipFinder.setServiceAccountP12FilePath();

• Set unique Google Storage bucket name to ipFinder.setBucketName(). Note, that the name must be unique across the whole Google Cloud Platform and not just in your project.

So these are all the steps you need to do if you want to run a cluster of Ignite nodes in Compute Engine network with nodes auto-discovery enabled.

In the next post I will share another one generic solution that will enable nodes auto-discovery not only for Compute Engine network but for many other cloud platforms. Stay turned!