This post I want to dedicate to one useful feature of Apache Ignite that I had a chance to contribute to the project and that became a part of it since release 1.1.0-incubating.
Actually, this will be my first post about this Apache project and for those who are unfamiliar with it I would recommend to overview a good quick start documentation that can be found here.
The feature I'm going to cover is related to nodes discovery process located in a cluster in case when the cluster is a set of virtual machines deployed on Google Compute Engine. Historically Apache Ignite offered us two ways for the discovery process - leveraging multicast protocol or pre-configuring nodes' IP addresses. Unfortunately, neither way works well for Google VMs (virtual machines) instances: multicast is not supported in Compute Engine network at all and static IP configuration is not flexible (machine IP address may change from time to time, machines can be removed or added to a cluster, etc.).
However, there is a solution to this problem called TcpDiscoveryGoogleStorageIpFinder.
This IP finder does exactly what its name says. It lets virtual nodes located in Compute Engine network discover each other automatically using Google Cloud Storage where Apache Ignite's kernal will keep track of all cluster's nodes.
What you need to do to activate this feature is set TcpDiscoveryGoogleStorageIpFinder as an IP finder for a VM node.
That's all on the code side. Easy enough, right?
Now let's go through the parameters that have to be set in order to make the IP finder workable.
Considering that you're setting up everything from scratch perform the following steps:
So these are all the steps you need to do if you want to run a cluster of Ignite nodes in Compute Engine network with nodes auto-discovery enabled.
In the next post I will share another one generic solution that will enable nodes auto-discovery not only for Compute Engine network but for many other cloud platforms. Stay turned!
Actually, this will be my first post about this Apache project and for those who are unfamiliar with it I would recommend to overview a good quick start documentation that can be found here.
The feature I'm going to cover is related to nodes discovery process located in a cluster in case when the cluster is a set of virtual machines deployed on Google Compute Engine. Historically Apache Ignite offered us two ways for the discovery process - leveraging multicast protocol or pre-configuring nodes' IP addresses. Unfortunately, neither way works well for Google VMs (virtual machines) instances: multicast is not supported in Compute Engine network at all and static IP configuration is not flexible (machine IP address may change from time to time, machines can be removed or added to a cluster, etc.).
However, there is a solution to this problem called TcpDiscoveryGoogleStorageIpFinder.
This IP finder does exactly what its name says. It lets virtual nodes located in Compute Engine network discover each other automatically using Google Cloud Storage where Apache Ignite's kernal will keep track of all cluster's nodes.
What you need to do to activate this feature is set TcpDiscoveryGoogleStorageIpFinder as an IP finder for a VM node.
TcpDiscoverySpi spi = new TcpDiscoverySpi(); TcpDiscoveryGoogleStorageIpFinder ipFinder = new TcpDiscoveryGoogleStorageIpFinder(); ipFinder.setServiceAccountId(yourServiceAccountId); ipFinder.setServiceAccountP12FilePath(pathToYourP12Key); ipFinder.setProjectName(yourGoogleCloudPlatformProjectName); ipFinder.setBucketName("your_bucket_name"); spi.setIpFinder(ipFinder); IgniteConfiguration cfg = new IgniteConfiguration(); // Override default discovery SPI. cfg.setDiscoverySpi(spi); // Start Ignite node. Ignition.start(cfg);
That's all on the code side. Easy enough, right?
Now let's go through the parameters that have to be set in order to make the IP finder workable.
Considering that you're setting up everything from scratch perform the following steps:
- First you need to sign in Google Developer Console;
- Create a new project, open its "Overview" section, find "Project Number" and pass it to ipFinder.setProjectName() method call;
- Activate Google Cloud Storage API by referring to this page and Google Cloud Storage JSON API by going to "APIs and auth"->"APIs" section in your console;
- Open "APIs and auth"->"Credentials" section and create a service account by pressing on "Create new Client ID" button. Set account's email address to ipFinder.setServiceAccountId(). Generate and download new account's P12 key and pass a full path to its location using ipFinder.setServiceAccountP12FilePath();
- Set unique Google Storage bucket name to ipFinder.setBucketName(). Note, that the name must be unique across the whole Google Cloud Platform and not just in your project.
So these are all the steps you need to do if you want to run a cluster of Ignite nodes in Compute Engine network with nodes auto-discovery enabled.
In the next post I will share another one generic solution that will enable nodes auto-discovery not only for Compute Engine network but for many other cloud platforms. Stay turned!
This comment has been removed by the author.
ReplyDelete