IMPORTANT:
a. Do not do this if the servers are already in domain and DNS is already handled by a dedicated DNS Server.
b. Its always better to assign static IP for the servers before doing these changes so that we don't chain the DNS entries every time the host machine IP changes due to restart.
c. You need to be an administrator to perform these changes.
Now that we have created these servers which are on the virtual network and have only an IPV4 number. We need to make sure that these machines in the network are accessible by its host name.
Most of the big data clusters need not be managed by a domain controller and can exist as standalone in the cloud or in your own virtualization host.
On each of the clusters the DNS entries for the other nodes that it uses in the cluster needs to be set in the /etc/hosts file.
For those clusters which are interacting with other clusters then the DNS entries needs to be set for all the other clusters the current cluster interacts with.
e.g
If we have a Elastic Search 3 node clusters then on each of the 3 node cluster we need to setup the DNS entries in /etc/hosts file
sudo vi /etc/hosts
Insert the following records and save on all the clusters. Keep the localhost entry as it is and append to it. (You can insert with the comment using the #)
if we have an Apache Storm 3 node clusters which inserts data to any of the Elastic Search clusters then on each of the Storm cluster we need to have both the entries for Storm nodes as well as the Elastic Search nodes.
sudo vi /etc/hosts
Insert the following records and save on all the clusters
Verify if its working by doing the following command
ping STORMNODE1
cancel by ctrl + c
You should not get any dropped packets.
a. Do not do this if the servers are already in domain and DNS is already handled by a dedicated DNS Server.
b. Its always better to assign static IP for the servers before doing these changes so that we don't chain the DNS entries every time the host machine IP changes due to restart.
c. You need to be an administrator to perform these changes.
Now that we have created these servers which are on the virtual network and have only an IPV4 number. We need to make sure that these machines in the network are accessible by its host name.
Most of the big data clusters need not be managed by a domain controller and can exist as standalone in the cloud or in your own virtualization host.
On each of the clusters the DNS entries for the other nodes that it uses in the cluster needs to be set in the /etc/hosts file.
For those clusters which are interacting with other clusters then the DNS entries needs to be set for all the other clusters the current cluster interacts with.
e.g
If we have a Elastic Search 3 node clusters then on each of the 3 node cluster we need to setup the DNS entries in /etc/hosts file
sudo vi /etc/hosts
Insert the following records and save on all the clusters. Keep the localhost entry as it is and append to it. (You can insert with the comment using the #)
# Elastic Nodes
192.168.0.6 SEARCHNODE1
192.168.0.7 SEARCHNODE2
192.168.0.8 SEARCHNODE3
if we have an Apache Storm 3 node clusters which inserts data to any of the Elastic Search clusters then on each of the Storm cluster we need to have both the entries for Storm nodes as well as the Elastic Search nodes.
sudo vi /etc/hosts
Insert the following records and save on all the clusters
# Storm Nodes
192.168.0.10 STORMNODE1
192.168.0.11 STORMNODE2
192.168.0.12 STORMNODE3
# Elastic Nodes
192.168.0.6 SEARCHNODE1
192.168.0.7 SEARCHNODE2
192.168.0.8 SEARCHNODE3
Verify if its working by doing the following command
ping STORMNODE1
cancel by ctrl + c
You should not get any dropped packets.
No comments:
Post a Comment