In October 2012, IBM announced the release of Virtual I/O Server(VIOS) 2.2.2.0. To read more about the major enhancements in this release, please refer to the release notes here : http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/topic/p7eeo/ipeeo_main.htm

Apart from the other major enhancements, VIOS version 2.2.2.0 has significant enhancements in the VIOS restricted shell networking commands to support configuration of multiple network interfaces.


The two major considerations with which these enhacements in the networking commands have been made are :
1. Configuration of additional network interfaces should not disturb the existing configuration and network connectivity of previosuly configured network interfaces of the VIOS.
2. The commands are made Shared Storage Pool(SSP) aware. Meaning, while you are restricted from altering the network attributes of the primary network interface used for SSP communication, there is no limitation on changing the SSP unrelated/unused attributes of a network interface.

The VIOS networking commands enhanced to support the above are :

For more details of changes made to each of these commands please go through the documentation of VIOS 2.2.2.0 here : http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/p7hcg/iphcgwhatsnew.htm

Another major announcement in VIOS Version 2.2.2.0 was the support of creating Shared Storage Pool(SSP) cluster over IPv6 network.
VIOS logical partitions in a Shared Storage Pools cluster can have hostnames that resolve to an IPv6 address.

Note: As per IBM documentation, it is suggested to use stateless autoconfiguration for IPv6 networks.

To create a SSP cluster over IPv6 network, the existing VIOS Command Line Interface(CLI) ‘cluster’ command will also support hostname that resolves to an IPv6 address or and IPv6 address for -hostname flag; in addition to the IPv4 address(supported previously).

From Virtual I/O Server (VIOS) Version 2.2.2.0 onwards, you can migrate an existing SSP cluster formed over Internet Protocol version 4 (IPv4) family to Internet Protocol version 6 (IPv6) family.
The steps on how to get it done are provided in the IBM documentation : http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/p7hb1/iphb1migrateipv4tov6cluster.htm

Rest of this article is a hands-on exercise of a ‘Step by Step guide on how to create a 2 node SSP cluster over IPv4 network and then use the steps provided in the documentation to migrate the SSP cluster from IPv4 to IPv6 address family‘.

The two VIOS partitions used here have hostnames of viosa and viosb.

(A) Create a Cluster with one node :  viosa

Command to display the Virtual I/O Server level :

Command to display the list of every known physical volume in the VIOS :

Command to display the list of physical volumes that can be used as a backing device (i.e. not already used as a backing device, not assigned to a shared memory pool or not already part of a volume group) :

Here we see that we have two physical volumes hdisk5 and hdisk6 which are not yet used, and can be used for Shared Storage Pool creation.

Command to display all of the networking interfaces configured on the VIOS :

In this setup, network interface en0 has an IPv4 address configured on it.

 

Command to display the hostname of the VIOS :

The IP address to which the the hostname of the VIOS resolves to can be determined by using the ‘host’ command of the root-shell. (Note: The ‘oem_setup_env’ command will place the padmin user of VIOS in non-restricted AIX root shell. Be careful of any command executed in the root-shell. It is not a supported Virtual I/O Server administration method). Both the forward and reverse lookup of hostname/IP address must be checked to confirm that the name resolution configurations are proper.

Command to create a SSP cluster named ‘test_cl’, using ‘hdisk5’ as repository physical volume and ‘hdisk6’ as the physical volume on which a shared storage pool can be created.

The details of SSP cluster can be determined by using the ‘-status’ flag of ‘cluster’ command :

The cluster configuration information can also be determined by using the ‘lscluster’ command :

 

B. Add another VIOS partition ‘viosb’ to the SSP cluster ‘test_cl’

Before adding a VIOS to an existing SSP cluster, you must take care of two things :

1. Make sure that the VIOS has access to the same physical volumes that are being used as Repository and Shared Storage Pool disks of the SSP.

The provisioning of the same disks to the VIOS has to be done by the SAN administrator. To check if the same disks are available on the added VIOS, use the ‘lsdev’ command.

Command to display the unique_id attribute of a physical volume :

Match the unique_id attribute of the PVs known to the VIOS with that of the PV used by SSP to confirm that the same PVs are also available on the (to be added) VIOS.

2. Make sure that the VIOS is able to gain network connectivity to the other VIOS(es) that are part of the SSP.

These commands can be used to test the connectivity of one VIOS to the other VIOS nodes.

In the root-shell determine the hostname/IP resolution of VIOS node :

Now perform a ping test to confirm that viosb is able to connect to viosa over network.

Now once it is confirmed that ‘viosb’ is eligible to be part of the same cluster ‘test_cl’ comprising of ‘viosa’, use the ‘cluster’ command on ‘viosa’ with ‘-addnode’ option to add a new node ‘viosb’ to the SSP cluster.

Use the ‘cluster -status’ command to check the details of the cluster.

From the command output above, we see that the SSP cluster presently has 2 VIOS nodes, with successful addition of ‘viosb’ to the cluster.

With a 2 node Shared Storage Pool ready, the next task is to migrate the existing SSP from IPv4 address family to IPv6 address family.

C : MIgrate the cluster ‘test_cl’ from IPv4 address family to IPv6 address family
Follow the steps in the IBM documentation on ‘Migrating a cluster from IPv4 to IPv6’ : http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/p7hb1/iphb1migrateipv4tov6cluster.htm

The important things to take care of are :
1.   You must not change the IP address of a VIOS logical partition in a cluster that resolves to the host name dynamically.
2.  You can migrate an existing cluster that is using IPv4 addresses to a cluster that is using IPv6 addresses only after each of the VIOS logical partitions are updated to VIOS Version 2.2.2.0, or later.

To migrate a SSP cluster from IPv4 to IPv6 (major steps in green):
  (1) On the VIOS command line, type mktcpip to add an IPv6 address to each of the VIOS logical partitions that are in the IPv4 cluster.
Note: Do not remove the IPv4 addresses that the host name of each VIOS logical partition are resolving to until after you complete step (2) for all VIOS logical partitions.

    On node ‘viosa’, use the VIOS ‘mktcpip’ command to run stateless autoconfiguration of IPv6 link-local addresses :

    Check the IP addresses configured on the interfaces by using the ‘lstcpip’ command with the ‘-interfaces’ option :

    Execute the same command on ‘viosb’ to get an autoconfigured IPv6 stateless address :

For an autoconfigured IPv6 stateless address,  mktcpip command would not created a mapping of hostname to IP address.
For us to map the VIOServer hostname to the autoconfigured IPv6 address we would be required to manually edit the ‘/etc/hosts’ file of VIOS.
To edit the ‘/etc/hosts’ file we need to get into the non-restricted root shell of VIOS by typing the command :

Note: We have just yet made sure that IPv6 addresses are configured on all the VIOServeres which are part of the Shared Storage Pool Cluster. We have not yet removed the IPv4 addresses of the VIOServers nor have we removed the hostname to IPv4 addresses resolution. The hostnames should still resolve to IPv4 addresses, or else the Shared Storage Pool would go into a bad state.

 

(2)    Complete the following steps on each VIOS logical partition in the cluster:
       a.  Stop cluster services on the VIOS logical partition by running the following command:

        clstartstop -stop -n clustername -m node_hostname

       b.  Make the required changes in the network configuration, Neighbor Discovery Protocol (NDP) daemon router, or Domain Name System (DNS) information so that the IPv6 address of the VIOS logical partition resolves to the same host name that earlier resolved to the IPv4 address. Ensure that both the forward and reverse lookup for the same host name resolves to the required IPv6 address.
        c.  On the VIOS command line, type the following command to restart cluster services on the VIOS logical partition:

        clstartstop -start -n clustername -m node_hostname

        Repeat steps 2a – 2c for each VIOS logical partition that belongs to the cluster.

Use the ‘clstartstop -stop’ command to stop cluster services on the VIOS logical partition ‘viosa’ :

lscluster command confirms the stopping of clustering services on ‘viosa’ :

You can confirm if cluster services on ‘viosa’ has been stopped by using the ‘cluster -status’ command on ‘viosb’:

Now, as a next step make the required changes on all the VIOS nodes in the SSP cluster; in our case ‘viosa’ and ‘viosb’ to resolve the hostname of ‘viosa’ to the IPv6 address.

In this case, go to the root-shell(oem_setup_env) and edit /etc/hosts file such that ‘viosa’ resolves to it’s IPv6 address. In this case, I removed the IPv4 address to hostname ‘viosa’ mapping from the /etc/hosts file.This needs to be done on both ‘viosa’ and ‘viosb’.

Now make sure that both the forward and reverse lookup of ‘viosa’ resolves to the IPv6 address.

After that, start cluster services of ‘viosa’ by running this command on ‘viosb’ :

‘cluster -status’ command shows if the cluster services on viosa  has been successfully started :

Steps 2a) to 2c) needs to be done on ‘viosb’, such that ‘viosb’ is also migrated to IPv6 address.

(3)    From the VIOS command line, type rmtcpip to remove the IPv4 address from each VIOS logical partition.

As, was seen earlier ‘viosa’ has both IPv4 and IPv6 addresses configured on the interface ‘en0’ :

In this release, ‘rmtcpip’ command has been made Shared Storage Pool(SSP) aware. It means the command would not allow removal of network attributes of the primary interface being used for SSP cluster communication.

Now, in my example the SSP cluster is presently using IPv6 address as it’s primary network interface; let’s try a few rmtcpip commands :

a) Trying to remove all the network attributes of interface ‘en0’ fails, because it is being used for SSP cluster :


b) Trying to remove the IPv6 address of interface ‘en0’ fails, because it is being used for SSP cluster :

c) Trying to remove the IPv4 address of interface ‘en0’ succeeds. The reason being IPv4 address of interface ‘en0’ is not being used for Shared Storage Pool cluster :

The migration of Shared Storage Pool cluster from IPv4 to IPv6 address family is complete.

 

Important points to take care of while setting up the networking configuration for a Shared Storage Pool cluster

  1.  Ensure that “both the forward and reverse lookup” of the hostname used by the VIOS logical partition for clustering resolves to the same IP address.
  2.  The hostname of each VIOS logical partition that belongs to the same SSP cluster must resolve to the same IP address family, which is “either” IPv4 or IPv6.
  3.  In a SSP cluster configuration, you “cannot” change the host name of a VIOS logical partition, after the Shared Storage Pool is created.
  4.  If any change to /etc/netsvc.conf file of VIOS logical partition is required, it should be made “before creating” the SSP cluster. This file is used to specify the ordering of name resolution for networking routines and commands.

 

 

Reference/Useful links :

Virtual I/O Server release notes : http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/p7eeo/ipeeokickoff.htm

 

What do you think ?

Set your Twitter account name in your settings to use the TwitterBar Section.
%d bloggers like this: