High availability of SAP HANA on Azure VMs on Red Hat Enterprise Linux

For on-premises development, you can use either HANA System Replication or use shared storage to establish high availability for SAP HANA. On Azure virtual machines (VMs), HANA System Replication on Azure is currently the only supported high availability function. SAP HANA Replication consists of one primary node and at least one secondary node. Changes to the data on the primary node are replicated to the secondary node synchronously or asynchronously.

This article describes how to deploy and configure the virtual machines, install the cluster framework, and install and configure SAP HANA System Replication. In the example configurations, installation commands, instance number 03, and HANA System ID HN1 are used.

Read the following SAP Notes and papers first:

Overview

To achieve high availability, SAP HANA is installed on two virtual machines. The data is replicated by using HANA System Replication.

SAP HANA high availability overview

SAP HANA System Replication setup uses a dedicated virtual hostname and virtual IP addresses. On Azure, a load balancer is required to use a virtual IP address. The following list shows the configuration of the load balancer:

  • Front-end configuration: IP address 10.0.0.13 for hn1-db
  • Back-end configuration: Connected to primary network interfaces of all virtual machines that should be part of HANA System Replication
  • Probe Port: Port 62503
  • Load-balancing rules: 30313 TCP, 30315 TCP, 30317 TCP, 30340 TCP, 30341 TCP, 30342 TCP

Deploy for Linux

The Azure Marketplace contains an image for Red Hat Enterprise Linux 7.4 for SAP HANA that you can use to deploy new virtual machines.

Deploy with a template

You can use one of the quickstart templates that are on GitHub to deploy all the required resources. The template deploys the virtual machines, the load balancer, the availability set, and so on. To deploy the template, follow these steps:

  1. Open the database template on the Azure portal.
  2. Enter the following parameters:
    • Sap System ID: Enter the SAP system ID of the SAP system you want to install. The ID is used as a prefix for the resources that are deployed.
    • Os Type: Select one of the Linux distributions. For this example, select RHEL 7.
    • Db Type: Select HANA.
    • Sap System Size: Enter the number of SAPS that the new system is going to provide. If you're not sure how many SAPS the system requires, ask your SAP Technology Partner or System Integrator.
    • System Availability: Select HA.
    • Admin Username, Admin Password or SSH key: A new user is created that can be used to sign in to the machine.
    • Subnet ID: If you want to deploy the VM into an existing VNet where you have a subnet defined the VM should be assigned to, name the ID of that specific subnet. The ID usually looks like /subscriptions/<subscription ID>/resourceGroups/<resource group name>/providers/Microsoft.Network/virtualNetworks/<virtual network name>/subnets/<subnet name>. Leave empty, if you want to create a new virtual network

Manual deployment

  1. Create a resource group.

  2. Create a virtual network.

  3. Create an availability set.
    Set the max update domain.

  4. Create a load balancer (internal).

    • Select the virtual network created in step 2.
  5. Create virtual machine 1.
    Use at least Red Hat Enterprise Linux 7.4 for SAP HANA. This example uses the Red Hat Enterprise Linux 7.4 for SAP HANA image https://portal.azure.com/#create/RedHat.RedHatEnterpriseLinux75forSAP-ARM Select the availability set created in step 3.

  6. Create virtual machine 2.
    Use at least Red Hat Enterprise Linux 7.4 for SAP HANA. This example uses the Red Hat Enterprise Linux 7.4 for SAP HANA image https://portal.azure.com/#create/RedHat.RedHatEnterpriseLinux75forSAP-ARM Select the availability set created in step 3.

  7. Add data disks.

  8. Configure the load balancer. First, create a front-end IP pool:

    1. Open the load balancer, select frontend IP pool, and select Add.
    2. Enter the name of the new front-end IP pool (for example, hana-frontend).
    3. Set the Assignment to Static and enter the IP address (for example, 10.0.0.13).
    4. Select OK.
    5. After the new front-end IP pool is created, note the pool IP address.
  9. Next, create a back-end pool:

    1. Open the load balancer, select backend pools, and select Add.
    2. Enter the name of the new back-end pool (for example, hana-backend).
    3. Select Add a virtual machine.
    4. Select the availability set created in step 3.
    5. Select the virtual machines of the SAP HANA cluster.
    6. Select OK.
  10. Next, create a health probe:

    1. Open the load balancer, select health probes, and select Add.
    2. Enter the name of the new health probe (for example, hana-hp).
    3. Select TCP as the protocol and port 62503. Keep the Interval value set to 5, and the Unhealthy threshold value set to 2.
    4. Select OK.
  11. For SAP HANA 1.0, create the load-balancing rules:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30315).
    3. Select the front-end IP address, the back-end pool, and the health probe that you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30315.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for port 30317.
  12. For SAP HANA 2.0, create the load-balancing rules for the system database:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30313).
    3. Select the front-end IP address, the back-end pool, and the health probe that you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30313.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for port 30314.
  13. For SAP HANA 2.0, first create the load-balancing rules for the tenant database:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30340).
    3. Select the frontend IP address, backend pool, and health probe you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30340.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for ports 30341 and 30342.

For more information about the required ports for SAP HANA, read the chapter Connections to Tenant Databases in the SAP HANA Tenant Databases guide or SAP Note 2388694.

Important

Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter net.ipv4.tcp_timestamps to 0. For details see Load Balancer health probes. See also SAP note 2382421.

Install SAP HANA

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [A] Set up the disk layout: Logical Volume Manager (LVM).

    We recommend that you use LVM for volumes that store data and log files. The following example assumes that the virtual machines have four data disks attached that are used to create two volumes.

    List all of the available disks:

    ls /dev/disk/azure/scsi1/lun*
    

    Example output:

    
    /dev/disk/azure/scsi1/lun0  /dev/disk/azure/scsi1/lun1  /dev/disk/azure/scsi1/lun2  /dev/disk/azure/scsi1/lun3
    

    Create physical volumes for all of the disks that you want to use:

    sudo pvcreate /dev/disk/azure/scsi1/lun0
    sudo pvcreate /dev/disk/azure/scsi1/lun1
    sudo pvcreate /dev/disk/azure/scsi1/lun2
    sudo pvcreate /dev/disk/azure/scsi1/lun3
    

    Create a volume group for the data files. Use one volume group for the log files and one for the shared directory of SAP HANA:

    sudo vgcreate vg_hana_data_HN1 /dev/disk/azure/scsi1/lun0 /dev/disk/azure/scsi1/lun1
    sudo vgcreate vg_hana_log_HN1 /dev/disk/azure/scsi1/lun2
    sudo vgcreate vg_hana_shared_HN1 /dev/disk/azure/scsi1/lun3
    

    Create the logical volumes. A linear volume is created when you use lvcreate without the -i switch. We suggest that you create a striped volume for better I/O performance, where the -i argument should be the number of the underlying physical volume. In this document, two physical volumes are used for the data volume, so the -i switch argument is set to 2. One physical volume is used for the log volume, so no -i switch is explicitly used. Use the -i switch and set it to the number of the underlying physical volume when you use more than one physical volume for each data, log, or shared volumes.

    sudo lvcreate -i 2 -l 100%FREE -n hana_data vg_hana_data_HN1
    sudo lvcreate -l 100%FREE -n hana_log vg_hana_log_HN1
    sudo lvcreate -l 100%FREE -n hana_shared vg_hana_shared_HN1
    sudo mkfs.xfs /dev/vg_hana_data_HN1/hana_data
    sudo mkfs.xfs /dev/vg_hana_log_HN1/hana_log
    sudo mkfs.xfs /dev/vg_hana_shared_HN1/hana_shared
    

    Create the mount directories and copy the UUID of all of the logical volumes:

    sudo mkdir -p /hana/data/HN1
    sudo mkdir -p /hana/log/HN1
    sudo mkdir -p /hana/shared/HN1
    # Write down the ID of /dev/vg_hana_data_HN1/hana_data, /dev/vg_hana_log_HN1/hana_log, and /dev/vg_hana_shared_HN1/hana_shared
    sudo blkid
    

    Create fstab entries for the three logical volumes:

    sudo vi /etc/fstab
    

    Insert the following line in the /etc/fstab file:

    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_data_HN1-hana_data> /hana/data/HN1 xfs  defaults,nofail  0  2
    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_log_HN1-hana_log> /hana/log/HN1 xfs  defaults,nofail  0  2
    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_shared_HN1-hana_shared> /hana/shared/HN1 xfs  defaults,nofail  0  2
    

    Mount the new volumes:

    sudo mount -a
    
  2. [A] Set up the disk layout: Plain Disks.

    For demo systems, you can place your HANA data and log files on one disk. Create a partition on /dev/disk/azure/scsi1/lun0 and format it with xfs:

    sudo sh -c 'echo -e "n\n\n\n\n\nw\n" | fdisk /dev/disk/azure/scsi1/lun0'
    sudo mkfs.xfs /dev/disk/azure/scsi1/lun0-part1
    
    # Write down the ID of /dev/disk/azure/scsi1/lun0-part1
    sudo /sbin/blkid
    sudo vi /etc/fstab
    

    Insert this line in the /etc/fstab file:

    /dev/disk/by-uuid/<UUID> /hana xfs  defaults,nofail  0  2
    

    Create the target directory and mount the disk:

    sudo mkdir /hana
    sudo mount -a
    
  3. [A] Set up host name resolution for all hosts.

    You can either use a DNS server or modify the /etc/hosts file on all nodes. This example shows you how to use the /etc/hosts file. Replace the IP address and the hostname in the following commands:

    sudo vi /etc/hosts
    

    Insert the following lines in the /etc/hosts file. Change the IP address and hostname to match your environment:

    10.0.0.5 hn1-db-0
    10.0.0.6 hn1-db-1
    
  4. [A] RHEL for HANA configuration

    Configure RHEL as described in SAP Note 2292690 and 2455582 and https://access.redhat.com/solutions/2447641.

  5. [A] Install the SAP HANA

    To install SAP HANA System Replication, follow https://access.redhat.com/articles/3004101.

    • Run the hdblcm program from the HANA DVD. Enter the following values at the prompt:
    • Choose installation: Enter 1.
    • Select additional components for installation: Enter 1.
    • Enter Installation Path [/hana/shared]: Select Enter.
    • Enter Local Host Name [..]: Select Enter.
    • Do you want to add additional hosts to the system? (y/n) [n]: Select Enter.
    • Enter SAP HANA System ID: Enter the SID of HANA, for example: HN1.
    • Enter Instance Number [00]: Enter the HANA Instance number. Enter 03 if you used the Azure template or followed the manual deployment section of this article.
    • Select Database Mode / Enter Index [1]: Select Enter.
    • Select System Usage / Enter Index [4]: Select the system usage value.
    • Enter Location of Data Volumes [/hana/data/HN1]: Select Enter.
    • Enter Location of Log Volumes [/hana/log/HN1]: Select Enter.
    • Restrict maximum memory allocation? [n]: Select Enter.
    • Enter Certificate Host Name For Host '...' [...]: Select Enter.
    • Enter SAP Host Agent User (sapadm) Password: Enter the host agent user password.
    • Confirm SAP Host Agent User (sapadm) Password: Enter the host agent user password again to confirm.
    • Enter System Administrator (hdbadm) Password: Enter the system administrator password.
    • Confirm System Administrator (hdbadm) Password: Enter the system administrator password again to confirm.
    • Enter System Administrator Home Directory [/usr/sap/HN1/home]: Select Enter.
    • Enter System Administrator Login Shell [/bin/sh]: Select Enter.
    • Enter System Administrator User ID [1001]: Select Enter.
    • Enter ID of User Group (sapsys) [79]: Select Enter.
    • Enter Database User (SYSTEM) Password: Enter the database user password.
    • Confirm Database User (SYSTEM) Password: Enter the database user password again to confirm.
    • Restart system after machine reboot? [n]: Select Enter.
    • Do you want to continue? (y/n): Validate the summary. Enter y to continue.
  6. [A] Upgrade the SAP Host Agent.

    Download the latest SAP Host Agent archive from the SAP Software Center and run the following command to upgrade the agent. Replace the path to the archive to point to the file that you downloaded:

    sudo /usr/sap/hostctrl/exe/saphostexec -upgrade -archive <path to SAP Host Agent SAR>
    
  7. [A] Configure firewall

    Create the firewall rule for the Azure load balancer probe port.

    sudo firewall-cmd --zone=public --add-port=62503/tcp
    sudo firewall-cmd --zone=public --add-port=62503/tcp --permanent
    

Configure SAP HANA 2.0 System Replication

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [A] Configure firewall

    Create firewall rules to allow HANA System Replication and client traffic. The required ports are listed on TCP/IP Ports of All SAP Products. The following commands are just an example to allow HANA 2.0 System Replication and client traffic to database SYSTEMDB, HN1 and NW1.

    sudo firewall-cmd --zone=public --add-port=40302/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40302/tcp
    sudo firewall-cmd --zone=public --add-port=40301/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40301/tcp
    sudo firewall-cmd --zone=public --add-port=40307/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40307/tcp
    sudo firewall-cmd --zone=public --add-port=40303/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40303/tcp
    sudo firewall-cmd --zone=public --add-port=40340/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40340/tcp
    sudo firewall-cmd --zone=public --add-port=30340/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=30340/tcp
    sudo firewall-cmd --zone=public --add-port=30341/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=30341/tcp
    sudo firewall-cmd --zone=public --add-port=30342/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=30342/tcp
    
  2. [1] Create the tenant database.

    If you're using SAP HANA 2.0 or MDC, create a tenant database for your SAP NetWeaver system. Replace NW1 with the SID of your SAP system.

    Execute as <hanasid>adm the following command:

    hdbsql -u SYSTEM -p "passwd" -i 03 -d SYSTEMDB 'CREATE DATABASE NW1 SYSTEM USER PASSWORD "passwd"'
    
  3. [1] Configure System Replication on the first node:

    Backup the databases as <hanasid>adm:

    hdbsql -d SYSTEMDB -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupSYS')"
    hdbsql -d HN1 -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupHN1')"
    hdbsql -d NW1 -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupNW1')"
    

    Copy the system PKI files to the secondary site:

    scp /usr/sap/HN1/SYS/global/security/rsecssfs/data/SSFS_HN1.DAT   hn1-db-1:/usr/sap/HN1/SYS/global/security/rsecssfs/data/
    scp /usr/sap/HN1/SYS/global/security/rsecssfs/key/SSFS_HN1.KEY  hn1-db-1:/usr/sap/HN1/SYS/global/security/rsecssfs/key/
    

    Create the primary site:

    hdbnsutil -sr_enable --name=SITE1
    
  4. [2] Configure System Replication on the second node:

    Register the second node to start the system replication. Run the following command as <hanasid>adm:

    sapcontrol -nr 03 -function StopWait 600 10
    hdbnsutil -sr_register --remoteHost=hn1-db-0 --remoteInstance=03 --replicationMode=sync --name=SITE2
    
  5. [1] Check replication status

    Check the replication status and wait until all databases are in sync. If the status remains UNKNOWN, check your firewall settings.

    sudo su - hn1adm -c "python /usr/sap/HN1/HDB03/exe/python_support/systemReplicationStatus.py"
    # | Database | Host     | Port  | Service Name | Volume ID | Site ID | Site Name | Secondary | Secondary | Secondary | Secondary | Secondary     | Replication | Replication | Replication    |
    # |          |          |       |              |           |         |           | Host      | Port      | Site ID   | Site Name | Active Status | Mode        | Status      | Status Details |
    # | -------- | -------- | ----- | ------------ | --------- | ------- | --------- | --------- | --------- | --------- | --------- | ------------- | ----------- | ----------- | -------------- |
    # | SYSTEMDB | hn1-db-0 | 30301 | nameserver   |         1 |       1 | SITE1     | hn1-db-1  |     30301 |         2 | SITE2     | YES           | SYNC        | ACTIVE      |                |
    # | HN1      | hn1-db-0 | 30307 | xsengine     |         2 |       1 | SITE1     | hn1-db-1  |     30307 |         2 | SITE2     | YES           | SYNC        | ACTIVE      |                |
    # | NW1      | hn1-db-0 | 30340 | indexserver  |         2 |       1 | SITE1     | hn1-db-1  |     30340 |         2 | SITE2     | YES           | SYNC        | ACTIVE      |                |
    # | HN1      | hn1-db-0 | 30303 | indexserver  |         3 |       1 | SITE1     | hn1-db-1  |     30303 |         2 | SITE2     | YES           | SYNC        | ACTIVE      |                |
    #
    # status system replication site "2": ACTIVE
    # overall system replication status: ACTIVE
    #
    # Local System Replication State
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #
    # mode: PRIMARY
    # site id: 1
    # site name: SITE1
    

Configure SAP HANA 1.0 System Replication

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [A] Configure firewall

    Create firewall rules to allow HANA System Replication and client traffic. The required ports are listed on TCP/IP Ports of All SAP Products. The following commands are just an example to allow HANA 2.0 System Replication. Adapt it to your SAP HANA 1.0 installation.

    sudo firewall-cmd --zone=public --add-port=40302/tcp --permanent
    sudo firewall-cmd --zone=public --add-port=40302/tcp
    
  2. [1] Create the required users.

    Run the following command as root. Make sure to replace bold strings (HANA System ID HN1 and instance number 03) with the values of your SAP HANA installation:

    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbsql -u system -i 03 'CREATE USER hdbhasync PASSWORD "passwd"'
    hdbsql -u system -i 03 'GRANT DATA ADMIN TO hdbhasync'
    hdbsql -u system -i 03 'ALTER USER hdbhasync DISABLE PASSWORD LIFETIME'
    
  3. [A] Create the keystore entry.

    Run the following command as root to create a new keystore entry:

    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbuserstore SET hdbhaloc localhost:30315 hdbhasync passwd
    
  4. [1] Back up the database.

    Back up the databases as root:

    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbsql -d SYSTEMDB -u system -i 03 "BACKUP DATA USING FILE ('initialbackup')"
    

    If you use a multi-tenant installation, also back up the tenant database:

    hdbsql -d HN1 -u system -i 03 "BACKUP DATA USING FILE ('initialbackup')"
    
  5. [1] Configure System Replication on the first node.

    Create the primary site as <hanasid>adm:

    su - hdbadm
    hdbnsutil -sr_enable –-name=SITE1
    
  6. [2] Configure System Replication on the secondary node.

    Register the secondary site as <hanasid>adm:

    HDB stop
    hdbnsutil -sr_register --remoteHost=hn1-db-0 --remoteInstance=03 --replicationMode=sync --name=SITE2
    HDB start
    

Create a Pacemaker cluster

Follow the steps in Setting up Pacemaker on Red Hat Enterprise Linux in Azure to create a basic Pacemaker cluster for this HANA server.

Create SAP HANA cluster resources

Install the SAP HANA resource agents on all nodes. Make sure to enable a repository that contains the package.

# Enable repository that contains SAP HANA resource agents
sudo subscription-manager repos --enable="rhel-sap-hana-for-rhel-7-server-rpms"
   
sudo yum install -y resource-agents-sap-hana

Next, create the HANA topology. Run the following commands on one of the Pacemaker cluster nodes:

sudo pcs property set maintenance-mode=true

# Replace the bold string with your instance number and HANA system ID
sudo pcs resource create SAPHanaTopology_HN1_03 SAPHanaTopology SID=HN1 InstanceNumber=03 --clone clone-max=2 clone-node-max=1 interleave=true

Next, create the HANA resources:

# Replace the bold string with your instance number, HANA system ID, and the front-end IP address of the Azure load balancer.

sudo pcs resource create SAPHana_HN1_03 SAPHana SID=HN1 InstanceNumber=03 PREFER_SITE_TAKEOVER=true DUPLICATE_PRIMARY_TIMEOUT=7200 AUTOMATED_REGISTER=false master notify=true clone-max=2 clone-node-max=1 interleave=true

sudo pcs resource create vip_HN1_03 IPaddr2 ip="10.0.0.13"

sudo pcs resource create nc_HN1_03 azure-lb port=62503

sudo pcs resource group add g_ip_HN1_03 nc_HN1_03 vip_HN1_03

sudo pcs constraint order SAPHanaTopology_HN1_03-clone then SAPHana_HN1_03-master symmetrical=false

sudo pcs constraint colocation add g_ip_HN1_03 with master SAPHana_HN1_03-master 4000

sudo pcs property set maintenance-mode=false

Make sure that the cluster status is ok and that all of the resources are started. It's not important on which node the resources are running.

sudo pcs status

# Online: [ hn1-db-0 hn1-db-1 ]
#
# Full list of resources:
#
# azure_fence     (stonith:fence_azure_arm):      Started hn1-db-0
#  Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
#      Started: [ hn1-db-0 hn1-db-1 ]
#  Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
#      Masters: [ hn1-db-0 ]
#      Slaves: [ hn1-db-1 ]
#  Resource Group: g_ip_HN1_03
#      nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-0
#      vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-0

Test the cluster setup

This section describes how you can test your setup. Before you start a test, make sure that Pacemaker does not have any failed action (via pcs status), there are no unexpected location constraints (for example leftovers of a migration test) and that HANA is sync state, for example with systemReplicationStatus:

[root@hn1-db-0 ~]# sudo su - hn1adm -c "python /usr/sap/HN1/HDB03/exe/python_support/systemReplicationStatus.py"

Test the migration

Resource state before starting the test:

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-0 ]
    Slaves: [ hn1-db-1 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-0
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-0

You can migrate the SAP HANA master node by executing the following command:

[root@hn1-db-0 ~]# pcs resource move SAPHana_HN1_03-master

If you set AUTOMATED_REGISTER="false", this command should migrate the SAP HANA master node and the group that contains the virtual IP address to hn1-db-1.

Once the migration is done, the 'sudo pcs status' output looks like this

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-1 ]
    Stopped: [ hn1-db-0 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-1
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-1

The SAP HANA resource on hn1-db-0 is stopped. In this case, configure the HANA instance as secondary by executing this command:

[root@hn1-db-0 ~]# su - hn1adm

# Stop the HANA instance just in case it is running
hn1adm@hn1-db-0:/usr/sap/HN1/HDB03> sapcontrol -nr 03 -function StopWait 600 10
hn1adm@hn1-db-0:/usr/sap/HN1/HDB03> hdbnsutil -sr_register --remoteHost=hn1-db-1 --remoteInstance=03 --replicationMod
e=sync --name=SITE1

The migration creates location constraints that need to be deleted again:

# Switch back to root
exit
[root@hn1-db-0 ~]# pcs resource clear SAPHana_HN1_03-master

Monitor the state of the HANA resource using 'pcs status'. Once HANA is started on hn1-db-0, the output should look like this

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-1 ]
    Slaves: [ hn1-db-0 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-1
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-1

Test the Azure fencing agent

Resource state before starting the test:

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-1 ]
    Slaves: [ hn1-db-0 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-1
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-1

You can test the setup of the Azure fencing agent by disabling the network interface on the node where SAP HANA is running as Master. See Red Hat Knowledgebase article 79523 for a description on how to simulate a network failure. In this example we use the net_breaker script to block all access to the network.

[root@hn1-db-1 ~]# sh ./net_breaker.sh BreakCommCmd 10.0.0.6

The virtual machine should now restart or stop depending on your cluster configuration. If you set the stonith-action setting to off, the virtual machine is stopped and the resources are migrated to the running virtual machine.

Note

It can take up to 15 minutes until the virtual machines is online again.

After you start the virtual machine again, the SAP HANA resource fails to start as secondary if you set AUTOMATED_REGISTER="false". In this case, configure the HANA instance as secondary by executing this command:

su - hn1adm

# Stop the HANA instance just in case it is running
hn1adm@hn1-db-1:/usr/sap/HN1/HDB03> sapcontrol -nr 03 -function StopWait 600 10
hn1adm@hn1-db-1:/usr/sap/HN1/HDB03> hdbnsutil -sr_register --remoteHost=hn1-db-0 --remoteInstance=03 --replicationMode=sync --name=SITE2

# Switch back to root and clean up the failed state
exit
[root@hn1-db-1 ~]# pcs resource cleanup SAPHana_HN1_03-master

Resource state after the test:

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-0 ]
    Slaves: [ hn1-db-1 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-0
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-0

Test a manual failover

Resource state before starting the test:

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-0 ]
    Slaves: [ hn1-db-1 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-0
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-0

You can test a manual failover by stopping the cluster on the hn1-db-0 node:

[root@hn1-db-0 ~]# pcs cluster stop

After the failover, you can start the cluster again. If you set AUTOMATED_REGISTER="false", the SAP HANA resource on the hn1-db-0 node fails to start as secondary. In this case, configure the HANA instance as secondary by executing this command:

[root@hn1-db-0 ~]# pcs cluster start
[root@hn1-db-0 ~]# su - hn1adm

# Stop the HANA instance just in case it is running
hn1adm@hn1-db-0:/usr/sap/HN1/HDB03> sapcontrol -nr 03 -function StopWait 600 10
hn1adm@hn1-db-0:/usr/sap/HN1/HDB03> hdbnsutil -sr_register --remoteHost=hn1-db-1 --remoteInstance=03 --replicationMode=sync --name=SITE1

# Switch back to root and clean up the failed state
hn1adm@hn1-db-0:/usr/sap/HN1/HDB03> exit
[root@hn1-db-1 ~]# pcs resource cleanup SAPHana_HN1_03-master

Resource state after the test:

Clone Set: SAPHanaTopology_HN1_03-clone [SAPHanaTopology_HN1_03]
    Started: [ hn1-db-0 hn1-db-1 ]
Master/Slave Set: SAPHana_HN1_03-master [SAPHana_HN1_03]
    Masters: [ hn1-db-1 ]
     Slaves: [ hn1-db-0 ]
Resource Group: g_ip_HN1_03
    nc_HN1_03  (ocf::heartbeat:azure-lb):      Started hn1-db-1
    vip_HN1_03 (ocf::heartbeat:IPaddr2):       Started hn1-db-1

Next steps