High availability of SAP HANA on Azure virtual machines

For on-premises development, you can use either HANA System Replication or use shared storage to establish high availability for SAP HANA. On Azure virtual machines (VMs), HANA System Replication on Azure is currently the only supported high availability function. SAP HANA Replication consists of one primary node and at least one secondary node. Changes to the data on the primary node are replicated to the secondary node synchronously or asynchronously.

This article describes how to deploy and configure the virtual machines, install the cluster framework, and install and configure SAP HANA System Replication. In the example configurations, installation commands, instance number 03, and HANA System ID HN1 are used.

Read the following SAP Notes and papers first:

  • SAP Note 1928533, which has:
    • The list of Azure VM sizes that are supported for the deployment of SAP software.
    • Important capacity information for Azure VM sizes.
    • The supported SAP software, and operating system (OS) and database combinations.
    • The required SAP kernel version for Windows and Linux on Microsoft Azure.
  • SAP Note 2015553 lists the prerequisites for SAP-supported SAP software deployments in Azure.
  • SAP Note 2205917 has recommended OS settings for SUSE Linux Enterprise Server for SAP Applications.
  • SAP Note 1944799 has SAP HANA Guidelines for SUSE Linux Enterprise Server for SAP Applications.
  • SAP Note 2178632 has detailed information about all of the monitoring metrics that are reported for SAP in Azure.
  • SAP Note 2191498 has the required SAP Host Agent version for Linux in Azure.
  • SAP Note 2243692 has information about SAP licensing on Linux in Azure.
  • SAP Note 1984787 has general information about SUSE Linux Enterprise Server 12.
  • SAP Note 1999351 has additional troubleshooting information for the Azure Enhanced Monitoring Extension for SAP.
  • SAP Community WIKI has all of the required SAP Notes for Linux.
  • Azure Virtual Machines planning and implementation for SAP on Linux guide.
  • Azure Virtual Machines deployment for SAP on Linux (this article).
  • Azure Virtual Machines DBMS deployment for SAP on Linux guide.
  • SAP HANA SR Performance Optimized Scenario guide. The guide contains all of the required information to set up SAP HANA System Replication for on-premises development. Use this guide as a baseline.

Overview

To achieve high availability, SAP HANA is installed on two virtual machines. The data is replicated by using HANA System Replication.

SAP HANA high availability overview

SAP HANA System Replication setup uses a dedicated virtual hostname and virtual IP addresses. On Azure, a load balancer is required to use a virtual IP address. The following list shows the configuration of the load balancer:

  • Front-end configuration: IP address 10.0.0.13 for hn1-db
  • Back-end configuration: Connected to primary network interfaces of all virtual machines that should be part of HANA System Replication
  • Probe Port: Port 62503
  • Load-balancing rules: 30313 TCP, 30315 TCP, 30317 TCP

Deploy for Linux

The resource agent for SAP HANA is included in SUSE Linux Enterprise Server for SAP Applications. The Azure Marketplace contains an image for SUSE Linux Enterprise Server for SAP Applications 12 that you can use to deploy new virtual machines.

Deploy with a template

You can use one of the quickstart templates that are on GitHub to deploy all the required resources. The template deploys the virtual machines, the load balancer, the availability set, and so on. To deploy the template, follow these steps:

  1. Open the database template or the converged template on the Azure portal. The database template creates the load-balancing rules for a database only. The converged template also creates the load-balancing rules for an ASCS/SCS and ERS (Linux only) instance. If you plan to install an SAP NetWeaver-based system and you want to install the ASCS/SCS instance on the same machines, use the converged template.

  2. Enter the following parameters:

    • Sap System ID: Enter the SAP system ID of the SAP system you want to install. The ID is used as a prefix for the resources that are deployed.
    • Stack Type: (This parameter is applicable only if you use the converged template.) Select the SAP NetWeaver stack type.
    • Os Type: Select one of the Linux distributions. For this example, select SLES 12.
    • Db Type: Select HANA.
    • Sap System Size: Enter the number of SAPS that the new system is going to provide. If you're not sure how many SAPS the system requires, ask your SAP Technology Partner or System Integrator.
    • System Availability: Select HA.
    • Admin Username and Admin Password: A new user is created that can be used to log on to the machine.
    • New Or Existing Subnet: Determines whether a new virtual network and subnet should be created or an existing subnet used. If you already have a virtual network that's connected to your on-premises network, select Existing.
    • Subnet ID: The ID of the subnet to which the virtual machines should be connected. To connect the virtual machine to your on-premises network, select the subnet of your VPN or Azure ExpressRoute virtual network. The ID usually looks like /subscriptions/<subscription ID>/resourceGroups/<resource group name>/providers/Microsoft.Network/virtualNetworks/<virtual network name>/subnets/<subnet name>.

Manual deployment

  1. Create a resource group.
  2. Create a virtual network.
  3. Create an availability set.
    • Set the max update domain.
  4. Create a load balancer (internal).
    • Select the virtual network created in step 2.
  5. Create virtual machine 1.
  6. Create virtual machine 2.
  7. Add data disks.
  8. Configure the load balancer. First, create a front-end IP pool:

    1. Open the load balancer, select frontend IP pool, and select Add.
    2. Enter the name of the new front-end IP pool (for example, hana-frontend).
    3. Set the Assignment to Static and enter the IP address (for example, 10.0.0.13).
    4. Select OK.
    5. After the new front-end IP pool is created, note the pool IP address.
  9. Next, create a back-end pool:

    1. Open the load balancer, select backend pools, and select Add.
    2. Enter the name of the new back-end pool (for example, hana-backend).
    3. Select Add a virtual machine.
    4. Select the availability set created in step 3.
    5. Select the virtual machines of the SAP HANA cluster.
    6. Select OK.
  10. Next, create a health probe:

    1. Open the load balancer, select health probes, and select Add.
    2. Enter the name of the new health probe (for example, hana-hp).
    3. Select TCP as the protocol and port 62503. Keep the Interval value set to 5, and the Unhealthy threshold value set to 2.
    4. Select OK.
  11. For SAP HANA 1.0, create the load-balancing rules:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30315).
    3. Select the front-end IP address, the back-end pool, and the health probe that you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30315.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for port 30317.
  12. For SAP HANA 2.0, create the load-balancing rules for the system database:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30313).
    3. Select the front-end IP address, the back-end pool, and the health probe that you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30313.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for port 30314.
  13. For SAP HANA 2.0, first create the load-balancing rules for the tenant database:

    1. Open the load balancer, select load balancing rules, and select Add.
    2. Enter the name of the new load balancer rule (for example, hana-lb-30340).
    3. Select the frontend IP address, backend pool, and health probe you created earlier (for example, hana-frontend).
    4. Keep the Protocol set to TCP, and enter port 30340.
    5. Increase the idle timeout to 30 minutes.
    6. Make sure to enable Floating IP.
    7. Select OK.
    8. Repeat these steps for ports 30341 and 30342.

For more information about the required ports for SAP HANA, read the chapter Connections to Tenant Databases in the SAP HANA Tenant Databases guide or SAP Note 2388694.

Create a Pacemaker cluster

Follow the steps in Setting up Pacemaker on SUSE Linux Enterprise Server in Azure to create a basic Pacemaker cluster for this HANA server. You can use the same Pacemaker cluster for SAP HANA and SAP NetWeaver (A)SCS.

Install SAP HANA

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [A] Set up the disk layout: Logical Volume Manager (LVM).

    We recommend that you use LVM for volumes that store data and log files. The following example assumes that the virtual machines have four data disks attached that are used to create two volumes.

    List all of the available disks:

    
    ls /dev/disk/azure/scsi1/lun*
    

    Example output:

    /dev/disk/azure/scsi1/lun0  /dev/disk/azure/scsi1/lun1  /dev/disk/azure/scsi1/lun2  /dev/disk/azure/scsi1/lun3
    

    Create physical volumes for all of the disks that you want to use:

    
    sudo pvcreate /dev/disk/azure/scsi1/lun0
    sudo pvcreate /dev/disk/azure/scsi1/lun1
    sudo pvcreate /dev/disk/azure/scsi1/lun2
    sudo pvcreate /dev/disk/azure/scsi1/lun3
    

    Create a volume group for the data files. Use one volume group for the log files and one for the shared directory of SAP HANA:

    
    sudo vgcreate vg_hana_data_HN1 /dev/disk/azure/scsi1/lun0 /dev/disk/azure/scsi1/lun1
    sudo vgcreate vg_hana_log_HN1 /dev/disk/azure/scsi1/lun2
    sudo vgcreate vg_hana_shared_HN1 /dev/disk/azure/scsi1/lun3
    

    Create the logical volumes. A linear volume is created when you use lvcreate without the -i switch. We suggest that you create a striped volume for better I/O performance, where the -i argument should be the number of the underlying physical volume. In this document, two physical volumes are used for the data volume, so the -i switch argument is set to 2. One physical volume is used for the log volume, so no -i switch is explicitly used. Use the -i switch and set it to the number of the underlying physical volume when you use more than one physical volume for each data, log, or shared volumes.

    
    sudo lvcreate -i 2 -l 100%FREE -n hana_data vg_hana_data_HN1
    sudo lvcreate -l 100%FREE -n hana_log vg_hana_log_HN1
    sudo lvcreate -l 100%FREE -n hana_shared vg_hana_shared_HN1
    sudo mkfs.xfs /dev/vg_hana_data_HN1/hana_data
    sudo mkfs.xfs /dev/vg_hana_log_HN1/hana_log
    sudo mkfs.xfs /dev/vg_hana_shared_HN1/hana_shared
    

    Create the mount directories and copy the UUID of all of the logical volumes:

    
    sudo mkdir -p /hana/data/HN1
    sudo mkdir -p /hana/log/HN1
    sudo mkdir -p /hana/shared/HN1
    # Write down the ID of /dev/vg_hana_data_HN1/hana_data, /dev/vg_hana_log_HN1/hana_log, and /dev/vg_hana_shared_HN1/hana_shared
    sudo blkid
    

    Create fstab entries for the three logical volumes:

    
    sudo vi /etc/fstab
    

    Insert the following line in the /etc/fstab file:

    
    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_data_HN1-hana_data> /hana/data/HN1 xfs  defaults,nofail  0  2
    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_log_HN1-hana_log> /hana/log/HN1 xfs  defaults,nofail  0  2
    /dev/disk/by-uuid/<UUID of /dev/mapper/vg_hana_shared_HN1-hana_shared> /hana/shared/HN1 xfs  defaults,nofail  0  2
    

    Mount the new volumes:

    
    sudo mount -a
    
  2. [A] Set up the disk layout: Plain Disks.

    For demo systems, you can place your HANA data and log files on one disk. Create a partition on /dev/disk/azure/scsi1/lun0 and format it with xfs:

    
    sudo sh -c 'echo -e "n\n\n\n\n\nw\n" | fdisk /dev/disk/azure/scsi1/lun0'
    sudo mkfs.xfs /dev/disk/azure/scsi1/lun0-part1
    
    # Write down the ID of /dev/disk/azure/scsi1/lun0-part1
    sudo /sbin/blkid
    sudo vi /etc/fstab
    

    Insert this line in the /etc/fstab file:

    
    /dev/disk/by-uuid/<UUID> /hana xfs  defaults,nofail  0  2
    

    Create the target directory and mount the disk:

    
    sudo mkdir /hana
    sudo mount -a
    
  3. [A] Set up host name resolution for all hosts.

    You can either use a DNS server or modify the /etc/hosts file on all nodes. This example shows you how to use the /etc/hosts file. Replace the IP address and the hostname in the following commands:

    sudo vi /etc/hosts
    

    Insert the following lines in the /etc/hosts file. Change the IP address and hostname to match your environment:

    
    10.0.0.5 hn1-db-0
    10.0.0.6 hn1-db-1
    
  4. [A] Install the SAP HANA high availability packages:

    sudo zypper install SAPHanaSR 
    

To install SAP HANA System Replication, follow chapter 4 of the SAP HANA SR Performance Optimized Scenario guide.

  1. [A] Run the hdblcm program from the HANA DVD. Enter the following values at the prompt:

    • Choose installation: Enter 1.
    • Select additional components for installation: Enter 1.
    • Enter Installation Path [/hana/shared]: Select Enter.
    • Enter Local Host Name [..]: Select Enter.
    • Do you want to add additional hosts to the system? (y/n) [n]: Select Enter.
    • Enter SAP HANA System ID: Enter the SID of HANA, for example: HN1.
    • Enter Instance Number [00]: Enter the HANA Instance number. Enter 03 if you used the Azure template or followed the manual deployment section of this article.
    • Select Database Mode / Enter Index [1]: Select Enter.
    • Select System Usage / Enter Index [4]: Select the system usage value.
    • Enter Location of Data Volumes [/hana/data/HN1]: Select Enter.
    • Enter Location of Log Volumes [/hana/log/HN1]: Select Enter.
    • Restrict maximum memory allocation? [n]: Select Enter.
    • Enter Certificate Host Name For Host '...' [...]: Select Enter.
    • Enter SAP Host Agent User (sapadm) Password: Enter the host agent user password.
    • Confirm SAP Host Agent User (sapadm) Password: Enter the host agent user password again to confirm.
    • Enter System Administrator (hdbadm) Password: Enter the system administrator password.
    • Confirm System Administrator (hdbadm) Password: Enter the system administrator password again to confirm.
    • Enter System Administrator Home Directory [/usr/sap/HN1/home]: Select Enter.
    • Enter System Administrator Login Shell [/bin/sh]: Select Enter.
    • Enter System Administrator User ID [1001]: Select Enter.
    • Enter ID of User Group (sapsys) [79]: Select Enter.
    • Enter Database User (SYSTEM) Password: Enter the database user password.
    • Confirm Database User (SYSTEM) Password: Enter the database user password again to confirm.
    • Restart system after machine reboot? [n]: Select Enter.
    • Do you want to continue? (y/n): Validate the summary. Enter y to continue.
  2. [A] Upgrade the SAP Host Agent.

    Download the latest SAP Host Agent archive from the SAP Software Center and run the following command to upgrade the agent. Replace the path to the archive to point to the file that you downloaded:

    sudo /usr/sap/hostctrl/exe/saphostexec -upgrade -archive <path to SAP Host Agent SAR>
    

Configure SAP HANA 2.0 System Replication

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [1] Create the tenant database.

    If you're using SAP HANA 2.0 or MDC, create a tenant database for your SAP NetWeaver system. Replace NW1 with the SID of your SAP system.

    Log in as <hanasid>adm and execute the following command:

    
    hdbsql -u SYSTEM -p "passwd" -i 03 -d SYSTEMDB 'CREATE DATABASE NW1 SYSTEM USER PASSWORD "passwd"'
    
  2. [1] Configure System Replication on the first node:

    Log in as <hanasid>adm and back up the databases:

    
    hdbsql -d SYSTEMDB -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupSYS')"
    hdbsql -d HN1 -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupHN1')"
    hdbsql -d NW1 -u SYSTEM -p "passwd" -i 03 "BACKUP DATA USING FILE ('initialbackupNW1')"
    

    Copy the system PKI files to the secondary site:

    
    scp /usr/sap/HN1/SYS/global/security/rsecssfs/data/SSFS_HN1.DAT   hn1-db-1:/usr/sap/HN1/SYS/global/security/rsecssfs/data/
    scp /usr/sap/HN1/SYS/global/security/rsecssfs/key/SSFS_HN1.KEY  hn1-db-1:/usr/sap/HN1/SYS/global/security/rsecssfs/key/
    

    Create the primary site:

    
    hdbnsutil -sr_enable –-name=SITE1
    
  3. [2] Configure System Replication on the second node:

    Register the second node to start the system replication. Log in as <hanasid>adm and run the following command:

    
    sapcontrol -nr 03 -function StopWait 600 10
    hdbnsutil -sr_register --remoteHost=hn1-db-0 --remoteInstance=03 --replicationMode=sync --name=SITE2 
    

Configure SAP HANA 1.0 System Replication

The steps in this section use the following prefixes:

  • [A]: The step applies to all nodes.
  • [1]: The step applies to node 1 only.
  • [2]: The step applies to node 2 of the Pacemaker cluster only.
  1. [1] Create the required users.

    Log in as root and run the following command. Make sure to replace bold strings (HANA System ID HN1 and instance number 03) with the values of your SAP HANA installation:

    
    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbsql -u system -i 03 'CREATE USER hdbhasync PASSWORD "passwd"' 
    hdbsql -u system -i 03 'GRANT DATA ADMIN TO hdbhasync' 
    hdbsql -u system -i 03 'ALTER USER hdbhasync DISABLE PASSWORD LIFETIME' 
    
  2. [A] Create the keystore entry.

    Log in as root and run the following command to create a new keystore entry:

    
    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbuserstore SET hdbhaloc localhost:30315 hdbhasync passwd
    
  3. [1] Back up the database.

    Log in as root and back up the databases:

    
    PATH="$PATH:/usr/sap/HN1/HDB03/exe"
    hdbsql -d SYSTEMDB -u system -i 03 "BACKUP DATA USING FILE ('initialbackup')"
    

    If you use a multi-tenant installation, also back up the tenant database:

       
    hdbsql -d HN1 -u system -i 03 "BACKUP DATA USING FILE ('initialbackup')"
    
  4. [1] Configure System Replication on the first node.

    Log in as <hanasid>adm and create the primary site:

    
    su - hdbadm
    hdbnsutil -sr_enable –-name=SITE1
    
  5. [2] Configure System Replication on the secondary node.

    Log in as <hanasid>adm and register the secondary site:

    
    sapcontrol -nr 03 -function StopWait 600 10
    hdbnsutil -sr_register --remoteHost=hn1-db-0 --remoteInstance=03 --replicationMode=sync --name=SITE2 
    

Create SAP HANA cluster resources

First, create the HANA topology. Run the following commands on one of the Pacemaker cluster nodes:


   sudo crm configure property maintenance-mode=true

   # Replace the bold string with your instance number and HANA system ID

   sudo crm configure primitive rsc_SAPHanaTopology_HN1_HDB03 ocf:suse:SAPHanaTopology \
     operations \$id="rsc_sap2_HN1_HDB03-operations" \
     op monitor interval="10" timeout="600" \
     op start interval="0" timeout="600" \
     op stop interval="0" timeout="300" \
     params SID="HN1" InstanceNumber="03"

   sudo crm configure clone cln_SAPHanaTopology_HN1_HDB03 rsc_SAPHanaTopology_HN1_HDB03 \
     meta is-managed="true" clone-node-max="1" target-role="Started" interleave="true"
   

Next, create the HANA resources:


   # Replace the bold string with your instance number, HANA system ID, and the front-end IP address of the Azure load balancer. 

   sudo crm configure primitive rsc_SAPHana_HN1_HDB03 ocf:suse:SAPHana \
     operations \$id="rsc_sap_HN1_HDB03-operations" \
     op start interval="0" timeout="3600" \
     op stop interval="0" timeout="3600" \
     op promote interval="0" timeout="3600" \
     op monitor interval="60" role="Master" timeout="700" \
     op monitor interval="61" role="Slave" timeout="700" \
     params SID="HN1" InstanceNumber="03" PREFER_SITE_TAKEOVER="true" \
     DUPLICATE_PRIMARY_TIMEOUT="7200" AUTOMATED_REGISTER="false"

   sudo crm configure ms msl_SAPHana_HN1_HDB03 rsc_SAPHana_HN1_HDB03 \
     meta is-managed="true" notify="true" clone-max="2" clone-node-max="1" \
     target-role="Started" interleave="true"

   sudo crm configure primitive rsc_ip_HN1_HDB03 ocf:heartbeat:IPaddr2 \
     meta target-role="Started" is-managed="true" \
     operations \$id="rsc_ip_HN1_HDB03-operations" \
     op monitor interval="10s" timeout="20s" \
     params ip="10.0.0.13"

   sudo crm configure primitive rsc_nc_HN1_HDB03 anything \
     params binfile="/usr/bin/nc" cmdline_options="-l -k 62503" \
     op monitor timeout=20s interval=10 depth=0

   sudo crm configure group g_ip_HN1_HDB03 rsc_ip_HN1_HDB03 rsc_nc_HN1_HDB03

   sudo crm configure colocation col_saphana_ip_HN1_HDB03 4000: g_ip_HN1_HDB03:Started \
     msl_SAPHana_HN1_HDB03:Master  

   sudo crm configure order ord_SAPHana_HN1_HDB03 2000: cln_SAPHanaTopology_HN1_HDB03 \
     msl_SAPHana_HN1_HDB03

   # Clean up the HANA resources. The HANA resources might have failed because of a known issue.
   sudo crm resource cleanup rsc_SAPHana_HN1_HDB03

   sudo crm configure property maintenance-mode=false
   

Make sure that the cluster status is ok and that all of the resources are started. It's not important on which node the resources are running.


   sudo crm_mon -r

   # Online: [ hn1-db-0 hn1-db-1 ]
   #
   # Full list of resources:
   #
   # stonith-sbd     (stonith:external/sbd): Started hn1-db-0
   # rsc_st_azure    (stonith:fence_azure_arm):      Started hn1-db-1
   # Clone Set: cln_SAPHanaTopology_HN1_HDB03 [rsc_SAPHanaTopology_HN1_HDB03]
   #     Started: [ hn1-db-0 hn1-db-1 ]
   # Master/Slave Set: msl_SAPHana_HN1_HDB03 [rsc_SAPHana_HN1_HDB03]
   #     Masters: [ hn1-db-0 ]
   #     Slaves: [ hn1-db-1 ]
   # Resource Group: g_ip_HN1_HDB03
   #     rsc_ip_HN1_HDB03   (ocf::heartbeat:IPaddr2):       Started hn1-db-0
   #     rsc_nc_HN1_HDB03   (ocf::heartbeat:anything):      Started hn1-db-0
   

Test the cluster setup

This section describes how you can test your setup. Every test assumes that you are root and the SAP HANA master is running on the hn1-db-0 virtual machine.

Test the fencing agent

You can test the setup of the fencing agent by disabling the network interface on the hn1-db-0 node:


sudo ifdown eth0

The virtual machine should now restart or stop depending on your cluster configuration. If you set the stonith-action setting to off, the virtual machine is stopped and the resources are migrated to the running virtual machine.

After you start the virtual machine again, the SAP HANA resource fails to start as secondary if you set AUTOMATED_REGISTER="false". In this case, configure the HANA instance as secondary by executing this command:


su - hn1adm

# Stop the HANA instance just in case it is running
sapcontrol -nr 03 -function StopWait 600 10
hdbnsutil -sr_register --remoteHost=hn1-db-1 --remoteInstance=03 --replicationMode=sync --name=SITE1

# Switch back to root and clean up the failed state
exit
crm resource cleanup msl_SAPHana_HN1_HDB03 hn1-db-0

Test a manual failover

You can test a manual failover by stopping the pacemaker service on the hn1-db-0 node:


service pacemaker stop

After the failover, you can start the service again. If you set AUTOMATED_REGISTER="false", the SAP HANA resource on the hn1-db-0 node fails to start as secondary. In this case, configure the HANA instance as secondary by executing this command:


service pacemaker start
su - hn1adm

# Stop the HANA instance just in case it is running
sapcontrol -nr 03 -function StopWait 600 10
hdbnsutil -sr_register --remoteHost=hn1-db-1 --remoteInstance=03 --replicationMode=sync --name=SITE1 

# Switch back to root and clean up the failed state
exit
crm resource cleanup msl_SAPHana_HN1_HDB03 hn1-db-0

Test the migration

You can migrate the SAP HANA master node by executing the following command:


crm resource migrate msl_SAPHana_HN1_HDB03 hn1-db-1
crm resource migrate g_ip_HN1_HDB03 hn1-db-1

If you set AUTOMATED_REGISTER="false", this sequence of commands should migrate the SAP HANA master node and the group that contains the virtual IP address to hn1-db-1. The SAP HANA resource on hn1-db-0 fails to start as secondary. In this case, configure the HANA instance as secondary by executing this command:


su - hn1adm

# Stop the HANA instance just in case it is running
sapcontrol -nr 03 -function StopWait 600 10
hdbnsutil -sr_register --remoteHost=hn1-db-1 --remoteInstance=03 --replicationMode=sync --name=SITE1 

The migration creates location constraints that need to be deleted again:


crm configure edited

# Delete location constraints that are named like the following contraint. You should have two constraints: one for the SAP HANA resource, and one for the IP address group.
location cli-prefer-g_ip_HN1_HDB03 g_ip_HN1_HDB03 role=Started inf: hn1-db-1

You also need to clean up the state of the secondary node resource:


# Switch back to root and clean up the failed state
exit
crm resource cleanup msl_SAPHana_HN1_HDB03 hn1-db-0

Next steps