Configure Always On availability group for SQL Server on Linux

This article describes how to create a SQL Server Always on availability group with three replicas for high availability on Linux. You can configure an availability group for SQL Server on Linux. There are two architectures for availability groups. A high availability architecture has three servers and uses a cluster manager to provide business continuity. This architecture can also include read-scale replicas. This document explains how to create the availability group high availability architecture.

You can also create a read-scale availability group without a cluster manager. This architecture only provides read-only scalability. It does not provide HA. To create a read-scale availability group, see Configure read-scale availability group for SQL Server on Linux.

For a configuration that will guarantee high availability and data protection, the availability group must include at least three synchronous commit replicas. For details see Balancing high availability and data protection.

All servers must be either physical or virtual, and virtual servers must be on the same virtualization platform. This is because the fencing agents are platform specific. See Policies for Guest Clusters.

Roadmap

The steps to create an availability group on Linux servers for high availability are different from the steps on a Windows Server failover cluster. The following list describes the high level steps:

  1. Configure SQL Server on three cluster servers.

    Important

    All three servers in the availability group need to be on the same platform - i.e. physical or virtual. This is because the Linux high availability uses fencing agents to isolate resources on servers. The fencing agents are specific for each platform.

  2. Create the availability group. This step is covered in this current article.

  3. Configure a cluster resource manager, like Pacemaker.

    The way to configure a cluster resource manager depends on the specific Linux distribution. Follow the distribution specific links below:

    Important

    Production environments require a fencing agent, like STONITH for high availability. The demonstrations in this documentation do not use fencing agents. The demonstrations are for testing and validation only.

    A Linux cluster uses fencing to return the cluster to a known state. The way to configure fencing depends on the distribution and the environment. At this time, fencing is not available in some cloud environments. See Support Policies for RHEL High Availability Clusters - Virtualization Platforms for more information.

    For SLES, see SUSE Linux Enterprise High Availability Extension.

  4. Add the availability group as a resource in the cluster.

    The way to add the availability group as a resource in the cluster depends on the Linux distribution.

    Follow the distribution specific links below:

Prerequisites

Before you create the availability group, you need to:

  • Set your environment so all servers that will host availability replicas can communicate
  • Install SQL Server
Note

On Linux, you must create an availability group before adding it as a cluster resource to be managed by the cluster. This document provides an example that creates the availability group. For distribution specific instructions to create the cluster and add the availability group as a cluster resource, see the links under Next steps.

  1. Update the computer name for each host

    Each SQL Server name must be:

    • 15 characters or less
    • Unique within the network

    To set the computer name, edit /etc/hostname. The following script lets you edit /etc/hostname with vi.

    sudo vi /etc/hostname
    
  2. Configure the hosts file

Note

If hostnames are registered with their IP in the DNS server, there is no need to do the steps below. Validate that all nodes that are going to be part of the availability group configuration can communicate with each other (pinging the hostname should reply with the corresponding IP address). Also, make sure that /etc/hosts file does not contain a record that maps localhost IP address 127.0.0.1 with the hostname of the node.

The hosts file on every server contains the IP addresses and names of all servers that will participate in the availability group.

The following command returns the IP address of the current server:

sudo ip addr show

Update /etc/hosts. The following script lets you edit /etc/hosts with vi.

sudo vi /etc/hosts

The following example shows /etc/hosts on node1 with additions for node1 and node2. In this document node1 refers to the primary SQL Server replica. node2 refers to the secondary SQL Server.;

127.0.0.1   localhost localhost4 localhost4.localdomain4
::1       localhost localhost6 localhost6.localdomain6
10.128.18.128 node1
10.128.16.77 node2

Install SQL Server

Install SQL Server. The following links point to SQL Server installation instructions for various distributions.

Enable Always On availability groups and restart sqlserver

Enable Always On availability groups on each node hosting SQL Server service, then restart mssql-server. Run the following script:

sudo /opt/mssql/bin/mssql-conf set hadr.hadrenabled  1
sudo systemctl restart mssql-server

Enable AlwaysOn_health event session

You can optionaly enable Always On Availability Groups specific extended events to help with root-cause diagnosis when you troubleshoot an availability group.

ALTER EVENT SESSION  AlwaysOn_health ON SERVER WITH (STARTUP_STATE=ON);
GO

For more information about this XE session, see Always On Extended Events.

Create db mirroring endpoint user

The following Transact-SQL script creates a login named dbm_login, and a user named dbm_user. Update the script with a strong password. Run the following command on all SQL Servers to create the database mirroring endpoint user.

CREATE LOGIN dbm_login WITH PASSWORD = '**<1Sample_Strong_Password!@#>**';
CREATE USER dbm_user FOR LOGIN dbm_login;

Create a certificate

SQL Server service on Linux uses certificates to authenticate communication between the mirroring endpoints.

The following Transact-SQL script creates a master key and certificate. It then backs the certificate up and secures the file with a private key. Update the script with strong passwords. Connect to the primary SQL Server and run the following Transact-SQL to create the certificate:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = '**<Master_Key_Password>**';
CREATE CERTIFICATE dbm_certificate WITH SUBJECT = 'dbm';
BACKUP CERTIFICATE dbm_certificate
   TO FILE = '/var/opt/mssql/data/dbm_certificate.cer'
   WITH PRIVATE KEY (
           FILE = '/var/opt/mssql/data/dbm_certificate.pvk',
           ENCRYPTION BY PASSWORD = '**<Private_Key_Password>**'
       );

At this point your primary SQL Server replica has a certificate at /var/opt/mssql/data/dbm_certificate.cer and a private key at var/opt/mssql/data/dbm_certificate.pvk. Copy these two files to the same location on all servers that will host availability replicas. Use the mssql user or give permission to mssql user to access these files.

For example on the source server, the following command copies the files to the target machine. Replace the values with the names of the SQL Server instances that will host the replicas.

cd /var/opt/mssql/data
scp dbm_certificate.* root@**<node2>**:/var/opt/mssql/data/

On the target server, give permission to mssql user to access the certificate.

cd /var/opt/mssql/data
chown mssql:mssql dbm_certificate.*

Create the certificate on secondary servers

The following Transact-SQL script creates a master key and certificate from the backup that you created on the primary SQL Server replica. The command also authorizes the user to access the certificate. Update the script with strong passwords. The decryption password is the same password that you used to create the .pvk file in a previous step. Run the following script on all secondary servers to create the certificate.

CREATE MASTER KEY ENCRYPTION BY PASSWORD = '**<Master_Key_Password>**';
CREATE CERTIFICATE dbm_certificate   
    AUTHORIZATION dbm_user
    FROM FILE = '/var/opt/mssql/data/dbm_certificate.cer'
    WITH PRIVATE KEY (
    FILE = '/var/opt/mssql/data/dbm_certificate.pvk',
    DECRYPTION BY PASSWORD = '**<Private_Key_Password>**'
            );

Create the database mirroring endpoints on all replicas

Database mirroring endpoints use Transmission Control Protocol (TCP) to send and receive messages between the server instances participating database mirroring sessions or hosting availability replicas. The database mirroring endpoint listens on a unique TCP port number.

The following Transact-SQL creates a listening endpoint named Hadr_endpoint for the availability group. It starts the endpoint, and gives connect permission to the user that you created. Before you run the script, replace the values between **< ... >**.

Note

For this release, do not use a different IP address for the listener IP. We are working on a fix for this issue, but the only acceptable value for now is '0.0.0.0'.

Update the following Transact-SQL for your environment on all SQL Server instances:

CREATE ENDPOINT [Hadr_endpoint]
    AS TCP (LISTENER_IP = (0.0.0.0), LISTENER_PORT = **<5022>**)
    FOR DATA_MIRRORING (
        ROLE = ALL,
        AUTHENTICATION = CERTIFICATE dbm_certificate,
        ENCRYPTION = REQUIRED ALGORITHM AES
        );
ALTER ENDPOINT [Hadr_endpoint] STATE = STARTED;
GRANT CONNECT ON ENDPOINT::[Hadr_endpoint] TO [dbm_login];
Important

The TCP port on the firewall needs to be open for the listener port.

For complete information, see The Database Mirroring Endpoint (SQL Server).

Create the availability group

Create the availability group. In order to create the availability group for HA on Linux, use the CREATE AVAILABILITY GROUP Transact-SQL DDL with CLUSTER_TYPE = EXTERNAL.

The EXTERNAL value for CLUSTER_TYPE option specifies that the an external cluster entity manages the availability group. Pacemaker is an example of an external cluster entity. When the availability group CLUSTER_TYPE = EXTERNAL, set each replica FAILOVER_MODE = EXTERNAL. After you create the availability group, configure the cluster resource for the availability group using the cluster management tools - for example with Pacemaker use pcs (on RHEL, Ubuntu) or crm (on SLES). See the Linux distribution specific cluster configuration section for an end-to-end example.

The following Transact-SQL script creates an availability group for HA named ag1. The script configures the availability group replicas with SEEDING_MODE = AUTOMATIC. This setting causes SQL Server to automatically create the database on each secondary server. Update the following script for your environment. Replace the **<node1>**, **<node2>**, and **<node3>** values with the names of the SQL Server instances that will host the replicas. Replace the **<5022>** with the port you set for the data mirroring endpoint. Run the following Transact-SQL on the SQL Server instance that will host the primary replica to create the availability group.

CREATE AVAILABILITY GROUP [ag1]
    WITH (DB_FAILOVER = ON, CLUSTER_TYPE = EXTERNAL)
    FOR REPLICA ON
        N'**<node1>**' 
        WITH (
            ENDPOINT_URL = N'tcp://**<node1>:**<5022>**',
            AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,
            FAILOVER_MODE = EXTERNAL,
            SEEDING_MODE = AUTOMATIC
            ),
        N'**<node2>**' 
        WITH ( 
            ENDPOINT_URL = N'tcp://**<node2>**:**<5022>**', 
            AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,
            FAILOVER_MODE = EXTERNAL,
            SEEDING_MODE = AUTOMATIC
            ),
        N'**<node3>**'
        WITH( 
            ENDPOINT_URL = N'tcp://**<node3>**:**<5022>**', 
            AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,
            FAILOVER_MODE = EXTERNAL,
            SEEDING_MODE = AUTOMATIC
            );

ALTER AVAILABILITY GROUP [ag1] GRANT CREATE ANY DATABASE;
Note

Running the CREATE AVAILABILITY GROUP command will complete with a warning: "Attempt to access non-existent or uninitialized availability group with ID . This is usually an internal condition, such as the availability group is being dropped or the local WSFC node has lost quorum. In such cases, and no user action is required.". This is a known issue and product team is working on a fix. Meanwhile, users should assume command completed successfully.

You can also configure an availability group with CLUSTER_TYPE=EXTERNAL using SQL Server Management Studio or PowerShell.

Join secondary replicas to the availability group

The following Transact-SQL script joins a SQL Server instance to an availability group named ag1. Update the script for your environment. On each SQL Server instance that will host a secondary replica, run the following Transact-SQL to join the availability group.

ALTER AVAILABILITY GROUP [ag1] JOIN WITH (CLUSTER_TYPE = EXTERNAL);

ALTER AVAILABILITY GROUP [ag1] GRANT CREATE ANY DATABASE;

Add a database to the availability group

Ensure the database you are adding to the Availability group is in full recovery mode and has a valid log backup. If this is a test database or a new database created, take a database backup. On the primary SQL Server, run the following Transact-SQL to create and back up a database called db1.

CREATE DATABASE [db1];
ALTER DATABASE [db1] SET RECOVERY FULL;
BACKUP DATABASE [db1] 
   TO DISK = N'var/opt/mssql/data/db1.bak';

On the primary SQL Server replica, run the following Transact-SQL to add a database called db1 to an availability group called ag1.

ALTER AVAILABILITY GROUP [ag1] ADD DATABASE [db1];

Verify that the database is created on the secondary servers

On each secondary SQL Server replica, run the following query to see if the db1 database has been created and is synchronized.

SELECT * FROM sys.databases WHERE name = 'db1';
GO
SELECT DB_NAME(database_id) AS 'database', synchronization_state_desc FROM sys.dm_hadr_database_replica_states;
Important

After you create the availability group, you must configure integration with a cluster technology like Pacemaker for HA. In the case of a read-only scale-out architecture using availability groups, starting with SQL Server 2017, setting up a cluster is not required. If you followed the steps in this document, you have an availability group that is not yet clustered. The next step is to add the cluster. While this is a valid configuration in read-scale/load balancing scenarios, it is not complete for high availability. To achieve high availability, you need to add the availability group as a cluster resource. See Next steps for instructions.

Notes

Important

After you configure the cluster and add the availability group as a cluster resource, you cannot use Transact-SQL to fail over the availability group resources. SQL Server cluster resources on Linux are not coupled as tightly with the operating system as they are on a Windows Server Failover Cluster (WSFC). SQL Server service is not aware of the presence of the cluster. All orchestration is done through the cluster management tools. In RHEL or Ubuntu use pcs. In SLES use crm.

Important

If the availability group is a cluster resource, there is a known issue in current release where forced failover with data loss to an asynchronous replica does not work. This will be fixed in the upcoming release. Manual or automatic failover to a synchronous replica will succeed.

Next steps

Configure Red Hat Enterprise Linux Cluster for SQL Server Availability Group Cluster Resources

Configure SUSE Linux Enterprise Server Cluster for SQL Server Availability Group Cluster Resources

Configure Ubuntu Cluster for SQL Server Availability Group Cluster Resources