Tutorial: Migrate from Azure Database for PostgreSQL - Single Server to Azure Database for PostgreSQL - Flexible Server using the migration service

APPLIES TO: Azure Database for PostgreSQL - Flexible Server

Using the Azure portal, you can migrate an instance of Azure Database for PostgreSQL – Single Server to Azure Database for PostgreSQL – Flexible Server. In this tutorial, we perform migration of a sample database from an Azure Database for PostgreSQL single server to a PostgreSQL flexible server using the Azure portal.

  • Configure your Azure Database for PostgreSQL Flexible Server
  • Configure the migration task
  • Monitor the migration
  • Cancel the migration
  • Post migration

You can migrate using the Azure portal.

Prerequisites (offline)

Before you start your migration with migration service in Azure Database for PostgreSQL, fulfilling the following prerequisites, which apply to offline migration scenarios is essential.

Verify the source version

Source PostgreSQL version should be >= 9.5. If the source PostgreSQL version is less than 9.5, upgrade the source PostgreSQL version to 9.5 or higher before migration.

Target setup

  • Azure Database for PostgreSQL must be set up in Azure before migration.

  • The SKU chosen for the Azure Database for PostgreSQL should correspond with the specifications of the source database to ensure compatibility and adequate performance.

  • For detailed instructions on creating a new Azure Database for PostgreSQL, refer to the following link: Quickstart: Create server.

Network setup

Proper networking setup is essential to ensure successful connectivity between the source and target during migration. Here's a guide to help you establish the network connection for different scenarios:

Networking requirements for migration:

  • ExpressRoute/IPsec VPN/VPN tunneling: When connecting your on-premises/AWS source to Azure, you might need to set up an ExpressRoute, IPsec VPN, or VPN tunneling to facilitate secure data transfer.

  • VNET peering: Establish virtual network peering between the two distinct VNets to enable direct network connectivity, a prerequisite for migration between the Azure VM and the Azure Database for PostgreSQL.

Connectivity Scenarios:

The following table can help set up the network between the source and target.

Source Target Connectivity Tips
Public Public No other action is required if the source is whitelisted in the target's firewall rules.
Private Public This configuration isn't supported; use pg_dump/pg_restore for data transfer.
Public Private No other action is required if the source is whitelisted in the target's firewall rules.
Private Private Establish an ExpressRoute, IPsec VPN, VPN Tunneling, or virtual network Peering between the source and target.
Private Private Endpoint This configuration isn't supported; contact Microsoft support.

Additional Networking Considerations:

  • pg_hba.conf Configuration: To facilitate connectivity between the source and target PostgreSQL instances, it is essential to verify and potentially modify the pg_hba.conf file. This file includes client authentication and must be configured to allow the target PostgreSQL to connect to the source. Changes to the pg_hba.conf file typically require a restart of the source PostgreSQL instance to take effect.

Note

The pg_hba.conf file is located in the data directory of the PostgreSQL installation. This file should be checked and configured if the source database is an on-premises PostgreSQL server or a PostgreSQL server hosted on an Azure VM. For PostgreSQL instances on AWS RDS or similar managed services, the pg_hba.conf file is not directly accessible or applicable. Instead, access is controlled through the service's provided security and network access configurations.

For more information about network setup, visit Network guide for migration service in Azure Database for PostgreSQL - Flexible Server.

Extensions

Extensions are extra features that can be added to PostgreSQL to enhance its functionality. Extensions are supported in Azure Database for PostgreSQL but must be enabled manually. To enable extensions, follow these steps:

  • Use the select command in the source to list all the extensions that are being used - select extname,extversion from pg_extension;

  • Search for azure.extensions server parameter on the Server parameter page on your Azure Database for PostgreSQL. Enable the extensions found in the source within PostgreSQL.

  • Save the parameter changes and restart the Azure Database for PostgreSQL to apply the new configuration if necessary.

    Screenshot of extensions.

  • Check if the list contains any of the following extensions:

    • PG_CRON
    • PG_HINT_PLAN
    • PG_PARTMAN_BGW
    • PG_PREWARM
    • PG_STAT_STATEMENTS
    • PG_AUDIT
    • PGLOGICAL
    • WAL2JSON

If yes, search the server parameters page for the shared_preload_libraries parameter. This parameter indicates the set of extension libraries that are preloaded at the server restart.

Server parameters

These parameters aren't automatically migrated to the target environment and must be manually configured.

  • Match server parameter values from the source PostgreSQL database to the Azure Database for PostgreSQL by accessing the "Server parameters" section in the Azure portal and manually updating the values accordingly.

  • Save the parameter changes and restart the Azure Database for PostgreSQL to apply the new configuration if necessary.

Disable high availability (reliability) and read replicas in the target

  • Disabling high availability (reliability) and reading replicas in the target environment is essential. These features should be enabled only after the migration has been completed.

  • By following these guidelines, you can help ensure a smooth migration process without the added variables introduced by HA and Read Replicas. Once the migration is complete and the database is stable, you can proceed to enable these features to enhance the availability and scalability of your database environment in Azure.

Configure your Azure Database for PostgreSQL Flexible Server

  • Create the target flexible server. For guided steps, refer to the quickstart Create an Azure Database for PostgreSQL flexible server using the portal.

  • Allowlist extensions whose libraries must be loaded at server start. It's essential that the extension is on the allowlist before you initiate a migration.

  • Check if the data distribution among a database's tables is skewed, with most of the data present in a single (or few) tables. If it's skewed, the migration speed could be slower than expected. In this case, the migration speed can be increased by migrating the large table in parallel.

Configure the migration task

The migration service comes with a simple, wizard-based experience on the Azure portal. Here's how to start:

  1. Open your web browser and go to the portal. To sign in, enter your credentials. The default view is your service dashboard.

  2. Go to your Azure Database for PostgreSQL Flexible Server target.

  3. In the Overview tab of the Flexible Server, on the left menu, scroll down to Migration and select it.

    Screenshot of the flexible Overview page.

  4. Select the Create button to start a migration from a single server to a flexible server. If this is your first time using the migration service, an empty grid appears with a prompt to begin your first migration.

    Screenshot of the migration tab in flexible server.

    If you've already created migrations to your Flexible Server target, the grid contains information about migrations that were attempted to this target from the Single Server.

  5. Select the Migrate from Single Server button. You go through a wizard-based series of tabs to create a migration into this Flexible Server target from any source Single Server.

Alternatively, you can initiate the migration process from the Azure Database for PostgreSQL Single Server.

  1. Open your web browser and go to the portal. To sign in, you must enter your credentials. The default view is your service dashboard.

  2. Upon selecting the Single Server, you can observe a migration-related banner in the Overview tab. Select Migrate now to get started.

    Screenshot to initiate migration from Single Server tab.

  3. You're taken to a page with two options. If you've already created a Flexible Server and want to use that as the target, choose Select existing, and select the corresponding Subscription, Resource group, and Server name details. Once the selections are made, select Go to Migration wizard and skip to the instructions under this page's Setup tab section.

    Screenshot to choose existing flexible server option.

  4. Should you choose to Create a new Flexible Server, select Create new and select Go to Create Wizard. This action takes you through the Flexible Server creation process and deploys the Flexible Server.

    Screenshot to choose new flexible server option.

After deploying the Flexible Server, follow the steps 3 to 5 under Configure the migration task.

Setup tab

The first tab is Setup. In case you missed it, allowlist necessary extensions as shown in It's essential to allowlist these extensions before you initiate a migration.

Screenshot of the details belonging to the set up tab for offline.

Migration name is the unique identifier for each migration to this Flexible Server target. This field accepts only alphanumeric characters and doesn't accept any special characters except a hyphen (-). The name can't start with a hyphen and should be unique for a target server. No two migrations to the same Flexible Server target can have the same name.

Source server type indicates the source. In this case, it's Azure Database for PostgreSQL Single server

Migration Option allows you to perform validations before triggering a migration. You can pick any of the following options.

  • Validate - Checks your server and database readiness for migration to the target.
  • Migrate - Skips validations and starts migrations.
  • Validate and Migrate - Performs validation before triggering a migration. Migration gets triggered only if there are no validation failures.

It's always a good practice to choose Validate or Validate and Migrate option to perform premigration validations before running the migration.

If the Online migration preview is selected, Logical replication must be turned on in the source Single server. If it's not turned on, the migration service automatically turns on logical replication at the source Single server. Replication can also be set up manually under the Replication tab in the Single server-side pane by setting the Azure replication support level to Logical. Either approach restarts the source single server.

Select the Next : Connect to Source button.

Source tab

The Source tab prompts you to give details related to the Single Server, which is the source of the databases.

After you make the Subscription and Resource Group selections, the dropdown list for server names shows Single Servers under that resource group across regions. Select the source that you want to migrate databases from. You can migrate databases from a Single Server to a target Flexible Server in the same region. Cross-region migrations are enabled only for India, China, and UAE servers.

After you choose the Single Server source, the Location, PostgreSQL version, and Server admin login name boxes are populated automatically. The server admin sign-in name is the admin username used to create the Single Server. In the Password box, enter the password for that admin user. The migration service migrates single server databases as the admin user.

After filling out all the fields, select the Connect to source link. This validates that the source server details entered are correct and that the source server is reachable.

Screenshot of source database server details.

Select the Next : Select migration target button to continue.

Target tab

The Target tab displays metadata for the Flexible Server target, such as subscription name, resource group, server name, location, and PostgreSQL version.

Screenshot of target database server details.

For Server admin login name, the tab displays the admin username used during creating the Flexible Server target. Enter the corresponding password for the admin user. After filling out the password, select the Connect to target link. This validates that the target server details entered are correct and target server is reachable.

Select the Next button to select the databases to migrate.

Select Databases for the migration tab

Under this tab, there's a list of user databases inside the Single Server. You can select and migrate up to eight databases in a single migration attempt. If there are more than eight user databases, the migration process is repeated between the source and target servers for the next set of databases. By default, selected databases with the same name on the target are overwritten.

Screenshot of Databases to migrate.

Select the Next button to review the details.

Summary

The Summary tab summarizes all the details for creating the validation or migration. Review the details and select on the start button.

Screenshot of details to review for the migration.

Monitor the migration portal

After you select the start button, a notification appears in a few seconds to say that the validation or migration creation is successful. You're redirected automatically to the Migration page of Flexible Server. This has a new entry for the recently created validation or migration.

Screenshot of recently created migration details.

The grid that displays the migrations has these columns: Name, Status, Migration type, Migration mode, Source server, Source server type, Databases, Start time and Duration. The entries are displayed in the descending order of the start time with the most recent entry on the top.

You can use the refresh button to refresh the status of the validation or migration. You can also select the migration name in the grid to see the associated details.

When the validation or migration is created, it moves to the InProgress state and PerformingPreRequisiteSteps substate. The workflow takes 2-3 minutes to set up the migration infrastructure and network connections.

Let us look at how to monitor migrations for each Migration Option.

Validate

After the PerformingPreRequisiteSteps substate is completed, the validation moves to the substate of Validation in Progress where checks are done on the source and target server to assess the readiness for migration.

The validation moves to the Succeeded state if all validations are either in Succeeded or Warning state.

Screenshot of the validation grid.

The validation grid has the

  • Validation details for instance and Validation details for databases sections, representing the validation rules used to check migration readiness.
  • Validation Status - Represents the result for each rule and can have any of the three values
    • Succeeded - If no errors were found.
    • Failed - If there are validation errors.
    • Warning - If there are validation warnings.
  • Duration - Time taken for the Validation operation.
  • Start and End time - Start and end time of the validation operation in UTC.

The Validation status moves to Failed state if there are any errors in the validation. Select the Validation name or Database name validation that has failed, and a fan-out pane gives the details and the corrective action you should take to avoid this error.

Screenshot of the validation grid with failed status.

Migrate

After the PerformingPreRequisiteSteps substrate is completed, the migration moves to the substrate of Migrating Data when the Cloning/Copying of the databases takes place. The time for migration to complete depends on the size and shape of the databases you're migrating. The migration is quick if the data is mostly evenly distributed across all the tables. Skewed table sizes take a relatively longer time.

When you select any of the databases in migration, a fan-out pane appears. It has all the table count - copied, queued, copying, and errors apart from the database migration status.

Screenshot of the migration grid containing all DB details.

The migration moves to the Succeeded state when the Migrating Data state finishes successfully. If there's an issue at the Migrating Data state, the migration moves into a Failed state.

Screenshot of the migration result.

Once the migration moves to the Succeeded state, schema and data migration from your Single Server to your Flexible Server target is complete. You can use the refresh button on the page to confirm the same.

Screenshot of the completed migrations.

Validate and Migrate

In this option, validations are performed first before migration starts. After the PerformingPreRequisiteSteps substate is completed, the workflow moves into the substate of Validation in Progress.

  • If validation has errors, the migration moves into a Failed state.
  • If validation is complete without any error, the migration starts, and the workflow moves into the substate of Migrating Data.

You can see the results of Validate and Migrate once the operation is complete.

Screenshot showing validations tab in details page.

Cancel the migration using the portal

You can cancel any ongoing validations or migrations. The workflow must be in the InProgress state to be canceled. You can't cancel a validation or migration that's in the Succeeded or Failed state.

Canceling a validation stops any further validation activity and the validation moves to a Canceled state. Canceling a migration stops further migration activity on your target server and moves to a Canceled state. It doesn't drop or roll back any changes on your target server. Be sure to drop the databases on your target server involved in a canceled migration.

Post migration

After completing the databases, you need to manually validate the data between source and target and verify that all the objects in the target database are successfully created.

After migration, you can perform the following tasks:

  • Verify the data on your flexible server and ensure it's an exact copy of the source instance.

  • Post verification, enable the high availability option on your flexible server as needed.

  • Change the SKU of the flexible server to match the application needs. This change needs a database server restart.

  • If you change any server parameters from their default values in the source instance, copy those server parameter values in the flexible server.

  • Copy other server settings like tags, alerts, and firewall rules (if applicable) from the source instance to the flexible server.

  • Make changes to your application to point the connection strings to a flexible server.

  • Monitor the database performance closely to see if it requires performance tuning.