Azure Red Hat OpenShift FAQ

This article answers frequently asked questions (FAQs) about Microsoft Azure Red Hat OpenShift.

Installation and upgrade

Where can I find information about pricing and service level agreements?

For pricing information, see Azure Red Hat OpenShift pricing.

For Service Level Agreement (SLA) information, see Service Level Agreements for online services.

Which Azure regions are supported?

For a list of supported regions for Azure Red Hat OpenShift 4.x, see Available regions.

What virtual machine sizes can I use?

For a list of supported virtual machine sizes for Azure Red Hat OpenShift 4, see Supported resources for Azure Red Hat OpenShift 4.

What is the maximum number of pods in an Azure Red Hat OpenShift cluster? What is the maximum number of pods per node in Azure Red Hat OpenShift?

The actual number of supported pods depends on an application’s memory, CPU, and storage requirements.

Azure Red Hat OpenShift 4.x has a 250 pod-per-node limit and a 60 compute node limit. These limits cap the maximum number of pods supported in a cluster to 250×60 = 15,000. For a private cluster without any public IPs (for example, created by using User Defined Routing (UDR) and running version 4.11 or higher), the limits are 120 compute nodes and 30,000 pods.

Can a cluster have compute nodes across multiple Azure regions?

No. All nodes in an Azure Red Hat OpenShift cluster must originate in the same Azure region.

Can a cluster be deployed across multiple availability zones?

Yes. A cluster can be deployed across multiple availability zones automatically if your cluster is deployed to an Azure region that supports availability zones. For more information, see Availability zones.

Are control plane nodes abstracted away as they are with Azure Kubernetes Service (AKS)?

No. All resources, including the cluster control plane nodes, run in your customer subscription. These types of resources are put in a read-only resource group.

Does the cluster reside in a customer subscription?

The Azure Managed Application lives in a locked Resource Group with the customer subscription. Customers can view objects in that resource group but not modify them.

Is there any element in Azure Red Hat OpenShift shared with other customers? Or is everything independent?

Each Azure Red Hat OpenShift cluster is dedicated to a given customer and lives within the customer's subscription.

Are infrastructure nodes available?

Yes, ARO allows you to use infrastructure machine sets to create machines that only host infrastructure components, such as the default router, the integrated container registry, and the components for cluster metrics and monitoring. See Deploy infrastructure nodes in an ARO cluster to learn more.

How do I handle cluster upgrades?

For information on upgrades, maintenance, and supported versions, see the support lifecycle guide.

How will the host operating system and OpenShift software be updated?

The host operating systems and OpenShift software are updated as Azure Red Hat OpenShift consumes minor release versions and patches from upstream OpenShift Container Platform.

What’s the process to reboot the updated node?

Nodes are rebooted as a part of an upgrade.

Cluster operations

Can I use Prometheus to monitor my applications?

Prometheus comes pre-installed and configured for Azure Red Hat OpenShift 4.x clusters. Read more about cluster monitoring.

In Azure Red Hat OpenShift 4.x: Yes.

Can logs of underlying VMs be streamed out to a customer log analysis system?

Logs from underlying VMs are handled by the managed service and aren't exposed to customers.

How can a customer get access to metrics like CPU/memory at the node level to take action to scale, debug issues, etc.? I can’t seem to run kubectl top on an Azure Red Hat OpenShift cluster.

For Azure Red Hat OpenShift 4.x clusters, the OpenShift web console contains all metrics at the node level. For more information, see the Red Hat documentation on viewing cluster information.

If we scale up the deployment, how do Azure fault domains map into pod placement to ensure all pods for a service don't get knocked out by a failure in a single fault domain?

There are by default five fault domains when using Virtual Machine Scale Sets in Azure. Each virtual machine instance in a scale set will get placed into one of these fault domains. This ensures that applications deployed to the compute nodes in a cluster will get placed in separate fault domains.

For more information, see Choosing the right number of fault domains for Virtual Machine Scale Set.

Is there a way to manage pod placement?

Customers have the ability to get nodes and view labels as the customer-admin. This will provide a way to target any VM in the scale set.

Caution must be used when using specific labels:

  • Hostname must not be used. Hostname gets rotated often with upgrades and updates and is guaranteed to change.
  • If the customer has a request for specific labels or a deployment strategy, this could be accomplished. However, it would require engineering efforts, and it isn't supported today.

For more information, see Controlling pod placement.

Is the image registry available externally so I can use tools such as Jenkins?

For 4.x clusters, you need to expose a secure registry and configure authentication. For more information, see the following Red Hat documentation:

Can I move/migrate my cluster between Azure tenants?

Moving your ARO cluster between tenants is currently unsupported.

Can I move my ARO clusters from the current Azure subscription to another?

Moving your ARO cluster and its associated resources between subscriptions isn't supported.

Can I move my ARO clusters or ARO infrastructure resources to other resource groups or rename them?

Moving or renaming your ARO cluster and its associated resources isn't supported.

Networking

Can I deploy a cluster into an existing virtual network?

In 4.x clusters, you can deploy a cluster into an existing VNet.

Is cross-namespace networking supported?

Customer and individual project admins can customize cross-namespace networking (including denying it) on a per-project basis using NetworkPolicy objects.

I'm trying to peer into a virtual network in a different subscription but getting Failed to get VNet CIDR error.

In the subscription that has the virtual network, make sure to register Microsoft.ContainerService provider with the following command: az provider register -n Microsoft.ContainerService --wait

Can we specify IP ranges for deployment on the private VNet, avoiding clashes with other corporate VNets once peered?

In 4.x clusters, you can specify your own IP ranges.

Is the Software Defined Network module configurable?

The Software Defined Network is openshift-ovs-networkpolicy and isn't configurable.

What Azure Load balancer is used by Azure Red Hat OpenShift? Is it Standard or Basic and is it configurable?

Azure Red Hat OpenShift uses Standard Azure Load Balancer, and it isn't configurable.

Permissions

Can an admin manage users and quotas?

Yes. An Azure Red Hat OpenShift administrator can manage users and quotas in addition to accessing all user created projects.

Can I restrict a cluster to only certain Microsoft Entra users?

Yes. You can restrict which Microsoft Entra users can sign in to a cluster by configuring the Microsoft Entra Application. For details, see How to: Restrict your app to a set of users.

Can I restrict users from creating projects?

Yes. Sign in to your cluster as an administrator and execute this command:

oc adm policy \
    remove-cluster-role-from-group self-provisioner \
    system:authenticated:oauth

For more information, see the OpenShift documentation on disabling self-provisioning for your cluster version:

Which UNIX rights (in IaaS) are available for Masters/Infra/App Nodes?

Node access is available through the cluster-admin role. For more information, see Kubernetes RBAC overview.

Which OCP rights do we have? Cluster-admin? Project-admin?

The cluster-admin role is available. For more information, see Kubernetes RBAC overview.

Which identity providers are available?

You configure your own identity provider. For more information, see the Red Hat documentation on configuring identity providers.

Storage

Is data on my cluster encrypted?

By default, data is encrypted at rest. The Azure Storage platform automatically encrypts your data before persisting it, and decrypts the data before retrieval. For more information, see Azure Storage Service Encryption for data at rest.

How are my storage accounts secured?

Storage accounts are set to private access only.

Storage accounts are encrypted (new clusters only). Existing clusters need to be re-created.

Storage accounts are created with general-purpose v2 for new clusters.

General-purpose v2 storage accounts support the latest Azure Storage features and incorporate all the functionality of general-purpose v1 and Blob storage accounts.

Storage accounts access is limited with firewall rules via Azure network security groups (NSGs), which filter network traffic to and from your storage accounts. For more information, see Azure network security groups overview.

Transport Layer Security (TLS) protocol version 1.2 provides secure communications, data privacy, and data integrity.

Is data stored in etcd encrypted on Azure Red Hat OpenShift?

Data isn't encrypted by default, but you do have the option to enable encryption. For more information, see the guide on encrypting etcd.

Can we choose any persistent storage solution, like OCS?

Azure Disk (Premium_LRS) is configured as the default storage class. For additional storage providers, and for configuration details (including Azure File), see the Red Hat documentation on persistent storage.

Does ARO store any customer data outside of the cluster's region?

No. All data created in an ARO cluster is maintained within the cluster's region.