Azure CycleCloud QuickStarts

There are four parts to the Azure CycleCloud QuickStart:

  1. Install and setup CycleCloud on a Virtual Machine
  2. Configure and create a simple HPC cluster consisting of a job scheduler and an NFS file server, and create a usage alert to monitor cost
  3. Submit jobs to observe the cluster autoscale up and down automatically
  4. Clean up resources

Working through all the quickstarts should take 60 to 90 minutes. You will get the most out of them if they are done in order. As most HPC environments run on Linux, this quickstart assumes basic Linux familiarity.

QuickStart 1: Install and Setup Azure CycleCloud

Azure CycleCloud is a free application that provides a simple, secure, and scalable way to manage compute and storage resources for HPC and Big Compute/Data workloads. In this quickstart, you will install CycleCloud on Azure resources, using an Azure Resource Manager (ARM) template that is stored on GitHub. The ARM template:

  1. Deploys a virtual network with three separate subnets:
    • cycle: The subnet in which the CycleCloud server is started in
    • compute: A /22 subnet for the HPC clusters
    • user: The subnet for creating user logins
  2. Provisions a VM in the cycle subnet and installs Azure CycleCloud on it.

For the purposes of this quickstart, much of the setup has been done via the ARM template. However, CycleCloud can also be installed manually, providing greater control over the installation and configuration process. For more information, see the Manual CycleCloud Installation documentation.

Prerequisites

For this quickstart, you will need:

  1. An Azure account with an active subscription.
  2. A Shell session in a terminal.
    • If you are using a Windows machine, use the browser-based Bash shell.
    • For non-Windows machines, install and use Azure CLI v2.0.20 or later. Run az --version to find your current version. If you need to install or upgrade, see Install Azure CLI 2.0.

Open Azure Cloud Shell

Azure Cloud Shell is a free, interactive shell that you can use to run the steps in this article. Common Azure tools are preinstalled and configured in Cloud Shell for you to use with your account. Just select the Copy button to copy the code, paste it in Cloud Shell, and then press Enter to run it. There are a few ways to open Cloud Shell:

Click the green "Try It" button in the upper right corner of a code block. Cloud Shell in this article
Open Cloud Shell in your browser. Launch Cloud Shell
Select the Cloud Shell button on the menu in the upper-right corner of the Azure portal. Cloud Shell in the portal

Note

The "Try It" button opens a Cloud Shell in your current browser window. It does not enter the command for you. You will need to click the "Copy" button to save to your clipboard, then paste the command into your Shell.

Select Bash as the shell type. If you are prompted to create storage, the default option is fine.

Service Principal

Azure CycleCloud requires a service principal with contributor access to your Azure subscription. If you do not have a service principal available, you can create one now. Note that your service principal name must be unique - in the example below, CycleCloudApp can be replaced with whatever you like:

az ad sp create-for-rbac --name CycleCloudApp --years 1

The output will display a series of information. You will need to save the appId, password, and tenant:

"appId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"displayName": "CycleCloudApp",
"name": "http://CycleCloudApp",
"password": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"tenant": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

SSH Keypair

An SSH key is needed to log into the CycleCloud VM and clusters. Generate an SSH keypair:

ssh-keygen -f ~/.ssh/id_rsa  -N "" -b 4096

Retrieve the SSH public key with:

cat ~/.ssh/id_rsa.pub

The output will begin with ssh-rsa followed by a long string of characters. Copy and save this key now.

On Linux, follow these instructions on GitHub to generate a new ssh keypair.

Deploy Azure CycleCloud

Click the button below to deploy Azure CycleCloud into your subscription:

Enter the required information.

Basics:

  • Subscription: If you have more than one active Azure subscription, select the one to use here
  • Resource Group: Enter the name of a (new) resource group that will hold everything generated by this quickstart
  • Location: Select a region in which to store your instance

Settings:

  • Tenant ID: The tenant from the service principal above
  • Application ID: The appId from the service principal above
  • Application Secret: The password from the service principal above
  • SSH Public Key: The public key used to log into the CycleCloud VM
  • Username: The username for the CycleCloud VM. Use your Azure Portal username without the domain (e.g. johnsmith instead of johnsmith@domain.com)

The remaining fields can be left as is. Agree to the terms and conditions, and click Purchase. Note that you are paying for the core hours used in Azure, and not the CycleCloud product (which is free).

The deployment process runs an installation script as a custom script extension, which installs and sets up CycleCloud. This process takes between 5 and 8 mins.

Log into the CycleCloud Application Server

Run export RESOURCE_GROUP="the name you gave your resource group". This will create allow you to copy the next command and enter it without modifying it.

To connect to the CycleCloud webserver, retrieve the Fully Qualified Domain Name (FQDN) of the CycleServer VM from either the Azure Portal or using the CLI:

az network public-ip show -g ${RESOURCE_GROUP?} -n cycle-ip --query dnsSettings.fqdn

Browse to https://[fqdn]/. The installation uses a self-signed SSL certificate, which may show up with a warning in your browser.

Create a Site Name for your installation. You can use any name here:

CycleCloud Welcome screen

The Azure CycleCloud End User License Agreement will be displayed - click to accept it. You will then need to create a CycleCloud admin user for the application server. We recommend using the same username used above. Ensure the password you enter meets the requirements listed. Click Done to continue.

CycleCloud Create New User screen

That's the end of QuickStart 1, which covered the installation and setup of Azure CycleCloud via ARM Template.