Install software and set up resources for the tutorial

In this three-part tutorial, you'll create a machine learning model to predict iron ore impurities as a percentage of Silica, and then deploy the model in Azure SQL Edge. In part one, you'll install the required software and deploy Azure resources.

Prerequisites

  1. If you don't have an Azure subscription, create a free account.
  2. Install Visual Studio 2019 with
    • Azure IoT Edge tools
    • .NET core cross-platform development
    • Container development tools
  3. Install Azure Data Studio
  4. Open Azure Data Studio and configure Python for notebooks. For details, see Configure Python for Notebooks. This step can take several minutes.
  5. Install the latest version of Azure CLI. The following scripts require that AZ PowerShell be the latest version (3.5.0, Feb 2020).
  6. Set up the environment to debug, run, and test IoT Edge solution by installing Azure IoT EdgeHub Dev Tool.
  7. Install Docker.
  8. Download the DACPAC file that will be utilized in the tutorial.

Deploy Azure resources using PowerShell Script

Deploy the Azure resources required by this Azure SQL Edge tutorial. These can be deployed either by using a PowerShell script or through the Azure portal. This tutorial uses a PowerShell script.

  1. Import the modules needed to run the PowerShell script in this tutorial.

    Import-Module Az.Accounts -RequiredVersion 1.7.3
    Import-Module -Name Az -RequiredVersion 3.5.0
    Import-Module Az.IotHub -RequiredVersion 2.1.0
    Import-Module Az.Compute -RequiredVersion 3.5.0
    az extension add --name azure-iot
    az extension add --name azure-cli-ml
    

    Note

    This article uses the newest version of the Azure IoT extension, called azure-iot. The legacy version is called azure-cli-iot-ext.You should only have one version installed at a time. You can use the command az extension list to validate the currently installed extensions.

    Use az extension remove --name azure-cli-iot-ext to remove the legacy version of the extension.

    Use az extension add --name azure-iot to add the new version of the extension.

    To see what extensions you have installed, use az extension list.

  2. Declare the variables required by the PowerShell script.

    $ResourceGroup = "<name_of_the_resource_group>"
    $IoTHubName = "<name_of_the_IoT_hub>"
    $location = "<location_of_your_Azure_Subscription>"
    $SubscriptionName = "<your_azure_subscription>"
    $NetworkSecGroup = "<name_of_your_network_security_group>"
    $StorageAccountName = "<name_of_your_storage_account>"
    
  3. Declare the rest of the variables.

    $IoTHubSkuName = "S1"
    $IoTHubUnits = 4
    $EdgeDeviceId = "IronOrePredictionDevice"
    $publicIpName = "VMPublicIP"
    $imageOffer = "iot_edge_vm_ubuntu"
    $imagePublisher = "microsoft_iot_edge"
    $imageSku = "ubuntu_1604_edgeruntimeonly"
    $AdminAcc = "iotadmin"
    $AdminPassword = ConvertTo-SecureString "IoTAdmin@1234" -AsPlainText -Force
    $VMSize = "Standard_DS3"
    $NetworkName = "MyNet"
    $NICName = "MyNIC"
    $SubnetName = "MySubnet"
    $SubnetAddressPrefix = "10.0.0.0/24"
    $VnetAddressPrefix = "10.0.0.0/16"
    $MyWorkSpace = "SQLDatabaseEdgeDemo"
    $containerRegistryName = $ResourceGroup + "ContRegistry"
    
  4. To begin creating assets, log in to Azure.

    Login-AzAccount
    
    az login
    
  5. Set the Azure subscription ID.

    Select-AzSubscription -Subscription $SubscriptionName
    az account set --subscription $SubscriptionName
    
  6. Create the resource group if it doesn't already exist.

    $rg = Get-AzResourceGroup -Name $ResourceGroup
    if($rg -eq $null)
    {
        Write-Output("Resource Group $ResourceGroup does not exist, creating Resource Gorup")
        New-AzResourceGroup -Name $ResourceGroup -Location $location
    }
    else
    {
        Write-Output ("Resource Group $ResourceGroup exists")
    }
    
  7. Create the storage account and storage account container in the resource group.

    $sa = Get-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
    if ($sa -eq $null)
    {
        New-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName -SkuName Standard_LRS -Location $location -Kind Storage
        $sa = Get-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
        $storagekey = Get-AzStorageAccountKey -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
        $storageContext = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $storagekey[0].Value
        New-AzStorageContainer -Name "sqldatabasedacpac" -Context $storageContext 
    }
    else
    {
       Write-Output ("Storage Account $StorageAccountName exists in Resource Group $ResourceGroup")     
    }
    
  8. Upload the database dacpac file to the storage account and generate a SAS URL for the blob. Make a note of the SAS URL for the database dacpac blob.

    $file = Read-Host "Please Enter the location to the zipped Database DacPac file:"
    Set-AzStorageBlobContent -File $file -Container "sqldatabasedacpac" -Blob "SQLDatabasedacpac.zip" -Context $sa.Context
    $DacpacFileSASURL = New-AzStorageBlobSASToken -Container "sqldatabasedacpac" -Blob "SQLDatabasedacpac.zip" -Context $sa.Context -Permission r -StartTime (Get-Date).DateTime -ExpiryTime (Get-Date).AddMonths(12) -FullUri
    
  9. Create an Azure container registry within this resource group.

    $containerRegistry = Get-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName 
    if ($containerRegistry -eq $null)
    {
        New-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName -Sku Standard -Location $location -EnableAdminUser 
        $containerRegistry = Get-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName 
    }
    else
    {
        Write-Output ("Container Registry $containerRegistryName exists in Resource Group $ResourceGroup")
    }
    
  10. Create the network security group within the resource group.

    $nsg = Get-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroup -Name $NetworkSecGroup 
    if($nsg -eq $null)
    {
        Write-Output("Network Security Group $NetworkSecGroup does not exist in the resource group $ResourceGroup")
    
        $rule1 = New-AzNetworkSecurityRuleConfig -Name "SSH" -Description "Allow SSH" -Access Allow -Protocol Tcp -Direction Inbound -Priority 100 -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 22
        $rule2 = New-AzNetworkSecurityRuleConfig -Name "SQL" -Description "Allow SQL" -Access Allow -Protocol Tcp -Direction Inbound -Priority 101 -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 1600
        New-AzNetworkSecurityGroup -Name $NetworkSecGroup -ResourceGroupName $ResourceGroup -Location $location -SecurityRules $rule1, $rule2
    
        $nsg = Get-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroup -Name $NetworkSecGroup
    }
    else
    {
        Write-Output ("Network Security Group $NetworkSecGroup exists in the resource group $ResourceGroup")
    }
    
  11. Create an Azure virtual machine enabled with SQL Edge. This VM will act as an Edge device.

    $AzVM = Get-AzVM -ResourceGroupName $ResourceGroup -Name $EdgeDeviceId
    If($AzVM -eq $null)
    {
        Write-Output("The Azure VM with Name- $EdgeVMName is not present in the Resource Group- $ResourceGroup ")
    
        $SingleSubnet = New-AzVirtualNetworkSubnetConfig -Name $SubnetName -AddressPrefix $SubnetAddressPrefix
        $Vnet = New-AzVirtualNetwork -Name $NetworkName -ResourceGroupName $ResourceGroup -Location $location -AddressPrefix $VnetAddressPrefix -Subnet $SingleSubnet
        $publicIp = New-AzPublicIpAddress -Name $publicIpName -ResourceGroupName $ResourceGroup -AllocationMethod Static -Location $location  
        $NIC = New-AzNetworkInterface -Name $NICName -ResourceGroupName $ResourceGroup -Location $location -SubnetId $Vnet.Subnets[0].Id -NetworkSecurityGroupId $nsg.Id -PublicIpAddressId $publicIp.Id
    
        ##Set-AzNetworkInterfaceIpConfig -Name "ipconfig1"  -NetworkInterface $NIC -PublicIpAddress $publicIp
    
        $Credential = New-Object System.Management.Automation.PSCredential ($AdminAcc, $AdminPassword);
    
        $VirtualMachine = New-AzVMConfig -VMName $EdgeDeviceId -VMSize $VMSize
        $VirtualMachine = Set-AzVMOperatingSystem -VM $VirtualMachine -Linux -ComputerName $EdgeDeviceId -Credential $Credential
        $VirtualMachine = Add-AzVMNetworkInterface -VM $VirtualMachine -Id $NIC.Id  
        $VirtualMachine = Set-AzVMSourceImage -VM $VirtualMachine -PublisherName $imagePublisher -Offer $imageOffer -Skus $imageSku -Version latest 
        $VirtualMachine = Set-AzVMPlan -VM $VirtualMachine -Name $imageSku -Publisher $imagePublisher -Product $imageOffer
    
        $AzVM = New-AzVM -ResourceGroupName $ResourceGroup -Location $location -VM $VirtualMachine -Verbose
        $AzVM = Get-AzVM -ResourceGroupName $ResourceGroup -Name $EdgeDeviceId
    
    }
    else
    {
        Write-Output ("The Azure VM with Name- $EdgeDeviceId is present in the Resource Group- $ResourceGroup ")
    }
    
  12. Create an IoT hub within the resource group.

    $iotHub = Get-AzIotHub -ResourceGroupName $ResourceGroup -Name $IoTHubName
    If($iotHub -eq $null)
    {
        Write-Output("IoTHub $IoTHubName does not exists, creating The IoTHub in the resource group $ResourceGroup")
        New-AzIotHub -ResourceGroupName $ResourceGroup -Name $IoTHubName -SkuName $IoTHubSkuName -Units $IoTHubUnits -Location $location -Verbose
    }
    else
    {
        Write-Output ("IoTHub $IoTHubName present in the resource group $ResourceGroup") 
    }
    
  13. Add an Edge device to the IoT hub. This step only creates the device digital identity.

    $deviceIdentity = Get-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId
    If($deviceIdentity -eq $null)
    {
        Write-Output("The Edge Device with DeviceId- $EdgeDeviceId is not registered to the IoTHub- $IoTHubName ")
        Add-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId -EdgeEnabled  
    }
    else
    {
        Write-Output ("The Edge Device with DeviceId- $EdgeDeviceId is registered to the IoTHub- $IoTHubName")
    }
    $deviceIdentity = Get-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId
    
  14. Get the device primary connection string. This will be needed later for the VM. The following command uses Azure CLI for deployments.

    $deviceConnectionString = az iot hub device-identity show-connection-string --device-id $EdgeDeviceId --hub-name $IoTHubName --resource-group $ResourceGroup --subscription $SubscriptionName
    $connString = $deviceConnectionString[1].Substring(23,$deviceConnectionString[1].Length-24)
    $connString
    
  15. Update the connection string in the IoT Edge configuration file on the Edge device. The following commands use Azure CLI for deployments.

    $script = "/etc/iotedge/configedge.sh '" + $connString + "'"
    az vm run-command invoke -g $ResourceGroup -n $EdgeDeviceId  --command-id RunShellScript --script $script
    
  16. Create an Azure Machine Learning workspace within the resource group.

    az ml workspace create -w $MyWorkSpace -g $ResourceGroup
    

Next Steps