I was able to connect my custom container through ssh following the instructions https://docs.microsoft.com/en-us/azure/app-service/containers/configure-custom-container#enable-ssh but after I put the App Service in a virtual network it stopped working as the port 2222 changed