When deleting the resource group with the "az" command line, after letting it run for a long time (I came back 1h30m later), it timeout and gives a clear error message: {"error":{"code":"ServiceResourceNotEmptyWithBackendMessage","message":"Recovery Services Vault cannot be deleted as there are existing resources within the vault. : foilenlabme. Please ensure all containers have been unregistered from the vault and all private endpoints associated with the vault have been deleted, and retry operation. For more details, see https://aka.ms/AB-AA4ecq5","target":null,"details":null,"innerError":null}}
Since the portal doesn't work with the Recovery Services Vault, I will try with the "az" command line to clean it up and delete it.
It is really sad that I cannot report that bug to Microsoft without paying for "support"...