Azure relay/hybrid not supported by containers as it requires support code (a tcp reverse proxy client) in the app service hosting. that is, its an app service feature only. there are samples of calling the hybrid connection directly, so in theory you could write the sqlserver reverse proxy client yourself and host in the container.
https://github.com/Azure/azure-relay/tree/master/samples/hybrid-connections/dotnet
you will probably need a more traditional vpn solution:
https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways