As a conclusive answer to this:
- Ensure connection policy is set to default (redirect), check it at Firewalls and virtual networks setting in Az SQL DB.
- Use service tag for SQL in nsg rule , same applies in case of app service inetgrated with vnet.
- If you have a JDBC connection string, ensure correct version of driver.
- Do not hardcode IP addresses in your on-premise firewall, use domain based rule: "*.database.windows.net" on port 1433.
- IF you have enabled service endpoint for subnet, then ensure the members of subnet can still contact gateway using NSG rules.
- If zone redundant configuration is used, then make sure all three gateway rings have NSG rule based access.