@Pascal Welcome to Microsoft Q&A forum, Thank you for posting your query here!
How much data are you downloading? You can use Azure speed test tool to check the speed: https://www.azurespeed.com/Azure/Download
For testing purpose can you use azcopy tool and check the performance.
This article lists some common problems related to Azure file shares. It provides potential causes and workarounds for when you encounter these problems. Azure file shares performance issues
Additional information: There is no throttling on the Azure side. If the above suggestion doesn't help you! I would recommend you create a technical support ticket to find the root cause of the issue. Since you are getting better speed in the other format The ticket enables you to work closely with the support engineers and get a quick resolution for your issue. If you don't have the support plan we will try and help you get a one-time free technical support. In this case, could you send an email to AzCommunity[at]Microsoft[dot]com referencing this thread as well as your subscription ID. Please mention "ATTN subm" in the subject field.
Thank you for your cooperation on this matter and look forward to your reply.
Hope this helps!
Kindly let us know if the above helps or you need further assistance on this issue.
----------------------------------------------------------------------------------------------------------------------------------------------------
Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.