Edit

Troubleshoot Azure Files issues in Linux (SMB)

  • Article

This article lists common issues that can occur when using SMB Azure file shares with Linux clients. It also provides possible causes and resolutions for these problems.

You can use AzFileDiagnostics to automate symptom detection and ensure that the Linux client has the correct prerequisites. It helps set up your environment to get optimal performance. You can also find this information in the Azure file shares troubleshooter.

Important

This article only applies to SMB shares. For details on NFS shares, see Troubleshoot NFS Azure file shares.

Applies to

File share type SMB NFS
Standard file shares (GPv2), LRS/ZRS
Standard file shares (GPv2), GRS/GZRS
Premium file shares (FileStorage), LRS/ZRS

Time stamps were lost when copying files

On Linux/Unix platforms, the cp -p command fails if different users own file 1 and file 2.

Cause

The force flag f in COPYFILE results in executing cp -p -f on Unix. This command also fails to preserve the time stamp of the file that you don't own.

Workaround

Use the storage account user to copy the files:

  • str_acc_name=[storage account name]
  • sudo useradd $str_acc_name
  • sudo passwd $str_acc_name
  • su $str_acc_name
  • cp -p filename.txt /share

ls: cannot access '<path>': Input/output error

When you try to list files in an Azure file share by using the ls command, the command hangs when listing files. You get the following error:

ls: cannot access'<path>': Input/output error

Solution

Upgrade the Linux kernel to the following versions that have a fix for this problem:

  • 4.4.87+
  • 4.9.48+
  • 4.12.11+
  • All versions that are greater than or equal to 4.13

Can't create symbolic links - ln: failed to create symbolic link 't': Operation not supported

Cause

By default, mounting Azure file shares on Linux by using SMB doesn't enable support for symbolic links (symlinks). You might see an error like this:

sudo ln -s linked -n t

ln: failed to create symbolic link 't': Operation not supported

Solution

The Linux SMB client doesn't support creating Windows-style symbolic links over the SMB 2 or 3 protocol. Currently, the Linux client supports another style of symbolic links called Minshall+French symlinks for both create and follow operations. Customers who need symbolic links can use the "mfsymlinks" mount option. We recommend "mfsymlinks" because it's also the format that Macs use.

To use symlinks, add the following to the end of your SMB mount command:

,mfsymlinks

So the command looks like the following:

sudo mount -t cifs //<storage-account-name>.file.core.windows.net/<share-name> <mount-point> -o vers=<smb-version>,username=<storage-account-name>,password=<storage-account-key>,dir_mode=0777,file_mode=0777,serverino,mfsymlinks

You can then create symlinks as suggested on the wiki.

Unable to access folders or files which name has a space or a dot at the end

You can't access folders or files from the Azure file share while mounted on Linux. Commands like du and ls and/or third-party applications might fail with a "No such file or directory" error while accessing the share; however, you're able to upload files to these folders via the Azure portal.

Cause

The folders or files were uploaded from a system that encodes the characters at the end of the name to a different character. Files uploaded from a Macintosh computer may have a "0xF028" or "0xF029" character instead of 0x20 (space) or 0X2E (dot).

Solution

Use the mapchars option on the share when mounting the share on Linux:

Instead of:

sudo mount -t cifs $smbPath $mntPath -o vers=3.0,username=$storageAccountName,password=$storageAccountKey,serverino

Use:

sudo mount -t cifs $smbPath $mntPath -o vers=3.0,username=$storageAccountName,password=$storageAccountKey,serverino,mapchars

DNS issues with live migration of Azure storage accounts

File I/Os on the mounted filesystem start giving "Host is down" or "Permission denied" errors. Linux dmesg logs on the client show repeated errors like:

Status code returned 0xc000006d STATUS_LOGON_FAILURE
cifs_setup_session: 2 callbacks suppressed
CIFS VFS: \\contoso.file.core.windows.net Send error in SessSetup = -13

You'll also see that the server FQDN now resolves to a different IP address than what it's currently connected to.

Cause

For capacity load balancing purposes, storage accounts are sometimes live-migrated from one storage cluster to another. Account migration triggers Azure Files traffic to be redirected from the source cluster to the destination cluster by updating the DNS mappings to point to the destination cluster. This blocks all traffic to the source cluster from that account. It's expected that the SMB client picks up the DNS updates and redirects further traffic to the destination cluster. However, due to a bug in the Linux SMB kernel client, this redirection doesn't take effect. As a result, the data traffic keeps going to the source cluster, which has stopped serving this account post migration.

Workaround

You can mitigate this issue by rebooting the client OS, but you might run into the issue again if you don't upgrade your client OS to a Linux distro version with account migration support. Note that umount and remount of the share might appear to fix the issue temporarily.

To better work around this issue, clear the kernel DNS resolver cache:

  1. Display the status of the kernel dns_resolver module by running the following command:

    grep '.dns_resolver' /proc/keys

    You should see command output like the following example:

    132b6bbf I------     1 perm 1f030000     0     0 keyring   .dns_resolver: 1

  2. Clear the kernel DNS resolver cache by running the following command:

    sudo keyctl clear $((16#$(grep '.dns_resolver' /proc/keys | cut -f1 -d\ ) ))

  3. Display the status of the kernel dns_resolver module again:

    grep '.dns_resolver' /proc/keys

    You should see command output like the following example, indicating that the cache is now empty:

    132b6bbf I------     1 perm 1f030000     0     0 keyring   .dns_resolver: empty

Solution

For a permanent fix, upgrade your client OS to a Linux distro version with account migration support. Several fixes for the Linux SMB kernel client have been submitted to the mainline Linux kernel. Kernel version 5.15+ and Keyutils-1.6.2+ have the fixes. Some distros have backported these fixes, and you can check if the following fixes exist in the distro version you're using:

Unable to mount SMB file share when FIPS is enabled

When Federal Information Processing Standard (FIPS) is enabled in a Linux VM, the SMB file share cannot be mounted. The Linux dmesg logs on the client display errors such as:

kernel: CIFS: VFS: Could not allocate crypto hmac(md5)
kernel: CIFS: VFS: Error -2 during NTLMSSP authentication
kernel: CIFS: VFS: \\contoso.file.core.windows.net Send error in SessSetup = -2
kernel: CIFS: VFS: cifs_mount failed w/return code = -2

Important

FIPS is a set of standards that the U.S. government uses to ensure the security and integrity of computer systems. When a system is in FIPS mode, it adheres to specific cryptographic requirements outlined by these standards.

Cause

The client of SMB file share uses the NTLMSSP authentication, which requires the MD5 hashing algorithm. However, in FIPS mode, the MD5 algorithm is restricted because it's not FIPS-compliant. MD5 is a widely used hash function that produces a 128-bit hash value. However, MD5 is considered insecure for cryptographic purposes.

How to check if FIPS mode is enabled

To verify if FIPS mode is enabled on the client, run the following command. If the value is set to 1, then FIPS is enabled.

sudo cat /proc/sys/crypto/fips_enabled

Solution

To resolve this issue, enable Kerberos authentication for SMB file share. If FIPS is enabled unintentionally, refer to option2 to disable it.

Option 1: Enable Kerberos authentication for SMB file share

To mount a SMB file share on the Linux VM where FIPS is enabled, use Kerberos/Azure AD authentication. For more information, see Enable Active Directory authentication over SMB for Linux clients accessing Azure Files.

Option 2: Disable FIPS to mount the Samba share

  1. Change the sysctl value of crypto.fips_enabled to 0 in /etc/sysctl.conf.

  2. Modify the GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub file and remove the parameter fips=1.

  3. Rebuilt the grub2 config file with the following command:

    sudo grub2-mkconfig -o /boot/grub2/grub.cfg

  4. Rebuild the initramfs image with the following command:

    sudo dracut -fv

  5. Reboot the VM.

For more information, see to the following documents from Linux distributors:

Need help?

If you still need help, contact support to get your problem resolved quickly.

See also

Contact us for help

If you have questions or need help, create a support request, or ask Azure community support. You can also submit product feedback to Azure feedback community.