Hi, I suggest you check Authentication Methods in Azure and have a look at Key Restriction Policy -> "Enforce key restrictions" should be se to "No" unless you're limiting usage to specific keys.
What kind of Yubikey are using? If it's an older model you can try to disable "Enforce attestation" if the above doesn't help. This disables the requirement for trusted certificate usage and will allow self-signed certificate of the key itself.