@Michael Hathaway Key Vault service uses a mix of Thales nShield F2 6000+ and Marvell LiquidSecurity HSM cards in the backend for HSM functionality. They are FIPS 140-2 Level 2 or greater validated. The relevant NIST certificates are here (Cryptographic Module Validation Program | CSRC (nist.gov)) and here (Cryptographic Module Validation Program | CSRC (nist.gov)).
This certificate is for the current generation of hardware/firmware. Microsoft regularly upgrades the hardware and firmware behind Azure Key Vault. It may change in future.
It's important to mention that there is no direct access to the HSMs in Azure Key Vault Premium or Azure Key Vault Managed HSM today. We do document the HSMs we're using and their FIPS certificates as above-shared, however, providing some kind of attestation from the HSM of the HSM protecting keys in AKV Premium and MHSM is in something we are considering in the future but not at the moment.