I found a workaround for anyone interested.
From the MS Account Security Dashboard, turn Off passwordless account, and set up a new password.
On the device you are trying to log into (if you set it up without a password during install at least) reboot and click "Forgot PIN"
From the recovery screen, choose to log in with a password, do so.
When it prompts you to change your PIN just click cancel. It will log you in to the device, and now associate the password with the device.
You should now be able to RDP in with the newly set password.
Looks like they know it's an issue too. Typical.
It doesn't accomplish using passwordless auth, but it does associate a password to a device that was initially set up passwordless, and makes RDP usable again.