Hello @roy lee ,
Yes , You can use Azure AD for authentication directly . You can setup Password Hash sync and Seamless SSO and you can get all the benefits of single sign-on without ADFS . while configuring the User sign-in you can configure it with Password hash sync and tick the checkbox called enable single sign-on
The above would work for setting up seamless SSO . But this setup without ADFS works best if you have all windows 10 devices with latest updates. You should make sure that If you have any application in your on-premise which depends on ADFS for authentication then you would need to migrate them to the cloud or use Azure AD application proxy for the same. The application migration for any app that was dependent on ADFS would need to be checked and applications updated accordingly. even though Azure AD supports all the auth protocols as ADFS , you may need to make some changes in the application code before migration. Any application which works on legacy protocols like Kerberos , NTLM in your local on-premise environment will continue to be accessed in same way as they will require you to make sure that the application has a line of sight AD domain controller for authentication.
If all your applications are already in azure and use azure AD for identity authentication/authorization and you have all clients as windows 10 (latest) hybrid azure AD joined then you do not need to worry about at all while removing ADFS. You can use the AD connect to setup PHS as well as seamless SSO by checking that option of enable Single sign-on . We have a great article for migrating from federation to Password hash sync for Azure AD which explains it all with all pre-requisites. In case you have any managed domains then you can setup hybrid Azure AD join for them as well. Please go through the articles and I am sure you will be able to plan your migration to Azure AD from ADFS better.
Hope the above clarifies your query and answers your doubts. If the information provided in the answer helps you , please do accept it as answer so that it increases the relevancy of the answer and it is easily found by community members searching for similar issues.
Thank you.