So after some digging around I found my issue. When Azure AD Connect was setup the Azure AD app and attribute filtering was set for only Office 365 ProPlus. This limits the attributes that are synced between on-prem and Azure AD. Office 365 ProPlus does not require the groups or membership to be synced to Azure so while it is pulled into the metaverse in Azure AD Connect it is not synced to Azure. So once I reconfigured it to include Lync (Teams) it pulled all the information that I was missing. Later we will add additional apps we intend to use or remove the filtering completely. I also disable Azure AD Cloud Sync at the same time because as @Andy David - MVP pointed out you have to make sure you are scoped properly to use both mechanisms in the same forest. The reason some of my groups were synced is because Azure AD Cloud Sync did not have any filtering so it pulled in some groups but then had issues because the scoping was not properly done.
The list of attributes that are synced by application can be found at the following url.
https://learn.microsoft.com/en-us/azure/active-directory/hybrid/reference-connect-sync-attributes-synchronized