I believe this may just be an issue with your build server. Normally when you restore packages from NuGet it caches the package on the machine so it doesn't ever have to do it again. This is done independent of any solutions or projects you build. Hence cleaning the solution/projects directory doesn't touch the Nuget package cache.
On the build server, go to the %user%\.nuget\packages
folder for whatever user account your build is running under. Find the package that is causing the problem and delete the folder. Then try running your build again. It should download a new copy of the package. Alternatively you can just run nuget install microsoft.extensions.logging.abstractions -version 7.0.1
to pull down the package into the cache.
My assumption here is that you're just using the public nuget.org source and not your own private feed. However I'm also assuming you don't have any customizations like nuget source mapping or whatnot contained in a nuget.config related to your project.
If that still doesn't work then I'm going to start wondering about a consolidation issue in your solution such that multiple projects are referencing different versions of the package. But that doesn't seem to line up with the error you're getting here.