Restricting access to a public ASP.NET Core WebAPI

Question

I'm building an application and I'm having a hard time understanding how to properly restrict access to the API for its data while still allowing the sites and apps that run on that data to function.

The API is on its own domain (say api.mysite.com) and exposes several endpoints for getting data that will show on the site, and would be accessed via HTTP requests.

The public app is on a different domain (say myappsite.com) and needs to hit the api to get the data. My understanding is that I need to enable CORS on the api site, which I've done, so that it can serve the data to the site.

But even though CORS is enabled for only the public app site, I can still hit the api with POSTMAN and a raw HTTP client and see the data.

So my first question is: how does the webapi know that one is coming from the browser and the others are not? I tried copying the network request as cURL using the browser tools and importing that into Postman to try and replicate the browser request, but it still worked, even when I changed the origin to some random string. Why doesn't this get blocked?

But the more important question I have is, how do I prevent the API from being accessible from anything other than my site? I believe I could create a custom middleware that would check for certain headers, and maybe even a custom API key within, but since this would be sent publicly with the site requests, anybody inspecting the network traffic would be able to grab that key and make the requests...

Is the only way to do this by requiring the user to log in so the authorization filters available in webapi can block access? I know that would work, but I want the api data publicly available to my site, such that users do not have to create an account to see or use it...

but I also don't want some random other site using that same public api endpoints to scrape the data on my site and steal the content.

this question is further compounded by the fact that I'd eventually like to build a mobile app for this, and it would hit the exact same endpoints as the website, and it doesn't feel like there's a way to both make the data public but also protected...

Surely I'm not the only site that has public data that they don't want accessible by just anyone, how do other sites achieve this experience?

Answer 1

CORS only effects when browser's javascript calls an api. The restriction is supported by the browser (older browsers do not support CORS).

There is really no way to do what you want. Your api's should not return data that does not show on the page. If the data shows on the page, then any screen scrapper api can be used to get the data. That is, because you have a public webpage, you api is public via the data on the page.

You can make a little harder by requiring a unique request key (that timeouts), that is returned by the page, but this is pretty easy to code up with scrapper api's.

You should never assume your app is requesting a webpage or an api call.

Your best bet is to clearly state the terms and conditions for using the data.

note: the screen scraper libraries and applications are pretty powerful nowadays. many are no-code solutions.