Non-extended public AAM will not allow you to set anonymous access
I had a customer that was having issues setting the anonymous access. While they could access the http://contoso/_layouts/setanon.aspx page, they couldn't actually see the link to that page when navigating to the Site Permissions.
When they were loading up the page, the "Entire Web Site" and "Lists and Libraries" were disabled allowing only the "None" radio button available. As it turns out, they were accessing a correctly configured Alternated Access Mapping as a public URL for their site, however, the 'setanon.aspx' admin page requires the access to be through a zone configured in the "Authentication Providers" for that Web Application.
What this customer had was a Default zone and Internet zone configured correctly (where the Internet zone was a Web Application extension), both of which are under a reverse proxy for external access. The reverse proxy does SSL termination as well so the actual MOSS servers do not have an SSL certificate.
They also had 2 other URLs used internally (Intranet public AAMs) so that it doesn't go through the publishing rule and thus allow administrators to access any server by specifying a host file. While they could have a host file for the published URL, there would be an issue as it expects SSL.
They could have set the anonymous access through the published anonymous URL (while being authenticated). If they would have absolutely liked to have the intranet URL to allow configuring anonymous, they would have had to remove their AAMs and then extend the Web Application for these AAMs. When you extend a Web Application, it creates an Authentication Provider configuration for that zone.
As a site note, if you did configure the anonymous access in the past and then you access through a non-extended URL, the radio buttons will still be disabled but selected (i.e.: Entire Web Site will be disabled but you can only set the value to None and not back to Entire Web Site).