facebookexternalhit/1.1 (http://www.facebook.com/externalhit_uatext.php) USER AGENT or
LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com) USER AGENT
when trying to crawl gets 500 error.
Solutions tried:
1. Rule set If user-agent begins with facebookexternalhit then delete user-agent Accept-Encoding.
2. Associated WAF rule set to allow user agent that begins with facebookexternalhit to allow traffic.
But the solution is not working as well.