There is no App server...
I had an opportunity recently to ask (repeatedly) various experts and product group team members a very simple question: Why do we (Microsoft) purpetuate the "N tier" (web/app/data) myth in SharePoint?
I'm not crazy... but I'm not living in 1998 anymore either. Back then (weird to say, isn't it?), everyone was on the "everything should be a SOAP web service!" bandwagon. It provided flexibility, reusability, data integrity, and all the other -ities you could ever need... and created the essence of the 3 tier architecture. Your data lived in your database and nothing touched your data except for a set of SOAP web services. Anyone could call your web services because they ensured that the business logic was always followed, regardless of the requesting application. You could let ANYONE use your web services because your application's rules would always be followed. Then, you created the primary web interface that would call the web services. It made things pretty and usable... but no actual data was placed in those pages... all data and functionality was actually handled by the web services. This created the "Presentation", "Application" (aka, Business Logic), and "Data" tiers. It was and continues to be a reasonable model.
But it isn't SharePoint... and hasn't been for years.
I know... you don't believe me. You'll say "But I have an Application server! It does the indexing for search! It does my Profile Sync! Users don't access it... it's 'back end'!" Yeah... I get it... but if you're having this debate in your head, put simply, you're wrong. You may have attempted to follow the 3 tier model with SharePoint... but it's not helping you. You can stack the Lego-that-is-SharePoint in whatever way you want, including into this antiquated model... but SharePoint is much more flexible than that.
I have one word to prove I'm right: ROLES.
SharePoint is a role based system. You take your various servers and you activate on each of those machines the roles you want that server to contribute to the overall environment. The key issue here is that any server can host any role (almost) any time. The roles can live anywhere. For example, the fact that the "Web Front End" role ("Presentation") is activated on a machine doesn't prevent you from also activating the Search Query role/component (search classically being considered an "application server" role) on that machine. In fact, this would ba a commonly deployed and frequently recommended topology. So... is that server now a "Web" server or an "App" server?
Also, in this 3 tier architecture, the web services are offered by the "App" tier... but in SharePoint, the web services are available through the Web Front End. So... is it a web server or an application server?
And finally, in many (most?) cases, the web server is talking directly to the database servers in order to respond to an end user request. They don't go through an intermediate "application" tier in order to do their job... they do it directly. Sure, we have various service applications that also do work... and SharePoint does call these services to perform more specialized functions... but this isn't absolute for all services. This only happens for the various Service Applications that may be created... and these service endpoints live on all of the servers (including those Web Front Ends we mentioned) so our logical separation is again broken.
This brings me to the most valuable point: A well designed SharePoint implementation should not be designed to be specifically aligned with this 3 tier architecture. The SharePoint servers in a given farm should have whatever roles activated on them to meet the needs of the business from the perspectives of performance, overhead, fault tolerance, resource utilization, and maintainability, in additional to SharePoint's own requirements as well, of course. For example, it may make PERFECT SENSE to push several of the roles that would be considered "application server" roles up to the web tier. Excel Calculation Services, InfoPath Form Services, Visio Graphics Service are all great examples of this. You want these services to scale to approximately the same degree as your web tier, so as long as your web servers wouldn't be negatively impacted with the additional load moving them into the web server shouldn't be a problem.
If doing this, it may be wise to ensure you're doing some significant application pool re-use. For example, most of the "services" can be run within a single shared, reused application pool. This optimizes your memory utilization (since the base SharePoint frameworks do not need to be loaded into every application pool) while doing little to your stability (it's all Microsoft code, after all). As always, there are exceptions, so plan and test properly.
So, I propose that we kill the "Application Server" term altogether and start asking ourselves what Roles should be run on each server in the infrastructure, and manage accordingly. This is how SharePoint sees the world... why shouldn't we?
To paraphrase one of my favorite movies:
"Do not try and force SharePoint. That's impossible. Instead... only try to realize the truth."
"There is no App server."
"There is no App server?"
"Then you'll see, that it is not SharePoint that bends, it is only yourself."
(yes, I'm a nerd. :) )