I suspect that hosting companies would actually love caching.  It would greatly reduce CPU and disk load and could even reduce network load.  The amount of memory used could be controlled by changing the expiration time, but even with no expiration (i.e. just explicit removal), I suspect the memory requirement wouldn't be too large.  For example, I just did a quick estimate that it would currently take < 12M to cache every page associated with the mojoportal.com Forums.

Now that I think of it, the trickiest issues with caching are security and personalization.  Those features mean that different users often get different views of the same URL  The simplest way to address that is to use VaryByHeader="Cookie", but that would result in an extra copy in the cache for each user which probably would use too much memory.  The best solution is probably to use VaryByCustom and base the custom string on the user's role most of the time.  Of course if the page being requested contains content that is specific to an individual user (e.g. Logged in as: Username), that isn't going to be sufficient.  Dynamic substituion would help in some cases.

In general, I think caching is probably the best way to improve performance but I think it is a large and relatively complex feature to add.  Basic outline:

• Maintain a last-modified time (preferably in memory) for each module or at least the entire site.  Everytime a module is modified (either content or metadata like permissions), update the last-modified time and invalidate cached pages that depend on that module.
• For each module, maintain (preferably in memory) a list of the sets of roles and users that will see different content.  For example, admins might see one thing, content authors and Bob might see another, and anonymous users might see something else.  So the list would contain 3 sets of roles/users.
• Write Global.GetVaryByCustomString() to get the above information for each of the modules on the requested page, concatenate the last-modified time of each of the modules and the number of the set in each module to which the requesting user belongs, and return a cryptographic hash of the result.
• Allow admins to turn off the display of "Logged in as: Username", or use javascript to extract the username from a cookie or from the server.  That would mean that the server wouldn't have to generate a new copy of the page for each user. 
• Use Location="Any" so that pages can be cached on the client as well as the server, to reduce network bandwidth.

Hope that makes sense...

--Dean