csshandler.ashx can be expensive on the first hit because it is combining a bunch of css files and then minifying the combined css. It gets cached on the server and in the browser so subsequent requests should be fast.
I would also review the mojoportal log under Administration > System Log
If you see the application start and end events logged repeatedly in short times it means the application pool is being recycled frequently. This can happen in a low memory hosting environment if the host forces the app pool to recycle when it reaches a certain memory limit. If the app is continually restarting it is usually an indicator that that is happening and this will kill performance.
For the css handler problem there is a way that you could remove it from the load equation. Copy the rendered minified css output from csshandler.ashx into a new file named style-combined.css and put it in your skin folder.
Then in the style.config file in the skin folder comment out all the files that are there and add one entry for the new combined file.
Then add this in user.config:
<add key="CombineCSS" value="false"/>
recycle the app or touch web.config to reload the above setting.
Now if you view the source it should link directly to the style-combined.css file instead of to csshandler.ashx
That will reduce the memory usage a little as well so it may help with other problems.
Hope that helps,