For your situation where your feed manager is likely the only consumer of the blog feed, I would set the blog cache duration to 0 to turn off caching of the outbound feed. Then the feed manager entry cache settings will be the only ones to be concerned with.
Basically the feed manager caches the data from the feeds in the db. This caching first occurs when the the page withe the feed manager is requested, the feed manager gets the feed data caches it in the db, then on subsequent requests for the page with the feed manager if the cache has not expired it will just serve the content from its cache, but on the first page request after the entry cache duration it will refresh the data from the feeds into the cache.
For public facing blog rss feeds its best to cache the outbound feed as well because there may be a lot of requests from different consumers, ie feed managers/aggregators from other sites.
Hope it helps,